AI Governance Briefing · Part 1 of 3
A frontline support employee asks the company’s AI assistant to explain the refund policy. The assistant retrieves the current policy, an expired version, and a restricted fraud memo that someone shared with a broad Drive group two years ago. It blends the three into a confident answer. The employee sends the reply to a customer.
That incident starts with an old access decision and ends with a new one. The source system made the memo available. The connector made it easier to find. The business workflow allowed an employee to act on the answer. Fixing the problem takes three owners: the person responsible for the source, the team that configures the assistant, and the manager who owns the customer communication.
Enterprise chat approval should cover those decisions explicitly. Turning on a workspace, connecting internal data, and granting permission to take actions each change what the organization is authorizing.
Six risks to manage before rollout
Risk | What the impact could be | How to manage it |
|---|---|---|
Sensitive data enters the wrong workspace. An employee pastes a customer file into a personal account or uploads a restricted record to managed chat. | The record may be stored or accessible under conditions the company never approved. Investigators may have limited visibility into what was shared. | Define permitted data classes and give staff concrete examples. Make managed accounts easy to identify. Where supported, enforce upload restrictions or data loss prevention and test them with synthetic records. |
A connector surfaces overshared content. A document is technically accessible to a broad group, even though its owner intended it for a small team. | Employees can discover sensitive material they would rarely encounter through ordinary search. The answer may reproduce excerpts in a new location. | Start with a small set of approved sources. Have source owners check access groups and remove broad grants. Test retrieval as a standard employee and as a user who should be denied. |
A document or email tries to steer the assistant. Retrieved content tells the assistant to fetch another file or forward information. | The assistant may disclose unrelated data or attempt an action the user never intended. | Start with read-only connectors. Restrict tools and destinations. Enforce authorization in the connected application and test with a document that attempts to trigger a prohibited action. |
A plausible answer is wrong. The assistant cites an expired policy, omits an exception, or answers with incomplete evidence. | Customers receive incorrect terms, staff handle similar cases inconsistently, and an inaccurate answer can become part of a decision record. | Name the authoritative source and the reviewer for consequential outputs. Include stale, conflicting, and missing source material in pilot tests. Give staff a clear way to escalate uncertain cases. |
A generated answer is shared too widely. A user shares a conversation link or pastes an internal summary into an external channel. | Sensitive facts can leave the approved audience even when the source document remains protected. | Restrict workspace sharing where supported. Apply existing data classification and external review rules to generated summaries. Test sharing separately from retrieval. |
Records persist longer than expected, or evidence expires too soon. Chats, attachments, and audit exports follow different schedules. | Sensitive data remains available longer than intended, or responders lack the evidence to reconstruct an incident. | Map each record type, who can access it, and how long it is kept. Verify deletion and audit access against the workspace settings and the features in use. |
NIST’s Generative AI Profile covers data privacy, information security, and confidently wrong output. The UK National Cyber Security Centre describes how malicious instructions can reach an assistant through the content it retrieves. NIST Generative AI Profile · NCSC guidance on adversarial attacks
Where the products change the test
Keep the baseline consistent across providers, then check how each workspace enforces it. In ChatGPT Enterprise, test which groups can use each enabled app. In Claude Enterprise, also check which tools a role can use within a connector. For Gemini with a managed Workspace account, test both Gemini’s access to Workspace data and the permissions on the underlying Drive files. In Grok Enterprise, test the widest audience a standard user can select under the team’s Product Sharing policy. These checks cover different points where access or sharing can expand. OpenAI app controls · Anthropic custom roles · Google Workspace data access controls · Grok team management
Give each decision an owner
The AI Governance Council sets the default and the exception process. Define the permitted data classes, connector types, sharing audiences, and uses that require human review. Delegate routine approvals. Escalate requests for sensitive data, write actions, external sharing, or use in consequential decisions to the authority named in the council charter.
The CISO sets the security baseline and verifies it. Require evidence that identity, source permissions, sharing restrictions, and action limits work under ordinary user roles. Give Security Operations and IT a way to disable an affected connector or capability during an incident.
IT and source system owners enforce the configuration. IT owns workspace settings and access groups. Source owners fix permissions in the underlying systems. Privacy and Records Management define requirements for sensitive data, retention, and access to evidence.
The business owner owns the outcome. Name the authoritative policy, the person who checks consequential outputs, and the conditions that pause the workflow. For refund replies, test whether support staff can spot an expired policy and know where to escalate an exception.
Figure 1: An enterprise chat approval starts with three questions: what can the workspace access, what can users share or act on, and who owns the outcome? Test those limits in the product your team uses, record the approved scope and who can suspend access, and retest when data, permissions, or features change.
Apply the frameworks to one rollout decision
Suppose Support requests a read-only connector to its policy library. Use ISO/IEC 42001 to put that request into a repeatable approval process with a named owner, documented scope, and review triggers. Apply your existing ISO/IEC 27001 access controls to the connected library. Use NIST AI RMF to turn the request into four concrete steps. ISO/IEC 42001 · ISO/IEC 27001 · NIST AI RMF
Govern. The council defines the default for read-only connectors. The business owner requests access. The CISO defines the evidence needed before enablement.
Map. The source owner lists the folders, employee groups, and restricted files. IT records the connector identity, workspace settings, and available sharing actions.
Measure. Security places a harmless marker in an approved policy and another in a restricted memo. The pilot user should retrieve the first and fail to retrieve the second. Security then attempts external sharing and tests whether an instruction in a document can trigger a prohibited action.
Manage. If the restricted marker appears, keep the connector disabled for the affected users until the source owner fixes access and Security retests. Record the approval, the person who can suspend it, and the changes that require another test.
The OECD AI Principles add a question about the person on the receiving end. If an inaccurate AI-assisted reply affects a customer’s refund, can they get it corrected? The business owner should be able to show who handles the challenge, which policy they consult, and how they correct the case. OECD AI Principles
For your next council meeting: Pick one live chat connector. Ask its owner to show what an ordinary employee can retrieve, what they can share, and who can suspend the connector. Then ask the business owner to show what happens when the answer is wrong. Record any missing evidence with an owner and a due date.
Need help putting AI governance into practice?
Reply to this email or fill out this form here and tell us what your organization is working through. Whether you’re rolling out enterprise chat, assigning responsibilities across Security and business teams, or deciding what evidence to require before launch, share the decision you’re facing and where you’re stuck. We can help you identify a practical next step.
Next issue: Who controls what a coding agent can do?
A coding agent can read a repository, run commands, and propose changes. What happens when it inherits a developer’s credentials or follows an instruction hidden in an issue? We’ll cover what Security should test, what Engineering should enforce, and which requests for broader access should go to the AI Governance Council.
Disclaimer. AI Governance Briefing provides general educational information. This newsletter does not constitute legal or regulatory advice, a determination of compliance, or a certification of conformity with any standard. Assess any guidance against your organization’s circumstances and consult qualified advisers as appropriate. If you reply to this email, please omit confidential, proprietary, or personal information and describe your situation in general terms.