Another Way to Look at Policy

Where an office assistant becomes high risk

The AI Act classifies a system by its purpose. An assistant drafting an email and the same assistant ranking applicants for a vacancy can run on the same model; only the second is high risk, because hiring is in Annex III. The annex lists the uses: recruitment, promotion and dismissal; credit, insurance and essential services; education and examinations; migration; law enforcement; justice; critical infrastructure.

Most companies never touch a frontier model directly. They meet it through the assistant built into their Microsoft 365 licence, which puts OpenAI’s and Anthropic’s models, and Microsoft’s own where it chooses to deploy them, over the company’s mail, files, chats and calendars under Microsoft’s name. In the Act’s terms, OpenAI, Anthropic and Microsoft provide the general-purpose models, Microsoft provides the system, and the company deploys it. That holds until someone in HR builds an agent in Copilot Studio to shortlist candidates, or finance builds one to flag customers for credit review. From that moment the company is the provider of a high-risk system under Article 25, with every provider obligation attached. Microsoft’s product terms place responsibility for how the service is used on the customer, which is the ordinary position and the one the Act takes too. The Commission’s draft classification guidelines of May 2026 add that a broadly marketed system may be presumed to carry high-risk purposes; a disclaimer in the terms does not lift the presumption.

The first piece of work is an inventory: every agent and every standing use of the assistant, its purpose written in a paragraph, checked against Annex III and the Article 6(3) exemptions. The agents built by business units without IT are where the surprises are.

What the Act requires of a high-risk use

Risk management covering intended use and reasonably foreseeable misuse (Article 9), governance of training data (10), technical documentation (11), automatic logging (12), instructions for the deployer (13), human oversight able to override and stop the system (14), accuracy, robustness and cybersecurity (15), a quality management system (17), registration in the EU database (49), and for banks, insurers and public bodies a fundamental rights impact assessment (27). Deployers must use the system as instructed, assign oversight to competent staff, check input data, keep logs for at least six months and inform affected workers (26).

None of this applies yet. Regulation 2026/1744, the Digital Omnibus on AI, moved the Annex III date from 2 August 2026 to 2 December 2027. Two new prohibitions apply from this December; the high-risk regime waits a year longer. The date is fixed, the standards are unfinished, and in most tenants the thirteen months are being spent on licence counts.

The gap the Act leaves open

The Act has no word for what the assistant does all day. Its obligations cover what a system was built for, what it learned from, and whether a human can stop it. The assistant was months old when the text was finalised, and it sits in no annex.

It reads with the user’s permissions through Microsoft Graph: Exchange, SharePoint, OneDrive, Teams, the calendar. For the chief executive’s own account, and every delegate and agent running under it, that reach spans finance, HR, legal and the diary. Across five legitimate reads it learns that cash reserves are falling, that several executives are being replaced, that a major programme has stopped, that legal holds unusual transaction papers, and that the chief executive keeps meeting the same outside firm. No document in the tenant states that a division is being sold under financial pressure. The assistant can state it, in a summary, a draft or a reply to the wrong question. If an agent built in Copilot Studio is given actions and runs with that user’s credentials, it can act on it.

Microsoft will say this is the customer’s permission model. It is. The assistant is the first reader that exercises all of it in one query.

Microsoft’s safeguards are real, and they address the provider side. The assistant “only surfaces organizational data to which individual users have at least view permissions”, and “prompts, responses, and data accessed through Microsoft Graph aren’t used to train foundation LLMs” [1]. For EU customers it is an EU Data Boundary service, with one exception Microsoft states in the same document: Anthropic’s models, offered since September 2025 as an option in the Researcher agent and in agent building, “are currently excluded from the EU Data Boundary” and run outside Microsoft-managed environments under Anthropic’s terms [1][2]. One checkbox in the admin centre changes the data-residency answer a board has been given.

Microsoft’s controls act per item: a label on a file, a flag on a site, a policy on a connector. Each keeps one source out. None evaluates what the remaining sources add up to. Microsoft’s own deployment blueprint assumes the problem: step one, before any licence is assigned, is finding “sites and files that are overshared, ownerless, inactive, or contain sensitive data that could be surfaced by Copilot” [3]. In June 2025 EchoLeak, CVE-2025-32711, rated critical at 9.3, showed what the join does under attack: a crafted email, no click from the user, and the assistant’s retrieval mixed the attacker’s text with internal content and leaked it through Teams and SharePoint links. Microsoft fixed it server-side, reported no exploitation in the wild and has since added prompt-injection defences [4]. The lesson stands: the exfiltration used the assistant’s own permitted retrieval.

No article of the AI Act addresses the combination. GDPR’s purpose limitation covers it where personal data is involved; for the company’s own strategy there is NIS2 and DORA’s general duty to control access and the Trade Secrets Directive’s demand for reasonable steps. Cambridge Analytica showed the pattern in 2018, lawful pieces assembled into harm, and Europe answered by regulating collection. The assistant assembles inside the tenant. Zero trust and Conditional Access remain important; they decide whether an identity reaches a resource. Purview can show the interactions afterwards; that is evidence, and it arrives after the summary has been read.