AI in healthcare teams
Can practice staff put patient information into ChatGPT or Copilot?
Not into a publicly available chatbot, and the same goes for participant information in an NDIS service. As a matter of best practice, the OAIC recommends that organisations do not enter personal information, and particularly sensitive information, into publicly available AI chatbots, and the RACGP tells GPs they should never enter sensitive or identifying data into a conversational AI tool. Tasks with no personal information in them, such as summarising a policy, are the lower-risk end, but they still need a tool your practice allows and a person who checks the output; de-identified information sits in between, because the OAIC warns it can still be re-identified. Whether a business plan, or a tool such as Copilot inside your own systems, changes the answer depends on its terms and settings, and that is for your practice AI policy and your privacy adviser to settle, not for a staff member on the day.
Can staff use ChatGPT or Copilot at work at all?
Neither the OAIC nor the RACGP says staff must never use them. They set conditions, and leave the choice of tools and tasks to the practice or service.
The OAIC's guidance on commercially available AI products names ChatGPT and Microsoft Copilot among the products that incorporate large language models. It says it also addresses AI products that are freely available, such as publicly accessible AI chatbots. It says a number of uses of AI are low-risk, but that using personal information in AI systems may, depending on the use case, be a high privacy risk activity, and it expects organisations to take a cautious approach to that. It adds that AI products "should not be used simply because they are available". It notes that AI in the healthcare sector may pose particular privacy risks because of the sensitivity of health information.
The RACGP's conversational AI page names ChatGPT and Microsoft Copilot among its examples of conversational AI. It says GPs "should be extremely careful" in using conversational AI in their practice at this time, and must ensure their use complies with any practice policies that "might impact, prohibit or govern its use". That page is written for GPs. The OAIC guidance is written for any organisation using AI systems that handle personal information, including for internal purposes, so its questions are as relevant to reception, administration and support staff as to GPs.
So the first question for staff is not which tool, but what the practice or service has approved. This page covers everyday administrative tasks, not clinical use.
Where should staff stop and check before using ChatGPT or Copilot?
Look at the information before the tool. The table crosses five everyday tasks staff might use ChatGPT or Copilot for with the kind of information involved, and shows where the OAIC guidance says to stop and check. It is an illustration of that guidance, not a ruling on any tool, plan or setting.
- Check the output: no personal information goes in. The tool must still be one your practice allows, and a person reads the result before it is used.
- Stop and check: could it identify someone? Names are gone, but the details left in may still point to a person.
- Stop and check: is this tool approved for it? Personal information would go in. As best practice, the OAIC recommends not entering it into a publicly available AI chatbot. Use only a tool your practice has assessed for this information, and only as your policy allows.
| Task | No personal information | De-identified | Identifiable |
|---|---|---|---|
| Drafting a letter to a patient or participant | Check the output. A general template, with names and details added afterwards outside the tool. | Stop and check: could it identify someone? A letter about one person's care or support can point to them without a name. | Stop and check: is this tool approved for it? Names and contact details are personal information, and health details are sensitive information. |
| Summarising a policy or guideline | Check the output. Compare the summary with the original before anyone relies on it. | Stop and check: could it identify someone? Worked cases or incident details in the document may still point to a person. | Stop and check: is this tool approved for it? Take the names and details out first, or keep the document out of the tool. |
| Rewriting a rostering message | Check the output. A general shift notice with no names in it. | Stop and check: could it identify someone? Initials, suburbs and shift times together may point to a staff member or participant. | Stop and check: is this tool approved for it? Staff names, and participant names and addresses, are personal information. |
| Replying to an email from a patient, participant or family member | Check the output. A general reply, such as opening hours, written without pasting in their email. | Stop and check: could it identify someone? The details in an email can identify the sender even with the name removed. | Stop and check: is this tool approved for it? Their email is personal information and may include health information. |
| Writing up notes from a team meeting | Check the output. Notes about rosters, equipment or training, with no one named. | Stop and check: could it identify someone? A case discussion can identify a person from the details alone. | Stop and check: is this tool approved for it? Discussion of a patient or participant is personal information and may include health information. |
Is ChatGPT fine to use if no personal information goes in?
Two checks remain: the tool must be one your practice allows, and a person must read the output before it is used. The OAIC says AI systems are known to produce inaccurate or false results, and that the risk is especially high with generative AI. It gives an example of a chatbot asked to generate fictional case studies that produced a real scenario, including the full names of the people involved. Privacy obligations apply to output that contains personal information, as well as to what goes in.
The OAIC also gives an example of an AI assistant that produces more errors over time as its underlying processes and training data become outdated, and warns that staff may come to over-rely on it. The RACGP says GPs should always check AI outputs, and that the need to check them comprehensively might partially negate the time saved.
Is taking the patient's name out enough?
Not always. The OAIC says personal information includes information that can reasonably be linked with other information to identify someone, and that information entered into AI systems may be at risk of re-identification even when it has been de-identified or anonymised.
The OAIC also expects organisations to consider what information is necessary, and whether there are ways to minimise the amount of personal information that goes into an AI product. This page does not set out a de-identification method. If staff need one, ask your privacy adviser to help you write it into your policy.
What if a task needs patient information, such as names or health details?
Not into a publicly available chatbot. The OAIC's practical tips say: "as a matter of best practice it is recommended that organisations do not enter personal information, and particularly sensitive information, into publicly available AI chatbots." Any other tool should be used only if your practice has assessed it for this information and your policy allows it. The OAIC describes health information as a kind of sensitive information, and says organisations must take particular care with sensitive information, which generally requires consent to be handled.
The OAIC's own example is staff at an insurance company who enter a customer's claim, including health information, into a publicly available chatbot. It says entering it discloses the information to the chatbot's owners, and that disclosure must comply with APP 6. The OAIC says that once personal information has been entered into AI systems, particularly generative AI products, it will be very difficult to track or control how it is used, and potentially impossible to remove. For any tool, it says to proceed cautiously before using personal information in an AI prompt or query, and only do so if you can comply with APP 6.
Avant, a medical defence organisation, in When AI enters your practice, recommends that practices have a clear, written policy on which AI tools staff can use and the categories of information that can and can't be entered. It says patient identifiable information and clinical details should never be entered into AI tools which are not approved for healthcare use.
Does a business plan, or Copilot inside our own systems, change the answer?
It can, but only your practice or service can decide whether it does, after checking the tool's terms and settings. Neither the OAIC nor the RACGP rates any plan, licence or setting, and this page does not either. The OAIC says to look at:
- whether the service terms give the developer access to data your organisation enters or generates;
- whether the terms or settings let the product owner collect what you enter for further training and development of AI;
- whether a third party receives personal information through the product's features, such as an interface with a search engine;
- if the tool is integrated into your systems with access to your documents, whether that happens in a secure environment on your premises or through the cloud, where the servers are, and whether personal information could be disclosed outside Australia.
The OAIC says that where a proprietary system has protections to ensure information entered is not disclosed outside the organisation, such as to the developer, entering it is a use rather than a disclosure. A use must still comply with APP 6. The measures the OAIC lists include turning off features that would disclose personal information, controls that stop it being entered, and prohibiting staff from entering it. It says any such controls or prohibitions "will need to be accompanied by robust training and auditing measures".
The RACGP says GPs should always check a tool's service terms, such as how it manages information and its indemnity clauses, before using it in practice. Avant notes that Microsoft 365 and Google Workspace now include AI features that may be active by default, and recommends reviewing what has been enabled. Your privacy adviser can help you read the terms against your obligations.
What should the practice or service put in place?
A written AI policy, training on it, and a regular look at which tools are actually in use. Among the processes an organisation should have, the OAIC lists internal policies that clearly define the permitted and prohibited uses of AI, and training staff on the specific product, including its limitations and risks and the permitted and prohibited uses. Avant recommends reinforcing the policy with staff regularly, including locums and temporary staff, and auditing which AI tools are actually in use, not just which ones have been officially sanctioned.
For a general practice, RACGP criterion F11.A says the practice "ensures that identified patient data is not used by AI tools unless its use is clinically necessary, explicitly authorised, and supported by documented governance and consent processes".
For NDIS providers, F11 is a general practice standard, so work from the OAIC guidance. The OAIC says, "If your organisation is covered by the Privacy Act, you will need to understand your obligations under the APPs when using AI." Whether it covers your service is a question for your privacy adviser. For the wider picture, see the AI in healthcare teams hub.
What's changing?
- 10 December 2026, OAIC. The OAIC's news item of 30 September 2026 says that from this date, APP entities must include information in their privacy policy where they have arranged for a computer program to make or inform decisions that significantly affect individuals' rights or interests. Whether any tool your team uses falls within it is worked through in automated decision-making and your privacy policy.
- 26 August 2026, RACGP. The RACGP released the sixth edition of its Standards for general practices, with artificial intelligence among its new and updated topics. It includes F11 on artificial intelligence, whose first criterion begins "Where the practice uses artificial intelligence". The release says transition and implementation arrangements will be published on the Australian Commission on Safety and Quality in Health Care's website.
The RACGP conversational AI page shows two dates: "Page last updated 29 June 2026" in its header, and "Last updated: September 2025" in its publication information. Check the current page before relying on this summary.
This page is general information, not legal advice. It summarises OAIC, RACGP and Avant guidance on staff using ChatGPT, Copilot and other AI chatbots, and does not say whether any tool, plan, licence or setting is safe or compliant for health information. Confirm your obligations with the OAIC or your own adviser: your privacy adviser, lawyer or medical defence organisation.
How we help: Make informed decisions about AI
Sources
- OAIC, Guidance on privacy and the use of commercially available AI products (Published 21 Oct 2024; updated 17 Jan 2025)
- RACGP, Conversational artificial intelligence (AI) (Header 29 Jun 2026; publication info: first published Jun 2025, updated Sep 2025)
- Avant, When AI enters your practice (10 Aug 2026; first published in Connect magazine issue 26, May 2026)
- RACGP, F11 – Artificial intelligence (AI), Standards for general practices (6th edition) (Undated)
- RACGP, New Standards for general practices put the focus on patients while supporting GPs with 'continuity amid change' (26 Aug 2026)
- OAIC, New resources on transparency for use of AI and automated decision-making (30 Sep 2026)
A useful place to start
Better technology.
Better care, together.
Tell us about your team and what you want to make better. We’ll find a useful place to begin.