By Valentyna Jurkiw, Special Counsel

AI is subject to good governance and is not merely something for the IT department.

As with all technological advances, the legal tenets of corporate responsibility and the duties and responsibilities of officers and directors continue to apply. But what are the particulars in a charity context now that AI has become a part of ordinary organisational conduct?The ACNC has published guidance on the matter against the background of Australia’s evolving regulatory approach to AI. It points particularly to Governance Standard 5 and the duties of responsible persons and the importance of maintaining public trust and confidence as an asset of a charity.

Adoption of AI within an organisation calls for deliberate governance. This includes considering where AI use may be appropriate, to what degree, how its use should be disclosed, who is accountable and the risks that arise from particular applications of technology.

Not all AI use carries the same risk and the approach to the use should be proportionate to the possible consequences. Using AI to brainstorm a newsletter idea is quite different from using it to assess a person’s eligibility for charitable assistance. Applications involving beneficiaries, sensitive information, employment decisions or the allocation of services should attract greater scrutiny, testing and approval than routine administrative uses.

Privacy and confidentiality require particular attention. Charities may hold sensitive information concerning a person’s health, disability, financial hardship or other personal circumstances. Directors themselves should also exercise care. Uploading confidential board papers to an AI platform to obtain a summary, for example, may create issues concerning confidentiality, privacy and data security.

Meaningful human oversight is equally important. AI can assist with analysis and recommendations but should never displace the engagement of a person. However, a person cannot meaningfully oversee an AI-generated recommendation if they do not understand its limitations or are unable to critically assess its output.

In many organisations, AI use may initially have been adopted informally by staff seeking efficiencies or additional support in an under resourced environment. What begins as individual or operational use can become an organisational governance issue as its use becomes more widespread or consequential.

This requires directors to develop a degree of AI literacy and respond to the growing adoption of AI by implementing an AI governance framework which will assist them to identify material risks, challenge management and satisfy themselves that appropriate controls are operating. A governance framework will also provide staff with the literacy they need to use AI safely and effectively.

A charity board might start by asking:

1. Where is AI currently being used in the organisation?
2. What information is being provided to AI systems?
3. Which uses could materially affect beneficiaries, employees or donors?
4. Which decisions must remain subject to human judgment?
5. How are privacy, confidentiality, cybersecurity and intellectual property being protected?
6. How are important AI outputs checked for accuracy and bias?
7. Who is accountable for AI governance at management level?
8. What AI-related matters should be reported to the board?

Ultimately, AI governance is an extension of good governance. The technology may change rapidly, but responsibility does not. Boards remain accountable for ensuring that AI is used consistently with the charity’s purposes, legal obligations and the public trust upon which the sector depends.