Introduction
The NSW Government has a comprehensive and structured approach to ensuring the safe and responsible use of artificial intelligence (AI) including AI governance and oversight, risk management and mitigation, ethics, capability and capacity building, and continuous improvement.
In May 2024, the NSW Government introduced a directive for the use of AI (DCS-2024-04 - Use of Artificial Intelligence by NSW Government Agencies). This directive requires that all uses of AI must manage risk through use of the NSW Artificial Intelligence Assessment Framework (AIAF), a self-assessment tool aligned with the NSW AI Ethics Policy. This framework is designed to enable agencies to evaluate the risk associated with their AI solution and understand the required mitigations.
When do I need to complete the AIAF?
The AIAF is to be used across all project stages and the solution lifecycle, emphasising continuous adherence to ethical principles. It calls for ongoing assessments from project inception to operations, highlighting the importance of understanding AI's limitations and risks, especially regarding data security and privacy. Examples of the milestones requiring an AIAF review include:
- Designing: it is important to assess any new AI solutions at the conclusion of the design phase to ensure you identify, understand and mitigate as many risks as possible before developing or procuring a solution. This review may result in your solution requiring a redesign to address risks or a more detailed assessment and review being conducted.
- Developing: once your approved design has been developed, it should be reviewed again before the testing phase. This review is to ensure that no new risk has been introduced during the development to work around technical limitations or due to a change in requirements.
- Deploying: when testing is complete on your developed solution, it should be reviewed once more to ensure that any fixes or changes applied during the testing phase did not impact the overall residual risk.
- Procuring: when procuring off-the-shelf or bespoke AI solutions, it is especially important to conduct an AIAF review as part of the procurement process prior to making any commitments. AI solutions commonly use data and content to 'train' the solution for ongoing improvement. Suitability of solutions will be dependent on the classification of the data and information accessible to the AI solution.
- Using: any new or existing solution which has AI components or capabilities requires assessment against the AIAF regardless of any prior security or software approval. Any cloud-based products such as Miro or Jira should have their AI components disabled until they can be assessed. As software products can be used in a variety of different ways, approval or software for one use case does not mean it is approved for all use cases. View the process for enabling AI features in software.
- Ongoing: given the rapid evolution and growth of AI, it is recommended that AI products are reviewed every year to ensure new or evolved features do not change the residual risk of using the solution.
What does the AIAF assess?
The AIAF is a mandatory Excel based risk management tool which evaluates proposed AI solutions against the 5 AI principles defined by NSW Government:
-
Community benefit
AI must prioritise community outcomes, ensuring alignment with laws, minimising harm, and maximising benefit.
Government must prioritise the wellbeing and interests of the community. AI must be the most appropriate solution for a service delivery or policy problem, aligning with government priorities, complying with laws and regulations, and balancing risk against community value. Careful evaluation should assess benefit against potential harm to individuals, communities, and the environment, along with the degree of reversibility and impact
-
Fairness
Use of AI will be fair, ensuring not to perpetuate bias and inequality by leveraging diverse representative datasets, monitoring performance, and using rigorous data governance.
The Fairness principle emphasises equitable AI, where decisions made by or with the assistance of AI do not perpetuate bias or inequality. It demands rigorous evaluation and management of data quality, advocating for diverse and representative datasets. AI systems must be designed to avoid unfairness, with strategies to detect and correct biases, ensuring fairness for all segments of society.
-
Privacy and Security
Ensure secure, transparent, and compliant data use to preserve public trust.
NSW citizens need assurance of safe, secure and privacy-compliant data use. Transparent review mechanisms and community engagement are essential. Explicit consent, cybersecurity compliance, and privacy legislation adherence are vital. Any project outcome will be undermined if there is a risk of data breaches or compromised personal data, eroding public trust.
-
Transparency
The use of AI will be transparent to the people it could impact, privacy preserving, cyber secure and ethical, and provide review mechanisms that allow concerns to be raised and addressed.
Transparency fosters public trust and accountability by ensuring community consultation, awareness of AI use, and the ability for individuals to seek explanations and challenge decisions that impact them directly, unless there is an overriding public interest in not doing so. The development of AI systems must be compliant with relevant legislation, cyber security policies and ethical.
-
Decision making
Decision making remains the responsibility of organisations and individuals.
Despite AI's autonomy, humans hold ultimate decision responsibility needing skilled operators with clear accountabilities. Establishing responsible parties across data, technology, models, and outcomes is crucial. Operators evaluate data inputs and outputs, understanding system limitations and model performance. The ability to identify and reverse AI decisions to prevent harm and apply human oversight to prevent over-reliance ensuring continuous review.
Complete the assessment
What to do with completed assessments
Once you complete an assessment, it will need to be reviewed by subject matter experts before you can proceed to the next phase of your project. To request a review, email a copy of your completed submission to aiaf@dphi.nsw.gov.au.
During the review, subject matter experts will review your submission against best practise and NSW Government policies. The review will be conducted by relevant groups, including:
- Governance
- Risk
- Ethics
- Legal
- Technology
- Cyber Security
- Records
- Procurement (if procuring a solution or a component in the proposed solution)
High risk AI solutions
If the outcome of your AIAF submission highlights the outstanding risk is rated 'High' or greater, the submission will need to be reviewed by the AI Review Committee in the Department of Customer Service (DCS). These submissions should still be submitted to the Digital Experience and Solutions team for evaluation and coordination with the AI Review Committee in DCS.