AI Audit
An independent review of your existing AI system, with actionable recommendations.
AI Audits are used to identify and mitigate risks in AI systems. These risks can occur internally within a company that uses AI or externally, in the case of an acquirer performing due diligence on an AI-driven target. AI systems carry specific risks unaddressed by a software audit because they are a function of not only code, but also data and the parameters used to combine them.
An overlooked benefit of an AI audit is to propagate industry-accepted best practices. The rapid evolution of Machine Learning means that AI systems can look very different from one company to the next.
What to Expect
In a Machine Learning audit, the evidence comes from interviewing key stakeholders and reading documentation, such as design documents. Additionally, auditors can review source code and run specific tests to evaluate model fairness and robustness if required.
The audit result is usually written up in a 15–20 page report. While the report will include observations and rank conformity of controls, the actionable part is the Recommendations section. Neural Swarm will make recommendations for each objective with major or minor non-conformities. Depending on the complexity and number of models, the process can take around two weeks.
Neural Swarm AI Audit Framework
Our proprietary AI audit framework consists of 85 controls covering 6 categories.
Determine that AI adds business value and that the AI system is feasible with clear requirements.
Ensure that data is suitably collected and that the data pipeline is created and tested correctly.
Check that suitable features and models are used, that training is optimised and that the model is correctly evaluated and tested.
Enforce MLOps best practices, including model monitoring and continual training.
Verify that the model is fair, explainable, compact and robust.
Certify that the system follows software engineering best practices around source code, documentation, version control and testing.
Get in Touch
Thank you for your interest in Neural Swarm. Please fill out the form below to enquire about our work, and we’ll respond as soon as possible.