Financial services firms are being encouraged to confidentially report artificial intelligence (AI) incidents and near misses, according to a proposal from HM Treasury. The initiative aims to create a voluntary repository for these reports, allowing the industry to learn collectively from AI deployments.
The proposal draws inspiration from the aviation sector, which has used a similar system for decades. NASA's Aviation Safety Reporting System, now 50 years old, has gathered more than 2.3 million confidential reports, which are then analysed and shared with the wider aviation community.
Currently, 75% of financial services firms are reportedly using AI in at least one business function. The new repository seeks to capture lessons from these implementations, including areas where AI performs poorly, where human judgement was necessary, and what controls prevented mistakes.
The proposal suggests that making the repository confidential would reduce the risk of repercussions for firms. Research indicates that factors like fear and embarrassment can deter firms from admitting mistakes, even when no harm has occurred.