The Queensland Audit Office is leading the way on analysing large amounts of complex audit data to unlock insights into government agencies.
Anthony Close and his colleagues at the Queensland Audit Office (QAO) knew something was up when a chart mapping the time patients spent waiting at the state’s hospital emergency departments suddenly spiked.
“It was like the data was hitting a brick wall,” recalls Close, acting Queensland auditor-general.
Is data analytics fundamentally transforming the audit? Listen now.
QAO officers were auditing the performance of Queensland hospitals after emergency departments were set a target for 90 per cent of their patients to have moved on within four hours of arrival.
Using advanced data analysis techniques, they found that before the four-hour target came into force, the time patients spent in hospital emergency departments followed “a nice orderly curve”.
However, after the target’s introduction there was a big build-up in the number of patients being processed from the two-hour mark, rising to a mad rush in the final few minutes before the four hours was up.
“At face value, hospitals appeared to be changing or manipulating the recorded time that patients were being admitted to wards,” Close says, and the audit team’s discovery prompted the Queensland Health Department to tighten controls on reporting and revising data.
More importantly, the results of the performance audit convinced health authorities to pause the introduction of the four-hour target and subject it to clinical review to ensure patient safety was not being compromised.
For Close and his colleagues it was gratifying confirmation that the leading-edge data analysis methods they had developed to improve their audits were worthwhile.
“It was quite a useful demonstration [of] the value to clients of unlocking that data out of their transactional systems,” he says.
Show me the audit data
In principle, the QAO operates much like any other auditor. It gathers data from its clients, analyses it, and reports on any anomalies.
However, instead of spending hours gathering data from different parts of an organisation and trying to match it up, the QAO audit process is far more efficient.
Clients lodge their data in a secure location, usually on a monthly basis, where it is retrieved by the QAO audit analytics team and fed into dedicated “dashboards” set up for each entity.
The dashboards, Close says, take vast amounts of data then aggregate and summarise it, giving auditors a wealth of information and the tools needed to interrogate it at their fingertips.
Is data analytics fundamentally transforming the audit?
“What we have found is the financial data is more easily able to inform us about the nature of our client’s transactions, to determine where the audit risk is present and where we need to direct our testing,” he says.
The breadth of the information on hand means that, in many cases, instead of relying on a random sample to do control testing, auditors can instantly check an entire population.
In some instances, the data is continuously monitored using computer-aided audit techniques built into the audit dashboards, providing early notice of any anomalies.
“That’s been a huge improvement,” says Close.
“Those red flags are now delivered straight to the financial auditor for investigation.”
It means that not only are QAO auditors able to work far more quickly and efficiently than in the past, they are able to provide clients with greater insights.
Close says the QAO is only just starting to “scratch the surface” of possibilities, such as giving clients feedback on how their performance compares against that of their competitors or peers.
“We have been able to do things like benchmarking, showing financial audit clients where they are positioned with organisations of similar size or characteristics, whether they are above, on or below benchmarks.”
How did we get here?
Close admits that getting to this point has not been easy.
“It’s certainly a journey not for the faint-hearted,” he says. “The rulebook has not been written in this space.”
The QAO’s IT systems had to be overhauled to ensure they were robust enough, and secure enough, to handle the workload and the organisation had to be prepared to accept inevitable missteps and setbacks.
“That is why we took an iterative and innovative approach, each time recognising that there was the potential to go down rabbit holes and learn from mistakes,” Close says.
“The old adage of failing fast and failing often was certainly one we were used to.”
Having said that, the acting Queensland auditor-general sees no reason why other auditors, including in the private sector, should not embrace the potential of what he calls audit analytics.
“It’s certainly a journey not for the faint-hearted,” he says. “The rulebook has not been written in this space.” Anthony Close, acting Queensland auditor-general.
He says developments in technology such as cloud computing mean that even smaller operators can have access to huge data storage capacity, while advances in data management mean “you do not have to feed the system anymore – you can pull out and aggregate entirely disparate systems in a cohesive way without a lot of effort”.
The QAO’s own audit data analytics team is, he says, very small.
Bringing together the necessary skills should not be a barrier, either. While the QAO is increasingly looking to recruit accounting graduates who also have a background in mathematics or science, existing staff are adapting to the use of audit data analytics.
They receive training in the use of audit analytics, and have access to “data champions” to help troubleshoot issues.
In the end, however, it is their auditing skills which remain the most important.
“The computer doesn’t answer it for you,” Close says.
“It’s not like it spits out the answer for you and you move on. It just gives you the ability to make a determination using the data.”
Data analytics for all?
While using audit data analytics in financial and performance audits can provide great insights for some organisations, they are less useful for others.
Of the hundreds of public sector clients on its books, the QAO currently judges that about 65 benefit from the use of audit analytics.
For each client, acting auditor-general Anthony Close and his team assessed how much time and effort would be saved using their data analytic methods, and what benefits and insights it might provide.
They decided that small and low-risk organisations such as Queensland’s water boards, which have water infrastructure assets and little else, would derive little benefit, while many government departments and entities would.
Ultimately, says Close, size is not the determinant. The QAO is looking to expand its use of audit analytics to examine assets and revenue systems, which will entail a big leap in complexity to encompass everything from student administration to energy trading systems.
Says Close: “It is really some leading edge capabilities that we are developing, which deliver significant benefits and value to our clients and to us. Our auditors are really enjoying the [new] capabilities they have.”
Turn insight into action with predictive analytics