The Department of National Defence received 71 proposals for a research competition to build systems that would measure the trust military operators place in artificial intelligence, the department told Vanguard Defence, with funding decisions expected this month.

The competition runs through the Innovation for Defence Excellence and Security (IDEaS) program and asks researchers to build systems that can track an operator's trust in an autonomous system as a task unfolds, hold it at an appropriate level and restore it after the machine gets something wrong.

The challenge, titled "Cognition and Trust: Real-Time Dynamic Calibration for Human-Autonomy Teams," carries up to $1 million of funding per team against a total pool of roughly $4 million. Winning teams must be led by a Canadian university and will have funding provided until March 2028.

As the Canadian Armed Forces look to adopt autonomous systems, trust has to sit at the centre of the human, organizational and technological factors that go into the decisions those systems make.

"If we can accurately measure trust, we can design, develop, and deploy systems that help people trust AI appropriately. The goal is to build trust, maintain it, and adjust it when needed so operations remain effective," the DND spokesperson told Vanguard Defence.

The work is tied to Canada's NORAD Modernization Plan, which identified "autonomy and counter-autonomy" as priority research areas. The federal government's Defence Research and Development Canada (DRDC) office is "expanding its concepts of employment" to include human-machine teaming and supporting AI prototyping, the DND spokesperson said.

The federal government has already tapped IDEaS to measure trust in a defence setting. Ottawa-based Calian Group built a synthetic NORAD surveillance environment under an earlier IDEaS challenge, reproducing military voice protocols and aircraft behaviour to mimic the pressure of a live command-and-control shift.

The system involved two RCAF operators with NORAD experience who wore caps that recorded brain activity. Machine learning models converted those signals into indicators associated with trust, and the operators were then asked whether the readings matched what they felt.

While the research hasn't been field tested, Calian says the models performed as expected and that training its AI algorithm with a wider training set would have likely strengthened it.

"The research underscored the importance of understanding when operators accept, question or override machine-generated input, because those moments reveal how trust is calibrated in practice," said Chad Watson, Calian's director of growth and strategy for defence, who led the project.

Henry Leung, a professor of electrical and software engineering at the University of Calgary who builds sensor fusion systems under contract to DND, including work as a subcontractor to Calian, is not convinced the underlying science is ready.

"I'm not sure it's mature to the level that you think it can [confidently] detect people's trust level," said Leung, who hasn't submitted a proposal for this IDEaS challenge.

The pressure to automate comes from volume. Operators receive more data than they can use, Leung said, arriving in formats that do not align. Radar returns, satellite imagery, video, time-series data and open-source text all reach the same floor in different shapes, and AI systems generally require consistent inputs.

His work with the DND involves developing an AI system that includes uncertainty instead of issuing a decision, while testing the model repeatedly to make sure its answers hold.

"You can't just give us a classification," Leung said. "Is this a ship? Or is this a fish? How confident are they with the classification?"

As AI systems take on a greater role in operations, the question of where human judgment sits becomes more pressing. The DND spokesperson said humans matter most where the risk of harm is greatest, including applications involving lethal force, because it allows biased data to be challenged and a decision-maker to be held accountable.

"This will strengthen our ability to uphold Canada's domestic and international legal obligations, including those applicable in armed conflict," the DND spokesperson said.

Watson said the commitment to keeping a human involved in the use of force gets harder to hold as decision-support systems get faster, because the hard part is defining what "appropriate" means in a given context. Operators who have worked with reliable systems grow more comfortable delegating tasks where speed or volume exceeds what a person can manage, though accountability and clear rules of engagement still have to hold, he said.

"The objective is not to remove the human from critical decisions," Watson said.