By now it is clear that using AI can do wonders for productivity and quality of work. However, a study by a team from Harvard, Wharton, Warwick, MIT Sloan, and BCG shows that using AI for the wrong tasks can reduce quality and productivity. That requires leaders to define when and how to use AI and create KPIs to measure success objectively.

The research, titled “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality,” involved 758 consultants in a controlled experiment. Participants were randomly assigned to work with or without AI assistance (GPT-4) on a set of 18 tasks.

The key takeaway? 

AI shines when handling tasks “within its frontier.” Consultants using AI for these tasks saw a significant boost in:

• Productivity: Completing 12.2% more tasks on average and finishing them 25.1% faster.
• Quality: Delivering work with over 40% higher quality compared to the control group.

This benefit extended across all skill levels, with less experienced consultants experiencing an even more dramatic improvement.

However, the study also revealed a crucial caveat. When presented with a task outside AI’s current capabilities, consultants using AI were 19% less likely to find the correct solution than those without AI.

This study highlights the concept of the “jagged technological frontier” – where AI excels in specific areas but stumbles in others. The ability to navigate this frontier skillfully is critical to maximizing AI’s potential. Moreover, with the frontier being jagged, it’s not always clear which tasks fall within and which fall outside.

Here’s what leaders should do: 

1. Develop clear guidelines – Establish a framework for when and when not to leverage AI. Update these guidelines regularly as AI capabilities evolve.
2. Invest in training – the same research showed that consultants who received prompt engineering training performed even better. 
3. Measure success – Set KPIs (key performance indicators) around AI usage. This seems obvious but another research shows that surprisingly, 60% of companies  lack them.