Duolingo Introduces AI-Powered Code Review Automation

Duolingo, a language learning service provider, is working on automating code review using AI. Sarah Deitke, an engineering leader at the company, has disclosed the specific methods and results externally. The key to this initiative is not only implementing tools but also emphasizing the creation of a culture that embeds AI throughout the organization.

As context, many companies face a challenge where AI tools are adopted but underutilized by engineers on the ground. Simply providing tools often results in engineers continuing their existing work procedures unchanged. Duolingo has adopted a strategy of combining tool development with efforts to increase employee understanding and trust in AI.

As concrete measures, Duolingo conducted workshops within the company aimed at improving AI literacy. Additionally, the company established an 'observability dashboard' that allows engineers themselves to see how AI operates, and created an environment for visualizing AI decision-making. Through these efforts, the company is building a system in which engineers can understand AI's behavior and apply it to their work.

In the code review domain, Duolingo introduced a bot that automatically assesses the risk of pull requests (PRs). A pull request is a unit of work in which an engineer submits a code change and requests confirmation from other team members—a task that occurs daily in software development. By having this bot automatically identify high-risk changes, it becomes easier to prioritize review work, and review efficiency increases.

According to Deitke, by combining developer education with AI safety measures (guardrails), the company achieved both improved development speed and maintained defect rates simultaneously. While concerns persist in the field that AI adoption might increase defects, this case demonstrates that both are achievable.

An important point demonstrated by this initiative is that the effectiveness of AI adoption is not determined solely by tool performance. Whether engineers accept AI as 'trustworthy' directly influences the degree of actual adoption. Duolingo's focus on education and observability stems from the recognition that building trust is the starting point.

As AI adoption advances in development environments, the question of how to combine tool implementation with talent development will remain a challenge going forward. Duolingo's case serves as one example of a comprehensive approach that includes organizational culture transformation, not just technology, and may be instructive for other companies. The track record in code review—a process at the core of development—is also noteworthy when exploring how far the scope of AI utilization can be extended.