The Hindu: How AI Powers Data Journalism & Investigative Reporting

The Hindu, a leading Indian newspaper, is quietly integrating large language models (LLMs) into its data journalism workflows, not to automate writing, but to dramatically expand the scale and speed of investigations. This shift, outlined by Srinivasan Ramani, Deputy National Editor and Senior Associate Editor at The Hindu, represents a strategic move to leverage artificial intelligence for complex data processing, code generation, and structuring investigations, rather than replacing journalistic judgment.

Recent projects demonstrate the breadth of this approach. Journalists at the publication parsed nearly 22 million voter records across Bihar, Tamil Nadu, and West Bengal, built an interactive election results interface without manual coding, and even assembled low-cost heat sensors to measure the working conditions of laborers in Chennai. These initiatives highlight a growing trend within newsrooms to adopt AI as a powerful tool for enhancing investigative capabilities.

Parsing India’s Voter Rolls

One of the most ambitious undertakings involved analyzing data from India’s Special Intensive Revision (SIR) of voter rolls. The Election Commission released records detailing voter deletions and the stated reasons, but the data presented significant challenges. The records were image-based PDFs, essentially photographs of forms, and were primarily in Hindi. Processing this volume of data – approximately 90,000 files covering 6.5 million records in Bihar alone, 78,000 files and 9.7 million records in Tamil Nadu, and 80,000 files and 5.8 million records in West Bengal – required a streamlined approach. Srinivasan Ramani and his team utilized optical character recognition (OCR) to convert the images into machine-readable text, translated the text into English, and stored the data in databases. LLMs were then employed to generate SQL queries from natural language prompts, bypassing the need for manual database command writing.

The analysis revealed concerning patterns. In Bihar, a disproportionate number of women were removed from the voter rolls compared to men, despite higher rates of male out-migration. A significant percentage of deleted voters across several polling booths were marked as deceased despite being under the age of 50. These findings prompted further investigation and scrutiny, particularly after the Supreme Court of India directed the Election Commission to release the full deletion records. The Hindu responded by creating a searchable database of deleted names and reasons, publishing state-level investigations that brought these discrepancies to light. As Ramani emphasized, “These were not conclusions drawn by AI. The hypothesis was ours. The political and social context was ours. AI helped us process the scale.” The resulting public pressure led to some corrections to the voter rolls in Bihar.

Building Interactive Election Maps

The application of AI extended beyond document processing to the creation of interactive tools. For both the 2019 and 2024 general elections in India, The Hindu’s team developed interactive maps allowing users to filter election results by region, state, rural-urban classification, and urban clusters. Remarkably, Ramani stated that he did not write a single line of code for these applications. Instead, the team leveraged models like ChatGPT, Gemini, and Claude, providing prompts to generate the necessary JavaScript, HTML, and D3 code. The process involved breaking down the interface into components – filters, maps, list views – and using the models to generate annotated code for each, which was then verified by the team. Given India’s nearly one billion eligible voters, building such tools at scale and under deadline is a significant technical undertaking, one that AI-assisted development substantially accelerated.

Measuring Heat Stress in Chennai

The integration of AI wasn’t limited to digital projects. In Chennai, where summer temperatures are often extreme, The Hindu investigated how heat stress varies across different occupations. Utilizing AI-assisted guidance, the newsroom assembled low-cost Arduino-based devices to record temperature and humidity every ten seconds. Components were sourced locally, keeping the project cost to approximately ₹15,000-₹20,000 (roughly $180-$240). Four devices were deployed with a cook, a fisherman, an industrial worker, and an autorickshaw driver, collecting data throughout a 24-hour period. The resulting data revealed a heat index peaking at 69°C (156.2 F) in some instances, highlighting the disparities in heat exposure experienced by different workers. Following publication, the Tamil Nadu government announced a heat management plan and expressed interest in using similar devices for further study.

AI as a Sophisticated Intern

Ramani consistently frames AI as a tool that augments, rather than replaces, journalistic perform. He describes it as “a very sophisticated intern” – capable of executing specific tasks when given clear instructions, but still requiring human oversight and control. This approach is reflected in The Hindu’s workflow, where AI assists with tasks like web scraping, document processing, query generation, and front-end interface development. However, human journalists remain responsible for formulating hypotheses, providing context, and ensuring the accuracy and integrity of the reporting.

One instance highlighted the need for human insight. An AI-generated script initially processed documents sequentially, slowing down the analysis. A technologist recognized the issue and suggested multi-threading, or parallel processing. When prompted accordingly, the model produced a more efficient version. This underscores the importance of human expertise in optimizing AI performance. Ramani also cautioned against relying on AI to draw editorial conclusions, arguing that the risk of “hallucination” is lower in structured tasks where outputs can be directly tested.

A Decade of Data-Driven Reporting

The Hindu’s journey with data journalism has evolved over the past decade, progressing from visual add-ons to traditional reporting to a dedicated function with data journalists, designers, and editorial coders. A significant early project involved an analysis of excess deaths during the COVID-19 pandemic, using civil registration data to estimate that official death counts were underreported by a factor of five to six. While initially contested, this finding was later supported by analyses from the World Health Organization and subsequent revisions to official data. Srinivasan Ramani notes that data-driven reporting is now integrated across all operations, leading to increased subscriptions and engagement. The newspaper’s commitment to this approach reflects a broader trend within the industry towards more informed and data-backed journalism.

The Hindu’s experience demonstrates that AI’s value in journalism lies not in automating storytelling, but in expanding the scale at which investigations can be conducted. By leveraging AI for data processing and code generation, news organizations can free up journalists to focus on critical thinking, analysis, and contextualization – the core elements of impactful reporting. The integration of these technologies, coupled with a commitment to human oversight, positions The Hindu to continue delivering in-depth, data-driven journalism to its audience.

Looking ahead, the continued refinement of LLMs and the development of new AI tools will likely further enhance the capabilities of data journalism teams. The key will be to maintain a balance between leveraging the power of AI and preserving the essential role of human journalists in ensuring accuracy, context, and ethical reporting. The focus remains on providing a more informed audience, and AI serves as a powerful tool in achieving that goal.