Author –Anchal Singh

Insights from SustainoMetric’s Assessment Across 50 Companies 

Artificial intelligence has quickly moved from being an experimental capability to a core layer in ESG analysis. As discussed in the previous article, AI excels at extracting structured disclosures, identifying policies, and processing quantitative datasets at scale. It has materially improved the speed, consistency, and scalability of ESG research. 

But ESG analysis is not a data extraction problem. It is fundamentally a contextual judgment problem, one that requires interpreting intent, validating credibility, and assessing whether disclosures are decision-useful.  And this is where, after years of working at the intersection of AI systems and ESG frameworks, a consistent limitation becomes evident: 

AI does not struggle with finding data; it struggles with understanding it. 

The Core Limitation: Detection Without Evaluation 

Across SustainoMetric’s assessment of 50 publicly listed companies and 30 ESG indicators, the most recurring issue was not extraction failure, but interpretation failure. In many cases, AI systems correctly identified relevant disclosures. However, they struggled to evaluate whether the disclosure was complete, aligned with the indicator methodology, reflected implementation rather than intent, or met the threshold for decision relevance. 

This creates a structural gap in AI-driven ESG analysis: 

AI can confirm the presence of disclosure. It cannot reliably assess its quality. 

Supply Chains: The Weakest Link in Automation 

Supply chain disclosures emerged as the most significant blind spot. Unlike emissions or governance data, supply chain information is fragmented across reports, expressed in highly non-standardized language, and predominantly qualitative in nature. 

One of the most critical failure points was the misinterpretation of intent versus enforcement. Statements such as “We engage with suppliers on sustainability practices” or “We encourage responsible sourcing” were often interpreted by AI systems as evidence of robust supply chain governance. 

However, in ESG evaluation, the distinction is critical. Engagement represents dialogue, expectation reflects stated intent, while enforcement requires measurable accountability mechanisms. AI systems frequently collapse these distinctions, resulting in an overestimation of supply chain rigor—precisely in the area where real-world risk exposure is often highest. 

Workforce Metrics: When Methodology Is Misunderstood 

Workforce disclosures exposed a different class of limitation: methodological inconsistency. While AI handles numbers efficiently, it often fails to preserve analytical integrity across definitions and reporting boundaries. 

Across assessments, recurring issues included mixing full-time equivalent (FTE) data with headcount figures, using total workforce numbers instead of permanent employee counts, selecting unconsolidated rather than consolidated figures, and recalculating metrics that companies had already reported. 

These are not computational mistakes; they are methodological breakdowns. In ESG analysis, even small inconsistencies in denominators, scope, or reporting boundaries can materially distort outcomes, particularly for metrics such as diversity ratios, attrition rates, and gender representation. 

The key insight is clear: AI can process numbers at scale, but it does not inherently understand how those numbers are constructed. 

Commitments vs. Aspirations: Where Language Misleads Machines 

Another major gap lies in how AI interprets sustainability commitments. When disclosures include structured targets, defined timelines, and measurable baselines, AI performs relatively well. 

The challenge emerges when companies rely on aspirational language. Statements such as “We aim to reduce emissions over time” or “We are committed to improving sustainability performance” were, in several cases, classified as formal commitments or interpreted as evidence of full compliance. 

This exposes a deeper limitation: AI struggles to distinguish between ambition and obligation. 

For ESG practitioners, that distinction defines credibility. A commitment without measurable targets or accountability mechanisms is not a commitment—it is intent. 

Governance: Structure Without Context 

Governance disclosures often appear highly structured, but they still require organizational interpretation. A recurring issue observed during assessments was the substitution of board-level disclosures in place of executive or management-level governance indicators. 

For example, when an indicator specifically required evidence of executive accountability, AI systems occasionally extracted general board composition disclosures instead. 

This is not a data availability problem; it is a contextual hierarchy problem. Organizational structures vary significantly across companies, and interpreting governance responsibilities requires understanding reporting lines, authority structures, and operational accountability—areas where rule-based extraction systems remain limited. 

Policy Disclosures: The Illusion of Completeness 

Policy indicators revealed another consistent pattern: over-classification. 

When policies were clearly structured and explicitly defined, AI systems performed reliably. However, when policy elements were embedded within broader narrative disclosures, AI frequently flagged partial mentions as complete policies, overlooked missing thresholds or criteria, and ignored the absence of enforcement mechanisms altogether. 

The result is a potentially dangerous assumption that the existence of policy language automatically indicates policy effectiveness. 

In ESG analysis, however, existence does not equal adequacy. 

Even Emissions Data Isn’t Immune 

Even emissions reporting, arguably one of AI’s strongest ESG use cases, was not free from contextual errors. 

Observed issues included selecting incorrect reporting periods due to fiscal-year misalignment, misinterpreting reporting boundaries between partial and full scope disclosures, and recalculating emissions metrics using incorrect baselines. 

Importantly, these were not calculation failures. They were input selection failures, where correct analytical logic was applied within the wrong contextual framework. 

This reinforces a broader reality: structured data may reduce ambiguity, but it does not eliminate the need for interpretation. 

The Pattern Is Clear: Context Is the Missing Layer 

Across all ESG themes, the same pattern consistently emerged. AI systems are highly effective at extracting disclosures, but considerably weaker at interpreting them accurately. 

Most observed failures fell into three categories: 

1. Context misinterpretation — using the right data in the wrong context  

1. Methodological misalignment — breaking consistency in definitions, scope, or denominators  

1. Over-classification — assuming completeness from partial signals  

Together, these issues point to a fundamental limitation: 

AI reads ESG disclosures. It does not fully understand them. 

Why This Matters: The Risk of False Confidence 

These limitations are not theoretical—they directly influence decision-making quality. 

When AI systems overstate policy strength, misclassify sustainability commitments, or misinterpret supply chain controls, ESG scores risk becoming directionally misleading rather than genuinely decision-useful. 

In an ecosystem increasingly dependent on ESG data, from investors to regulators, this creates a critical tension: efficiency gains cannot come at the expense of analytical integrity. 

Reframing the Role of AI in ESG 

The conclusion is not that AI is ineffective. On the contrary, it is indispensable for scaling ESG analysis. However, its role must be clearly and realistically defined. 

AI is not a decision-maker. It is an analytical accelerator. 

Treating it otherwise introduces systemic risk into ESG evaluation frameworks. 

What Comes Next 

If AI performs best in structured environments but weakens in context-heavy analysis, the path forward becomes increasingly clear: 

👉 The future of ESG analysis is not full automation – it is hybrid intelligence. 

In the final article of this series, we explore how combining AI with expert judgment can create a more robust, scalable, and decision-useful ESG analysis model—and what both systems and disclosures must do to evolve.