Measuring Cooperation in Human-Machine Teams: An Information-Theoretic Approach

This paper proposes a method to measure cooperation using entropy over communication logs, validated on the Enron corpus, and designed to guide Cooperative AI evaluation.

Overview

As large language models and multi-agent systems increasingly integrate into organizational workflows, we face a core question:
How do we know if they're cooperating?

This paper introduces a metric to answer that: entropy, a measure of communication complexity across human and machine agents.

The idea

High entropy = more diverse, informative communication = more cooperation.

Low entropy = compressed or controlled communication = less cooperative or more constrained behavior.

What we did

We applied this metric to the Enron email corpus—tracking how vocabulary diversity and entropy changed over time, especially during Enron’s corporate crisis. The results show:

  • Increased entropy during healthy periods of growth
  • Sharp entropy drops during periods of internal control or crisis
  • A clear relationship between communication patterns and organizational health
Entropy is not just a signal of technical noise—it’s a potential early indicator of breakdowns in cooperation, trust, or knowledge flow.

Why it matters

This work lays the foundation for a Cooperative AI benchmark that goes beyond accuracy to measure usefulness, alignment, and teaming behavior. The method can be applied to:

  • AI agent communication logs
  • Human-AI chat sessions
  • Internal enterprise systems that rely on LLM workflows

Our other articles

All articles