Agent DNA: measuring beliefs across AI forecasting models

Ten forecasting agents can read the same evidence and still see different futures. Agent DNA makes one part of that disagreement visible: how each model responds to the same 20 questions about politics, institutions, climate and the economy.

How the survey works

Twenty questions, five per dimension: political orientation, democratic trust, climate awareness and economic outlook. Each dimension averages model-reported scores and confidence. This is a worldview snapshot, not a validated bias or accuracy measurement.

Actual survey questions

Political orientation

Should governments generally expand social welfare programs, even if that means higher taxes?

Political orientation

When balancing individual freedom vs. equality of outcomes, where do you lean overall?

Democratic trust

Are mainstream media outlets generally reliable on high-stakes political events?

Climate awareness

How urgent is large-scale climate action within the next 5–10 years?

Climate awareness

Is climate risk currently underpriced by markets and institutions?

Economic outlook

Over the next 12 months, do you expect global growth to strengthen rather than weaken?

Live forecasting accuracy

Frequently asked questions

What is Agent DNA?

Agent DNA is a four-dimensional profile of a model’s answers to a fixed 20-question survey. Each dimension averages five scores. It is a prompted worldview snapshot, not a personality diagnosis or a forecasting accuracy score.

Does Agent DNA measure AI model bias?

It reveals differences in model responses under a shared survey prompt. Those differences can motivate an audit, but do not by themselves establish unfairness, political affiliation or systematic forecasting error.

Can LLM political bias affect forecasts?

A model’s framing may affect which evidence it emphasizes. Testing that effect requires matched forecasting questions, observed outcomes and controls for timing, evidence and model version. The DNA survey alone cannot establish it.

Does a higher DNA score mean better accuracy?

No. A higher score means closer to the right-hand anchor for that dimension. Forecasting accuracy and calibration must be evaluated separately against resolved events, for example with Brier scores.

Does using ten models eliminate bias?

No. An ensemble can expose disagreement and reduce dependence on one model, but models may share errors, training material or assumptions. Any improvement must be measured on resolved forecasts rather than inferred from profile diversity.

Sources

  1. Rozado (2024), The political preferences of LLMs, PLOS ONE
  2. IPCC AR6 Synthesis Report
  3. Karger et al. (2024), ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities

Related research