Large language models are rapidly entering economic and financial decision-making, but their use also introduces new forms of model risk. This project develops rigorous and practical tools to diagnose bias, simulate alternative risk narratives, and improve macro-financial forecasting with LLM-based features.

Background

Recent advances in large language models have created major new opportunities for analyzing economic and financial text, supporting forecasting, and improving decision-making under uncertainty. At the same time, these models raise serious concerns for risk-sensitive applications: they may inherit human biases, react unpredictably to small prompt changes, or appear accurate because they rely on memorized information rather than robust reasoning.

These issues matter for insurance and reinsurance. If LLM-based systems distort survey expectations, misread regulatory communication, or misstate downside risks in financial markets, they can affect solvency assessment, capital allocation, and portfolio resilience. Despite rapid progress in AI, there is still limited evidence on how these models behave in periods of stress, under changing narratives, or in high-stakes economic settings. This project addresses that gap.

Objectives

1. Diagnosing and correcting bias in LLMs for economic applications
The first objective is to measure how LLMs respond to framing, prompt wording, option ordering, demographic steering, and confidence calibration in economic and financial tasks. The goal is to build a reproducible diagnostic pipeline that identifies when these models produce systematically distorted or unreliable outputs.

2. Counterfactual simulation of economic, insurance, and financial risk narratives
The second objective is to use LLMs to generate coherent counterfactual versions of policy statements, market commentary, and risk narratives, and then estimate how those textual changes would alter expected economic and financial outcomes. This creates a new tool for scenario analysis, stress testing, and risk communication.

3. Forecasting macroeconomic and financial downturns using LLM-based features
The third objective is to extract rich semantic signals from disclosures, news, policy texts, and surveys, in order to improve forecasts of GDP growth, unemployment, volatility, and broader downturn risk. The project will benchmark these LLM-based signals against traditional econometric and sentiment-based methods.

Relevance for the SCOR Foundation

This project is closely aligned with the SCOR Foundation's mission to deepen scientific understanding of risk and to support methods that are both rigorous and useful in practice.

For insurers and reinsurers, the work has direct benefits: tools to audit bias and instability in AI-assisted risk analysis, a framework to test how alternative policy or regulatory narratives may affect expectations and solvency perceptions, and forecasting methods that may detect macro-financial stress earlier than conventional approaches.

(All subtitles in this video are generated by YouTube)

Steering Committee

• Sofia Kyriakopoulou, Chief Technology Data and AI Officer, Data Office, SCOR
• Dacheng Xiu, Professor of Econometrics and Statistics, University of Chicago Booth School of Business

The SCOR Foundation is invited permanently to attend the steering committee.

Expected Outcomes

• A validated bias-diagnostic pipeline for LLMs in economic and financial applications
• A prototype counterfactual scenario generation tool for narrative-based stress testing
• New forecasting models for macro-financial and insurance-related risk
• Academic publications and practitioner-facing workshops with SCOR and industry audiences