CHAOS: Converting Historical Accounts into Occupational Scores

Torben Johansen, University of Southern Denmark
Matthew Curtis, University of Southern Denmark
Christian Vedel, University of Southern Denmark
Julius Koschnick, University of Southern Denmark

History is rich in information about people’s occupations but far less forthcoming on outcomes such as income, wealth, or skills. For this reason, a widely used approach is to proxy these outcomes by averages or medians within each occupational category – so-called occupational scores – of which IPUMS’ OCCSCORE (Sobek, 1995) is the most prevalent. This strategy has enabled new insights into topics as varied as technological change, inequality, and institutions. But existing OCCSCOREs are limited: they rely on fixed benchmarks such as 1950 U.S. wages, discard ambiguity in historical titles, and are difficult to adapt across time and space, generating downstream biases that are hard to correct (Inwood, Minns, & Summerfield, 2019; Saavedra & Twinam, 2020). A key problem is that researchers cannot feasibly construct a new occupational score by hand for every project. This paper introduces a way to automatically estimate occupational scores tailored to any project. CHAOS is a replicable, fully automatic, and interpretable framework for converting historical occupational descriptions into outcome estimates given a source. The key insight is that any occupational score is a weighted average of observed outcomes, where the weights reflect the relevance of each piece of source information. These relevance weights can be estimated automatically using a classification algorithm such as OccCANINE (Dahl, Johansen, & Vedel, 2024), enabling entirely new scales of data collection in economic history. We demonstrate the utility of CHAOS through an application to historical U.S. wage reports covering around 100 thousand occupation-income pairs across U.S. states throughout the 19th century. From this source we recover annual wage estimates by state and use them to estimate the evolution of skill biases across an entire century of the Industrial Revolution.

No extended abstract or paper available

 Presented in Session 70. Wrangling Occupational and Denominational Data