Doctors across Decades: Longitudinal Data Linkage for the American Medical Directories Project

Sean M. Smith, Rice University
Ben Chrisinger, Tufts University

This paper details a key framework developed by the American Medical Directories (AMD) Project to transform archival print medical directories spanning from 1906 to 1938 into a high-fidelity longitudinal dataset for modern social science research, containing an unparalleled individual-level record of professional medicine. A central contribution of this project is the development of a robust temporal linkage protocol to connect individual physicians' entries across AMDs. The construction of longitudinal career trajectories relies on a multi-tiered linkage infrastructure designed to balance the precision of archival data with the inherent variability of historical records, e.g., "William" vs. "Wm." Matches consider as many as five unique variables: physician name, medical school, graduation year, licensure year, and birth years. This multi-identifier approach facilitates the longitudinal tracking of career trajectories spanning almost 800,000 individual entries from selected AMD volumes, enabling the reliable tracking of physician's professional development and movement across the first half of the twentieth century. The methodology employs a series of matching protocols—ranging from high-strictness exact identifiers to probabilistic fuzzy matching—to generate multiple longitudinal sets of matches that include information about how the records were matched across volumes. Individual records are assigned identifiers, which are referenced in tables of matches that also note the strictness of the match. We report on the accuracy and precision of these various matching approaches and consider the inherent trade-offs. This research positions the AMD dataset as a foundational infrastructure for social science history, enabling—for the first time—large-scale quantitative analyses of physician migration, professionalization, and the persistence of racial disparities during critical junctures such as the Great Migration and the post-Flexner Report era. These longitudinal career records will provide a sustainable, interoperable resource for the broader research community that will be archived in the Tufts University instance of the Harvard Dataverse.

No extended abstract or paper available

 Presented in Session 125. Beyond the Census I