Please join us Wednesday, November 15th, 2023 from 12:30p-1:30p in room 2071 of the Social Science Building as we welcome our very own Dr. Sarah Dreier. Talk title and abstract are below. This event is free and open to the public.

If you are interested in an online broadcast of this talk, please contact Department Administrator Marliss McGarvey.

An event flyer can be viewed and downloaded at this link [PDF].

Title:

Troubles in Text: Using AI to Recognize Government Rationalizations for Rights Abuses

Abstract:

Can AI innovations in text analysis — popularized by public interfaces like ChatGPT — help researchers computationally analyze archives of classified government documents?Automation expands researchers’ abilities to examine political phenomena documented within texts generated during real political processes. Do recent natural language processing (NLP) innovations in capturing words’ contextual meanings yield increased performance in computationally recognizing political concepts in text? We evaluate how one model (RoBERTa) performs on a uniquely challenging task: classifying a complex concept — British rationalizations for internment without trial in N. Ireland (1969-73), within imperfectly digitized archive text. RoBERTa reliability outperforms conventional supervised methods at identifying and classifying rationalizations for internment, but performance may not always be adequate for specific objectives. However, with proper model specifications, RoBERTa demonstrates an improved ability to computationally annotate data and significantly reduce the amount of data requiring manual annotation. This article demonstrates the utility of and limits to using advanced NLP models to classify political concepts, discusses when applying such models could be beneficial, and offers practical instruction for NLP application in social science.

Speaker Bio:

Dr. Sarah Dreier received her Ph.D. in political science from the University of Washington, where she also served as an NSF Postdoctoral Research Fellow in political science and computer science and an eScience Data Science Fellow. She previously worked at the Center for American Progress in Washington, D.C. Dreier received her B.A. from Northwestern University.