Neue Publikation: "Introducing HALC: a general pipeline for the systematic and reliable construction of prompts for automated coding with LLMs in the computational social sciences" (CMM) [10.07.26]
Andreas Reich, Dr. Claudia Thoms und Tobias Schrimpf stellen in der Studie HALC vor, die Hohenheim automated LLM coding pipeline. HALC unterstützt Forschende bei der Entwicklung zuverlässiger Prompts für die Inhaltsanalyse mithilfe von LLMs.
Abstract
LLMs are seeing widespread use for task automation, including automated coding in the social sciences. However, even though researchers have proposed different prompting strategies, their effectiveness varies across LLMs and tasks. Often trial and error practices are still widespread. Our study aims to fill this gap and evaluate how LLMs can be used in a systematic and transparent way to produce reliable codings in content analyses. We propose HALC — a general pipeline that allows for the systematic and reliable construction of prompts for any given coding task and model. We develop this pipeline based on current literature and findings of a prestudy investigating consistency and influencing factors of LLM codings. We also apply HALC on two other datasets covering different thematic contexts, document types, languages, and coding units to test its applicability. Based on more than three million LLM requests, our results demonstrate that the pipeline is capable of identifying prompts for reliable codings in different settings. We also discuss shortcomings and further potential for development.
Zitation
Reich, A., Thoms, C., & Schrimpf, T. (2026). Introducing HALC: a general pipeline for the systematic and reliable construction of prompts for automated coding with LLMs in the computational social sciences. Communication Methods and Measures, 1–30. https://doi.org/10.1080/19312458.2026.2693637
