Towards Philosophical Reasoning with Agentic LLMs: Socratic Method for Scientific Assistance

Journal Article · Tue Dec 30 00:00:00 UTC 2025 · Machine Learning: Science and Technology

DOI: https://doi.org/10.1088/2632-2153/ae277f · OSTI ID:3363835

Harb, Hassan; Sun, Yunkai; Unal, Mustafa; Aggarwal, Abhishek; Bissolotti, Chiara; Buyuker, Isiksu; Hong, Sungil; Johnson, Luke; Jolaoso, Lateef A.; Sengupta, Bratin; Stuhr, Michael; Yang, Zhenzhen; Ingram, Brian J.; Surendran Assary, Rajeev

As large language models (LLMs) become central tools in science, improving their reasoning capabilities is critical for meaningful and trustworthy applications. We introduce a Socratic agent for scientific reasoning, implemented through a structured system prompt that guides LLMs via classical principles of inquiry. Unlike typical prompt engineering or retrieval-based methods, our approach leverages definition, analogy, hypothesis elimination, and other Socratic techniques to generate more coherent, critical, and domain-aware responses. We evaluate the agent across diverse scientific domains and benchmark it on the abstraction and reasoning corpus challenge dataset, achieving 97.15% under a fixed prompting protocol and without fine-tuning or external tools. Expert evaluation shows improved reasoning depth, clarity, and adaptability over conventional LLM outputs, suggesting that structured prompting rooted in philosophical reasoning can improve the scientific utility of language models.

• Research Organization: Argonne National Laboratory (ANL)
• Sponsoring Organization: Ames Laboratory - Laboratory Directed Research and Development (LDRD); US Department of Energy
• Grant/Contract Number: AC02-06CH11357
• OSTI ID: 3363835
• Journal Information: Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 4 Vol. 6
• Country of Publication: United States
• Language: English