EMNLP: Educator-role Moral and Normative Large Language Models Profiling

#### Контекст Обучение с помощью искусственного интеллекта (AI) adheres to the principles of human-centered design, ensuring that AI systems are aligned with human values and ethical standards. One of the critical areas of application is education, where AI systems are designed to emulate the roles of educators. However, existing approaches to simulating professional roles (Simulating Professions, SP) often lack comprehensive psychological and ethical evaluations. This creates a gap in understanding how AI systems, particularly Large Language Models (LLMs), perform in roles that require moral and normative decision-making. The Educator-role Moral and Normative LLMs Profiling (EMNLP) framework addresses this gap by providing a structured approach to profiling teacher-role LLMs, focusing on moral and ethical dimensions. #### Метод EMNLP is designed as a comprehensive framework for profiling teacher-role LLMs, encompassing three main components: personality profiling, moral development stage measurement, and ethical risk assessment under soft prompt injection. The framework extends existing psychological scales and constructs 88 teacher-specific moral dilemmas, enabling a profession-oriented comparison between AI systems and human teachers. To evaluate compliance and vulnerability, a targeted soft prompt injection set is introduced, simulating real-world scenarios where ethical and psychological alignment is crucial. This methodology allows for a detailed analysis of the strengths and limitations of teacher-role LLMs, providing insights into their performance and potential risks. #### Результаты Experiments conducted on 12 LLMs revealed that teacher-role LLMs tend to exhibit more idealized and polarized personalities compared to human teachers. They demonstrate strong abstract moral reasoning but struggle with emotionally complex situations. The study also identified a paradox: models with stronger reasoning capabilities are more vulnerable to harmful prompt injection, highlighting the trade-off between capability and safety. Hyperparameters such as model temperature had limited influence on these behaviors except in specific risk scenarios. These findings provide a nuanced understanding of the ethical and psychological alignment of teacher-role LLMs, offering valuable insights for the development of ethical AI systems in education. #### Значимость The EMNLP framework has significant implications for educational AI, offering a benchmark for assessing the ethical and psychological alignment of teacher-role LLMs. It enables educators and developers to evaluate the performance and safety of AI systems in educational settings, ensuring that these systems adhere to ethical standards and support effective learning environments. The resources and benchmarks developed through EMNLP provide a foundation for future research in the ethical profiling of AI systems across various professional roles, paving the way for safer and more effective AI integration in education. #### Выводы The EMNLP framework represents a groundbreaking approach to profiling teacher-role LLMs, offering a detailed analysis of their moral, ethical, and psychological alignment. The findings highlight the strengths and limitations of current AI systems in educational roles, providing actionable insights for future research and development. Future work should focus on addressing the identified limitations, particularly in emotional reasoning and vulnerability to prompt injection, to enhance the safety and effectiveness of AI systems in educational settings.