Development, Content Validation, and Preliminary Effectiveness of the NEXUS Multicomponent Educational Protocol Integrating AI-Mediated Socratic Inquiry, Identity Meaning-Making and Impactful Expression in Adolescents: A Small-Cluster Randomized Trial with Three-Month Follow-Up

Document Type : Original Article

Authors

1 Department of Psychology, Payame Noor University, Takestan, Iran

2 PhD Student in Health Psychology, Department of Psychology, To.C., Islamic Azad University, Tonekabon, Iran

10.5281/zenodo.21862976
Abstract
Background: Generative artificial intelligence (GenAI) can support explanation and feedback, but unrestricted answers may encourage cognitive offloading, weak source evaluation, and diminished learner authorship. This study developed and content-validated NEXUS, which restricts GenAI to Socratic coaching, and examined preliminary effectiveness for learning agency, intrinsic motivation, and critical-thinking disposition among male upper-secondary students in Qazvin, Iran. Methods: Phase I combined needs mapping in 463 students, co-design, cognitive interviews, and two-round Delphi validation by 15 experts. Phase II randomized 12 classes (106 students; six clusters per arm) to ten weekly NEXUS sessions or a dose- and technology-matched active control. Three-month follow-up was primary. Mixed-model estimates were paired with CR2 inference, CR3 sensitivity analysis, restricted wild-cluster bootstrap-t intervals, and exact within-pair randomization tests. Model-based Hedges g used total class-plus-student variance. Results: The 55-element manual achieved S-CVI/Ave = 0.91; universal agreement was 0.31, and Fleiss kappa was 0.66 for relevance and 0.61 for necessity. Follow-up completion was 89.6%. Adjusted follow-up differences were 0.44 for agentic engagement (95% CI 0.10 to 0.78; Holm-adjusted p = 0.042; g = 0.43), 0.31 for intrinsic motivation (95% CI -0.04 to 0.66; adjusted p = 0.086; g = 0.30), and 6.1 for critical-thinking disposition (95% CI 0.6 to 11.6; adjusted p = 0.078; g = 0.38). The exact global multivariate randomization test was p = 0.031. Fidelity averaged 86.8%; minor AI-output and participant-burden events occurred, but no serious related event was observed. Conclusion: NEXUS showed credible content validity and a promising joint signal, strongest for agentic engagement. With only 12 clusters, incomplete blinding, and no prospective registration, independent replication is required.

Graphical Abstract

Development, Content Validation, and Preliminary Effectiveness of the NEXUS Multicomponent Educational Protocol Integrating AI-Mediated Socratic Inquiry, Identity Meaning-Making and Impactful Expression in Adolescents: A Small-Cluster Randomized Trial with Three-Month Follow-Up

Keywords

Subjects

[1]                L. Yan, S. Greiff, Z. Teuber, D. Gasevic, Promises and challenges of generative artificial intelligence for human learning, Nature Human Behaviour, 2024, 8, 1839-1850.          
[2]                R. Deng, M. Jiang, X. Yu, Y. Lu, S. Liu, Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies, Computers & Education, 2025, 227, 105224.    https://doi.org/10.1016/j.compedu.2024.105224
[3]                M.Y.I. Helal, I.A. Elgendy, M.A. Alb Ashrawi, Y.K. Dwivedi, M.S. Al-Ahmadi, I. Jeon, the impact of generative AI on critical thinking skills: A systematic review, conceptual framework and future research directions, Information Discovery and Delivery, 2025, advance online publication.  https://doi.org/10.1108/IDD-05-2025-0125
[4]                N. Cong-Lem, N.T. Thuy-Dung, towards a framework for understanding and assessing critical thinking in generative AI-enhanced learning environments: A scoping review, European Journal of Psychology of Education, 2026, 41(2), Article 41.    https://doi.org/10.1007/s10212-026-01089-y
[5]                S. Tao, M. Lan, M. Wang, H. Li, Potential risks of generative artificial intelligence integration into K-12 education: A scoping review, Computers and Education: Artificial Intelligence, 2026, 10, 100561.         https://doi.org/10.1016/j.caeai.2026.100561
[6]                Y. Fan, L. Tang, H. Le, K. Shen, S. Tan, Y. Zhao, Y. Shen, X. Li, D. Gasevic, Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance, British Journal of Educational Technology, 2025, 56(2), 489-530. https://doi.org/10.1111/bjet.13544
[7]                G. Brod, Agency does not equal choice: Conceptualizing agency for learning in the age of AI, Learning and Individual Differences, 2026, 125, 102841.      https://doi.org/10.1016/j.lindif.2025.102841
[8]                G. Brod, E. Holstein, L. Weindorf, J. Colantonio, E. Bonawitz, M. Theobald, do it yourself: Discerning the effects of self-directed activity on conceptual learning, npj Science of Learning, 2025, 10, 70.   https://doi.org/10.1038/s41539-025-00364-9
[9]                P.-B. Degen, I. Asanov, beyond automation: Socratic AI, epistemic agency, and the implications of the emergence of orchestrated multi-agent learning architectures, arXiv, 2025. https://doi.org/10.48550/arXiv.2508.05116
[10]             J. Lee, J.-T. Hung, M.Y. Soylu, D. Popescu, C.Z. Cui, G. Grigoryan, D.A. Joyner, S.W. Harmon, Socratic Mind: Impact of a novel GenAI-powered assessment tool on student learning and higher-order thinking, arXiv, 2025. https://doi.org/10.48550/arXiv.2509.16262
[11]             J. Reeve, C.-M. Tseng, Agency as a fourth aspect of students’ engagement during learning activities, Contemporary Educational Psychology, 2011, 36(4), 257-267.           https://doi.org/10.1016/j.cedpsych.2011.05.002
[12]             C. Mameli, S. Passini, Development and validation of an enlarged version of the Student Agentic Engagement Scale, Journal of Psychoeducational Assessment, 2019, 37(4), 450-463. https://doi.org/10.1177/0734282918757849
[13]             R.M. Ryan, E.L. Deci, Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions, Contemporary Educational Psychology, 2020, 61, 101860.              https://doi.org/10.1016/j.cedpsych.2020.101860
[14]             X. Fang, J. Feng, Research on the application behavior of generative artificial intelligence learning of college students based on self-determination theory, Frontiers in Psychology, 2026, 17, 1805498.           https://doi.org/10.3389/fpsyg.2026.1805498
[15]             H. Galindo-Dominguez, N. Delgado, M.V. Urruzola, J.M. Etxabe, L. Campo, using artificial intelligence to promote adolescents’ learning motivation: A longitudinal intervention from self-determination theory, Journal of Computer Assisted Learning, 2025, 41(2), e70020.          https://doi.org/10.1111/jcal.70020
[16]             L. Zhang, W. Zhang, integrating large language models into project-based learning based on self-determination theory, Interactive Learning Environments, 2025, 33(5), 3580-3592.   https://doi.org/10.1080/10494820.2024.2446535
[17]             S. Choi, H. Kim, The impact of a large language model-based programming learning environment on students’ motivation and programming ability, Education and Information Technologies, 2025, 30(6), 8109-8138.
https://doi.org/10.1007/s10639-024-13107-x
[18]             E. Crocetti, M. Pagano, F. De Lise, F. Maratia, B. Bobba, W. Meeus, V. Bacaro, Promoting adolescents’ personal and social identities: A meta-analysis of psychosocial interventions, Identity, 2025, 25(2), 167-196.  https://doi.org/10.1080/15283488.2024.2394888
[19]             E. Crocetti, F. De Lise, M. Pagano, B. Bobba, F. Maratia, W. Meeus, V. Bacaro, What’s your vocation? A meta-analysis of interventions tackling youth educational and professional identity, Identity, 2025, 25(2), 197-215.  https://doi.org/10.1080/15283488.2024.2394861
[20]             Rezai, R. Ahmadi, P. Ashkani, G.H. Hosseini, implementing active learning approach to promote motivation, reduce anxiety, and shape positive attitudes: A case study of EFL learners, Acta Psychologica, 2025, 253, 104704. https://doi.org/10.1016/j.actpsy.2025.104704
[21]             J.A. Alqurni, Exploring the role of agentic AI in fostering self-efficacy, autonomy support, and self-learning motivation in higher education, Frontiers in Artificial Intelligence, 2026, 9, 1738774. https://doi.org/10.3389/frai.2026.1738774
[22]             M. Badali, M. Shah Alizadeh, M. Alimirzaie, O. Noroozi, S.K. Bani Hashem, Bridging AI chatbot use and critical thinking: The mediating role of AI literacy, Interactive Learning Environments, 2026, advance online publication. https://doi.org/10.1080/10494820.2025.2605487
[23]             Y. Zhao, Y. Yue, Z. Sun, Q. Jiang, G. Li, does generative artificial intelligence improve students’ higher-order thinking? A meta-analysis based on 29 experiments and quasi-experiments, Journal of Intelligence, 2025, 13(12), 160.   https://doi.org/10.3390/jintelligence13120160
[24]             F. Shirdel, N. Mobasheri, M.H. Kaveh, J. Hassanzadeh, L. Ghahremani, A randomized controlled trial to evaluate the effectiveness of a theory of planned behavior-based educational intervention in reducing internet addiction among adolescent girls in southern Iran, Adolescents, 2025, 5(3), 33.    https://doi.org/10.3390/adolescents5030033
[25]             S. Bashirinia, N. Motamed, M. Bahreini, M. Ravani pour, The effects of an empowering self-management model on sense of coherence and self-efficacy among adolescent girls with internet addiction in Bushehr, Iran: A randomized controlled trial, BMC Public Health, 2026, 26, 424.            https://doi.org/10.1186/s12889-025-26117-2
[26]             K. Skivington, L. Matthews, S.A. Simpson, P. Craig, J. Baird, J.M. Blazeby, K.A. Boyd, N. Craig, D.P. French, E. McIntosh, M. Petticrew, J. Rycroft-Malone, M. White, L. Moore, A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance, BMJ, 2021, 374, n2061.
https://doi.org/10.1136/bmj.n2061
[27]             T.C. Hoffmann, P.P. Glasziou, I. Boutron, R. Milne, R. Perera, D. Moher, D.G. Altman, V. Barbour, H. Macdonald, M. Johnston, S.E. Lamb, M. Dixon-Woods, P. McCulloch, J.C. Wyatt, A.-W. Chan, S. Michie, better reporting of interventions: Template for Intervention Description and Replication (Tidier) checklist and guide, BMJ, 2014, 348, g1687. https://doi.org/10.1136/bmj.g1687
[28]             A.-W. Chan, I. Boutron, S. Hopewell, D. Moher, K.F. Schulz, G.S. Collins, et al., SPIRIT 2025 statement: Updated guideline for protocols of randomized trials, BMJ, 2025, 389, e081477.
https://doi.org/10.1136/bmj-2024-081477
[29]             S. Hopewell, A.-W. Chan, G.S. Collins, A. Hrobjartsson, D. Moher, K.F. Schulz, et al., CONSORT 2025 statement: Updated guideline for reporting randomized trials, BMJ, 2025, 389, e081123.
https://doi.org/10.1136/bmj-2024-081123
[30]             Baba, M. Smith, B.K. Potter, A.-W. Chan, D. Moher, A. Toulany, et al., CONSORT-Children and Adolescents (CONSORT-C) 2026 extension statement: Enhancing the reporting and impact of pediatric randomized trials, JAMA Pediatrics, 2026, 180(4), 413-425. https://doi.org/10.1001/jamapediatrics.2026.0104
[31]             R.J. Vallerand, L.G. Pelletier, M.R. Blais, N.M. Briere, C. Senecal, E.F. Vallieres, The Academic Motivation Scale: A measure of intrinsic, extrinsic, and amotivation in education, Educational and Psychological Measurement, 1992, 52(4), 1003-1017. https://doi.org/10.1177/0013164492052004025
[32]             N. Bagheri, M. Shahraray, V. Farzad, Standardization of Academic Motivation Scale (AMS) among the high school students in Tehran, Clinical Psychology and Personality, 2003, 1(1), 11-24.         https://cpap.shahed.ac.ir/article_2771.html?lang=en
[33]             J. Cavoukian, P. Kadivar, M. Shahraray, A.A. Sheikhi Fini, V. Farzad, Standardization of the Vallerand Academic Motivation Scale, Journal of Educational Sciences, 2010, 5(4), 103-130. https://www.noormags.ir/view/fa/articlepage/922938/
[34]             J.C. Ricketts, The Efficacy of Leadership Development, Critical Thinking Dispositions, and Student Academic Performance, doctoral dissertation, University of Florida, Gainesville, 2003.
https://ufdc.ufl.edu/UFE0000777/00001
[35]             H. Pakmehr, F. Mirdoraghi, A. Ghanaei Chamanabad, M. Karami, Reliability, validity and factor analysis of Ricketts’ Critical Thinking Disposition Scales in high school, Quarterly of Educational Measurement, 2013, 4(11), 33-54.        https://jem.atu.ac.ir/article_2678_en.html
[36]             F. Miao, W. Holmes, Guidance for Generative AI in Education and Research, UNESCO, Paris, 2023. https://unesdoc.unesco.org/ark:/48223/pf0000386693
[37]             F. Miao, K. Shiohira, N. Lao, AI Competency Framework for Students, UNESCO, Paris, 2024. https://unesdoc.unesco.org/ark:/48223/pf0000391105
[38]             S. Gupta, S. Grover, V. Menon, P.V. Indu, K. Vidhu Kumar, D. Chacko, Item generation and establishing face and content validity of a rating scale: A primer, Indian Journal of Psychiatry, 2025, 67(8), 816-822.    https://doi.org/10.4103/indianjpsychiatry_750_25
[39]             O. Quintin, J. Kasza, N. Davies, S. Markham, R. Hooper, Small numbers of clusters in cluster randomized trials: A scoping review of problems and proposed solutions, Journal of Clinical Epidemiology, 2026, online ahead of print, 112395.  https://doi.org/10.1016/j.jclinepi.2026.112395
[40]             A.-L. Schubert, M. Steinhilber, H. Kang, D.S. Quintana, Improving statistical reporting in psychology, Communications Psychology, 2025, 3, 156.
https://doi.org/10.1038/s44271-025-00356-w
[41]             F.L. Huang, when cluster-robust inferences fail, Educational and Psychological Measurement, 2026, 86(3), 579-601. https://doi.org/10.1177/00131644251393203
[42]             I.A. Canay, A. Santos, A.M. Shaikh, The wild bootstrap with a “small” number of “large” clusters, Review of Economics and Statistics, 2021, 103(2), 346-363.             https://doi.org/10.1162/rest_a_00887
[43]             S. Grund, O. Lüdtke, A. Robitzsch, Multiple imputation of multilevel data with single-level models: A fully conditional specification approach using adjusted group means, Behavior Research Methods, 2026, 58, 76.       https://doi.org/10.3758/s13428-025-02915-9