nikkolas.cheongtrillo@ontariotechu.net

Ontario Tech University

Abstract

ChatGPT, released by OpenAI (2023a) in November 2022, was the first widely available chatbot with artificial intelligence integration. Chatbots are software applications that can generate text based on user input. The utilization of chatbots has increased substantially since the release of ChatGPT and is rapidly integrating into everyday life. This chapter aimed to identify the potential benefits of using chatbots in education, the barriers involved in chatbot adoption, and provide recommendations to overcome these barriers. The results of the studies reviewed in this chapter identified that chatbots can provide students with immediate feedback on assessments, individualized learning experiences, and immediate access to information. The findings also indicate that the barriers to chatbot adoption in education are related to the perceived usefulness, perceived ease of use, perceived organizational support, and perceived risk associated with their use. Based on these findings it is recommended that organizations provide continuing education programs for pre-service and in-service teachers and dedicate funding to ongoing technical support to successfully integrate chatbots into an educational system.

Keywords

AI, artificial intelligence, chatbots, ChatGPT, education, TAM, Technology Acceptance Model, Unified Theory of Acceptance and Use of Technology, UTAUT

Introduction

The release of Chat Generative Pre-Trained Transformer (ChatGPT) (OpenAI, 2023a) in November 2022 sparked the rise of the rapid development of chatbots utilizing artificial intelligence (AI). Chatbots are software applications with the ability to respond to human prompting (Cunningham-Nelson et al., 2019). At the time of its release, ChatGPT was the first widely available chatbot capable of generating text indistinguishable, in some cases, from human-generated text (Gao et al., 2022). Due to this novel ability, ChatGPT garnered more than 120 million users within the first two months of release, becoming the fastest-growing software application of all time (Milmo, 2023).

Historically, educators viewed their interactions with chatbots negatively, citing that the responses from chatbots were rigid and unoriginal (Kim & Kim, 2022), only capable of answering simple questions (Cunningham-Nelson et al., 2019). This is likely because these traditional chatbots, or frequently asked questions (FAQ)-type chatbots (Cunningham-Nelson et al., 2019; Merelo et al., 2022), do not utilize AI and are trained to respond using predetermined criteria (Smutny & Schreiberova, 2020). These FAQ-type chatbots are commonly used for automating customer service processes like booking a car service appointment or receiving help from a phone service provider. Alternatively, ChatGPT is powered by the large language models (LLMs), GPT-3.5, and GPT-4 (OpenAI, 2023b). LLMs are AI models trained using large quantities of text, generating comprehensive human-like text, unlike previous chatbot iterations (Birhane et al., 2023).

Due to AI integration in the workplace, the World Economic Forum (2023) estimates that by 2027, 25% of companies expect job loss, while 50% expect job growth. In May 2023, Google (2023) and Microsoft (Panay, 2023) announced that their products would integrate AI. As chatbots become more popular and AI becomes increasingly integrated into day-to-day life, it is important to prepare students for the future, as skills using these technologies may be a requirement when entering the future workforce. In addition, these technologies can potentially enhance student learning over traditional learning methods. It is the job of the educator to provide the best learning experience to each learner. However, teachers may feel uncomfortable adopting new technologies in the classroom (Tallvid, 2016; Zimmerman, 2006). The aim of this chapter is to identify the potential benefits of adopting chatbots in education to provide teachers with the necessary foundational information to decide whether the inclusion of chatbots in their pedagogy will be beneficial for their students. In addition, this chapter outlines the potential barriers teachers may face if choosing to adopt chatbots and provides recommendations to help facilitate successful chatbot integration.

Potential Benefits of Chatbots in Education

Chatbots in Education

Prior to the release of ChatGPT, chatbots in education have been studied extensively. Several systematic literature reviews have been conducted outlining the benefits of chatbot use in education. However, at the time of writing this chapter, there has been limited peer-reviewed research on chatbots utilizing LLMs, specifically. The chatbots studied in the current literature are traditional, FAQ-type chatbots.

Since the release of ChatGPT, several chatbots and LLMs have been released, such as Bing (Microsoft, 2023), LLaMA (Touvron et al., 2023), Google Bard (Pichai, 2023, February 6), Alpaca (Taori et al., 2023), Vicuna (Chiang et al., 2023) and Guanaco (Dettmers et al., 2023). It is expected that as these models become more widely available for commercial use, research on the benefits of their use will also increase. Because chatbots using LLMs have vastly more capabilities than their traditional counterparts, it is expected that there are additional benefits not currently identified in the literature. Therefore, this section outlines the benefits of traditional chatbot use in education.

Benefits

Chatbot use in education can provide benefits to both the student and the teacher. Chatbots have been shown to be capable of providing students with immediate feedback, quick access to information, increasing engagement and interest, and creating course material individualized to the learner.

Immediate Feedback

Chatbots can provide students with immediate feedback, assisting the metacognitive processes of learning (Chang et al., 2022; Cunningham-Nelson et al., 2019; Guo et al., 2022; Okonkwo & Ade-Ibijola, 2021; Wollny et al., 2021). Similar feedback functions are incorporated on a smaller scale into software applications such as Grammarly, Microsoft Word, and Google Docs. Utilizing chatbots, students can make their statements more clear and concise (Cunningham-Nelson et al., 2019) and receive assistance solving difficult problems (Kaur et al., 2021). In one study, students used chatbots to provide continuous feedback on their argumentative essays to assist with writing (Guo et al., 2022). Typically, this feedback is received after peer review or first draft submissions rather than concurrently within the writing process. Students can make revisions and reflect on their learning without the need to interact with their teacher (Cabales, 2019), which can sometimes be difficult in an online learning environment where interactions with teachers are limited (Chang et al., 2022).

Individualized Learning

Automated teaching systems like chatbots can be used to analyze and assess student learning to help teachers identify a student’s level of understanding of a topic (Okonkwo & Ade-Ibijola, 2021). Students that struggle with specific materials can be provided individualized learning materials based on the information collected. In addition, this collected data can provide educators and administration with useful information to profile and predict the likelihood of success a student may have in a course (Zawacki-Richter et al., 2019). This means that teachers can develop systems to identify students at risk of failing and offer appropriate guidance and intervention.

Access to Information

Students can access information by interacting with chatbots faster than by interacting with their teacher (Murad et al., 2019; Okonkwo & Ade-Ibijola, 2021; Wollny et al., 2021; Wu et al., 2020). Chatbots are capable of replying to emails to provide students with instant answers (Merelo et al., 2022). Students no longer need to wait for a response from their teacher, and the teacher does not need to spend time writing emails. Sandoval (2018) noted that teachers used FAQ-type chatbots to answer common administrative questions related to the course syllabus, and Google (2023) demonstrated enhanced capabilities with chatbot integration in their email service Gmail at the 2023 I/O Conference.

Barriers to Chatbot Acceptance in Education

Perceptions of Chatbots

Despite the benefits outlined above, integration of technology in the classroom is not always an easy process. Several models have been created to explain the factors that affect technology adoption. The widely accepted models regarding technology adoption are the Technology Acceptance Model (TAM) (Davis, 1993; Davis et al., 1989), the revised TAM3 (Venkatesh & Bala, 2008), and the unified theory of acceptance and use of technology (UTAUT) (Venkatesh et al., 2003). TAM identifies two major factors influencing the adoption of new technologies: perceived usefulness (PU) and perceived ease of use (PEU) (Davis, 1993; Davis et al., 1989). The revisions made by Venkatesh and Bala (2008) added that perceived risks (PR) also influence the intention to use technology. Finally, UTAUT (Venkatesh et al., 2003) identified that individuals are more likely to adopt a technology when there is support when using the technology. The current literature on chatbots identifies that chatbot adoption in education aligns with the TAM, TAM3, and UTAUT models.

Recommendations

The following recommendations for increased technology adoption are based on the current perceptions of chatbots in education.

Perceived Usefulness

PU is the belief that a particular technological system will be beneficial if adopted, such that the more useful a technology is perceived, the more likely it will be used (Davis et al., 1989). PU has been identified in the literature as a factor determining whether teachers and students adopt chatbots (Chocarro et al., 2021; Malik et al., 2021; Mohd Rahim et al., 2022). The usefulness of AI in education is unfamiliar to some teachers (Hrastinski et al., 2019), and many have had negative experiences using chatbots (Kim & Kim, 2022). It is recommended that continuing education programs be made available for in-service and pre-service teachers outlining the benefits and practical applications of chatbot use.

Perceived Ease of Use

PEU is the degree to which an individual feels like a technology is easy to use (Davis et al., 1989). As the PEU increases, the intention to adopt a technology also increases. This general relationship is also true for chatbots. As PEU increases, the intention to use chatbots by teachers and administrators (Pillai et al., 2023) and post-graduate students increases (Mohd Rahim et al., 2022). In addition, the students surveyed by Mohd Rahim et al. (2022) indicated that if chatbots increased the PEU of other tasks, they would be more inclined to adopt the technology.

Initial use of chatbots can be challenging, and some students may not understand how to prompt them correctly to achieve the desired result (Kaur et al., 2021). In addition to the courses recommended above for educating teachers and students on practical uses of chatbots, hands-on courses should be developed for teachers and students on how to use the technology. Although chatbot technology is novel, PEU may increase over time as the public becomes more accustomed to using the technology.

Facilitating Conditions

Facilitating conditions refer to the degree to which an individual believes that there will be technological support from their system or organization (Chan et al., 2010). Chatterjee and Bhattacharjee (2020), Merelo et al. (2022), and Kim and Kim (2022) noted that teachers will be more likely to adopt chatbots if there is continued support and professional development provided by their organizations for chatbot use. Funding for technology support should be taken into consideration by the administration when deciding whether to adopt chatbot technologies in education. Teachers and students should be provided initial training to increase PEU (Chatterjee & Bhattacharjee, 2020) and have continued support if they require assistance when using chatbots.

Perceived Risk

PR is the perception that adopting a technology, despite having benefits, may also have negative outcomes. As the PR increases when using a technology, the intention to use that technology will decrease (Venkatesh & Bala, 2008). Privacy (Chatterjee & Bhattacharjee, 2020) and security of personal information (Merelo et al., 2022) are both concerns raised by educators. Several teachers are concerned that students will misuse chatbots to plagiarize work (Dehouche, 2021; O’Connor, 2023; Stokel-Walker, 2022; Westfall, 2023). In addition, some researchers are concerned about the spread of misinformation from the text produced by chatbots (Hsu & Thompson, 2023, February 8). ChatGPT is widely considered to be the highest quality chatbot currently available and is only accurate approximately 60% of the time when tested with OpenAI’s internal testing and TruthfulQA’s external benchmarking (OpenAI, 2023a).

The collection of information is necessary for chatbots to function, and the risks involved with using chatbots need to be clearly outlined for teachers. Informed consent in plain language should be addressed prior to the use of chatbots and is currently a concern for the Canadian government (CBC News, 2023). Students and teachers should be educated on the accuracy of the text produced by chatbots and always fact-check the information produced by them.

Conclusion

There are several benefits to using chatbots in education. Chatbots can provide students with immediate feedback on their assessments, analyze student submissions to provide individualized learning materials depending on the student’s needs, and reduce the amount of time required to complete a task by providing quick access to information.

Chatbot use aligns with TAM, TAM3, and UTAUT regarding the adoption of chatbots in education. Chatbots, like many other technologies, are more likely to be adopted in education if they are perceived to be beneficial, easy to use, have technological support, and are low risk. To successfully adopt chatbots in education, courses should be developed to provide in-service and pre-service teachers with the necessary information on how chatbots can be used to benefit learning, how to use chatbots, and the risks associated with chatbot use. In addition, the administration should set up the infrastructure required to provide ongoing technological support for teachers.

More research on AI-driven chatbot models like ChatGPT, Bard, and LLaMa is necessary. One field requiring research and development that will be useful for teachers is the accessibility of fine-tuning LLMs with specific course information. Although methods that require less expensive hardware are being developed (Dettmers et al., 2023), it is still inaccessible to the general public without costly computers. AI and chatbots are continuing to develop at a rapid rate and will undoubtedly be a part of the future. To better prepare students and teachers, education on chatbot use should be integrated into the current curriculums as more research is conducted on best practices.

References

Birhane, A., Kasirzadeh, A., Leslie, D., & Wachter, S. (2023). Science in the age of large language models. Nature Reviews Physics, 5(5), 277-280. https://doi.org/10.1038/s42254-023-00581-4

Cabales, V. (2019). Muse: Scaffolding metacognitive reflection in design-based research. 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290607.3308450

CBC News. (2023, May 25). More Canadian privacy authorities investigating ChatGPT’s use of personal information. https://www.cbc.ca/news/canada/british-columbia/canada-privacy-investigation-chatgpt-1.6854468

Chan, F. K., Thong, J. Y., Venkatesh, V., Brown, S. A., Hu, P. J., & Tam, K. Y. (2010). Modeling citizen satisfaction with mandatory adoption of an E-government technology. Journal of the Association for Information Systems, 11(10), 519-549. https://doi.org/10.17705/1jais.00239

Chang, C.-Y., Kuo, S.-Y., & Hwang, G.-H. (2022). Chatbot-facilitated nursing education: Incorporating a knowledge-based chatbot system into a nursing training program. Educational Technology & Society, 25(1), 15-27. https://www.j-ets.net/collection/published-issues/25_1

Chatterjee, S., & Bhattacharjee, K. K. (2020). Adoption of artificial intelligence in higher education: a quantitative analysis using structural equation modelling. Education and Information Technologies, 25(5), 3443-3463. https://doi.org/10.1007/s10639-020-10159-7

Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., & Xing, E. P. (2023). Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality. https://vicuna.lmsys.org

Chocarro, R., Cortiñas, M., & Marcos-Matás, G. (2021). Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educational Studies, 1-19. https://doi.org/10.1080/03055698.2020.1850426

Cunningham-Nelson, S., Boles, W., Trouton, L., & Margerison, E. (2019). A review of chatbots in education: Practical steps forward. In 30th Annual Conference for the Australasian Association for Engineering Education (AAEE 2019) : Educators Becoming Agents of Change: Innovate, Integrate, Motivate (pp. 299-306). Engineers Australia. https://eprints.qut.edu.au/134323/

Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38(3), 475-487. https://doi.org/10.1006/imms.1993.1022

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003. https://doi.org/10.1287/mnsc.35.8.982

Dehouche, N. (2021). Plagiarism in the age of massive Generative Pre-trained Transformers (GPT-3). Ethics in Science and Environmental Politics, 21, 17-23. https://www.int-res.com/abstracts/esep/v21/p17-23/

Dettmers, T., Pagnoni, A., Holtzman, A., & Zettlemoyer, L. (2023). Qlora: Efficient finetuning of quantized LLMs. arXiv. https://doi.org/10.48550/arXiv.2305.14314

Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv. https://doi.org/10.1101/2022.12.23.521610

Google. (2023, May 10). Google Keynote (Google I/O ’23) [Video]. Google. https://www.youtube.com/watch?v=cNfINi5CNbY

Guo, K., Wang, J., & Chu, S. K. W. (2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666. https://doi.org/10.1016/j.asw.2022.100666

Hrastinski, S., Olofsson, A. D., Arkenback, C., Ekström, S., Ericsson, E., Fransson, G., Jaldemark, J., Ryberg, T., Öberg, L.-M., & Fuentes, A. (2019). Critical imaginaries and reflections on artificial intelligence and robots in postdigital K-12 education. Postdigital Science and Education, 1, 427-445. https://doi.org/10.1007/s42438-019-00046-x

Hsu, T., & Thompson, S. A. (2023, February 8). Disinformation researchers raise alarms about A.I. chatbots. The New York Times. https://www.nytimes.com/2023/02/08/technology/ai-chatbots-disinformation.html

Kaur, A., Singh, S., Chandan, J. S., Robbins, T., & Patel, V. (2021). Qualitative exploration of digital chatbot use in medical education: A pilot study. Digital Health, 7. https://doi.org/10.1177/20552076211038151

Kim, N. J., & Kim, M. K. (2022). Teacher’s perceptions of using an artificial intelligence-based educational tool for scientific writing. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.755914

Malik, R., Shrama, A., Trivedi, S., & Mishra, R. (2021). Adoption of chatbots for learning among university students: Role of perceived convenience and enhanced performance. International Journal of Emerging Technologies in Learning, 16(18), 200-212. https://doi.org/10.3991/ijet.v16i18.24315

Merelo, J. J., Castillo, P. A., Mora, A. M., Barranco, F., Abbas, N., Guillén, A., & Tsivitanidou, O. (2022). Exploring the role of chatbots and messaging applications in higher education: A teacher’s perspective. Learning and Collaboration Technologies: Novel Technological Environments, 205-223. https://doi.org/10.1007/978-3-031-05675-8_16

Microsoft. (2023). Introducing the new Bing. Your AI-powered copilot for the web. Microsoft. https://www.microsoft.com/en-us/bing?form=MA13FJ

Milmo, D. (2023, February 3). ChatGPT reaches 100 million users two months after launch. The Guardian. https://www.theguardian.com/technology/2023/feb/02/chatgpt-100-million-users-open-ai-fastest-growing-app

Mohd Rahim, N. I., A. Iahad, N., Yusof, A. F., & A. Al-Sharafi, M. (2022). AI-based chatbots adoption model for higher-education institutions: A hybrid PLS-SEM-neural network modelling approach. Sustainability, 14(19), 12726. https://doi.org/10.3390/su141912726

Murad, D. F., Irsan, M., Akhirianto, P. M., Fernando, E., Murad, S. A., & Wijaya, M. H. (2019). Learning support system using chatbot in “Kejar C Package” homeschooling program. 2019 International Conference on Information and Communications Technology (ICOIACT), 32-37. https://doi.org/10.1109/ICOIACT46704.2019.8938479

O’Connor, S. (2023). Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66, 103537. https://doi.org/10.1016/j.nepr.2022.103537

Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033

OpenAI. (2023a). GPT-4. OpenAI. https://openai.com/research/gpt-4

OpenAI. (2023b). Models. OpenAI API. https://platform.openai.com/docs/models

Panay, P. (2023, May 23). Bringing the power of AI to Windows 11 – unlocking a new era of productivity for customers and developers with Windows Copilot and Dev Home. Microsoft. https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/

Pichai, S. (2023, February 6). An important next step on our AI journey. Google. https://blog.google/technology/ai/bard-google-ai-search-updates/

Pillai, R., Sivathanu, B., Metri, B., & Kaushik, N. (2023). Students’ adoption of AI-based teacher-bots (T-bots) for learning in higher education. Information Technology & People. https://doi.org/10.1108/ITP-02-2021-0152

Sandoval, Z. V. (2018). Design and implementation of a chatbot in online higher education settings. Issues in Information Systems, 19(4). https://www.researchgate.net/publication/328245964_Designing_Chatbots_for_Higher_Education_Practice

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers and Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862

Stokel-Walker, C. (2022). AI bot ChatGPT writes smart essays – should professors worry? Nature. https://www-nature-com.uproxy.library.dc-uoit.ca/articles/d41586-022-04397-7

Tallvid, M. (2016). Understanding teachers’ reluctance to the pedagogical use of ICT in the 1:1 classroom. Education and Information Technologies, 21(3), 503-519. https://doi.org/10.1007/s10639-014-9335-7

Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., & Hasimoto, T. B. (2023). Alpaca: A strong, replicable instruction-following model. Stanford University Center for Research on Foundation Models. https://crfm.stanford.edu/2023/03/13/alpaca.html

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., & Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv. https://arxiv.org/abs/2302.13971

Venkatesh, V., & Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39(2), 273-315. https://doi.org/10.1111/j.1540-5915.2008.00192.x

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. https://doi.org/10.2307/30036540

Westfall, C. (2023, January 28). Educators battle plagiarism as 89% of students admit to using OpenAI’s ChatGPT for homework. Forbes. https://www.forbes.com/sites/chriswestfall/2023/01/28/educators-battle-plagiarism-as-89-of-students-admit-to-using-open-ais-chatgpt-for-homework/?sh=403f6c69750d

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet? – A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4. https://doi.org/10.3389/frai.2021.654924

World Economic Forum. (2023, May 2). Future of jobs report 2023: Up to a quarter of jobs expected to change in next five years. World Economic Forum. https://www.weforum.org/press/2023/04/future-of-jobs-report-2023-up-to-a-quarter-of-jobs-expected-to-change-in-next-five-years/

Wu, E. H.-K., Lin, C.-H., Ou, Y.-Y., Liu, C.-Z., Wang, W.-K., & Chao, C.-Y. (2020). Advantages and constraints of a hybrid model K-12 E-learning assistant chatbot. IEEE, 8, 77788-77801. https://doi.org/10.1109/ACCESS.2020.2988252

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1-27. https://doi.org/10.1186/s41239-019-0171-0

Zimmerman, J. (2006). Why some teachers resist change and what principals can do about it. NASSP, 90(3), 238-249. https://doi.org/10.1177/0192636506291521