Abstract

This chapter reports an empirical research project on the design, development, implementation, and evaluation of an original team-based flipped learning (FL) framework, combined with the use of a newly developed application. This combination aimed to increase students’ engagement in both in-class and out-of-class learning activities. It was applied in a university-level English as a foreign language (EFL) course in Japan to improve students’ presentation skills in English through sustained study both in and out of class. The development of the project proceeded as follows: (1) a team-based FL framework was created; (2) an application that allows students to repeatedly practice their presentations with a text-to-speech (TTS) synthesis engine was developed; (3) a function that simultaneously visualizes self-, peer, and teacher assessments using a rubric was created; and (4) another function for small-group discussions outside the classroom was added to the application. When developing stages (1)–(4), the ADDIE model, a practical model for the design and development of instructional materials that centers on all component processes in a holistic systematic planning environment, was applied. Based on the analyses of questionnaire results and students’ performance data, the team-based FL framework through the application engaged the students in in-class and out-of-class learning and can lead to achieving the desired learning outcomes.

Keywords: team-based flipped learning framework, text-to-speech technology, rubric-based self- and peer assessment, out-of-class student interactions, student engagement in learning

Introduction

Student Engagement in Blended Learning

Blended learning (BL) is “the integration of face-to-face and online learning activities” (Vaughan et al., 2013, p. 8); it aims to enhance “engagement through the innovative adoption of purposeful online activities” (Vaughan et al., 2013, p. 9). Dziuban et al. (2013) indicated that BL approaches would provide a suitable environment for enhancing student engagement. However, educational technology research has struggled to identify a common definition for student engagement, and its definition remains inconsistent (Bond et al., 2020). Therefore, it is important to start by defining student engagement to understand how BL affects student engagement (Boekaerts, 2016).

In this study, we adapted Travers’ (2015) definition of student engagement, defining it as a student’s ability to participate in learning opportunities using effective learning processes, based on taking responsibility for their choices. This definition also involves receiving feedback from oneself and other participants to assess personal performance and undertake autonomous actions for self- and peer progress regarding learning targets. Therefore, learning engagement is defined in this paper as communication in the learning process among the participants that can be measured quantitatively and evaluated qualitatively. The participants should be satisfied with the engagement, such that they perceive it to be sufficiently productive while not being overly challenging; thus, engagement would ideally be sustained over the whole course of learning.

Flipped Learning

The flipped learning (FL) approach is a promising response to developments in combination with the BL teaching methodology (Stunz, 2013; Bishop & Verleger, 2013), which has changed conventional classroom learning patterns (Ishikawa et al., 2020). Yarbro et al. (2014) define FL as

a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space, and the resulting group space is transformed into a dynamic, interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter” (p. 5).

FL facilitates active collaborative learning during class time by allowing teachers to respond to individual differences in the comprehension of course content. Furthermore, students are given opportunities to identify learning methods and materials that suit their learning preferences by engaging in small-group discussions in project-based and problem-solving activities. Therefore, FL has the greatest chance of success with small-sized classes that make peer interaction manageable and allow teachers to adopt a coaching role (Ishikawa et al., 2020).

Many studies have reported on the effectiveness of FL (e.g., Gilboy et al., 2015; Hsieh et al., 2017; Hung, 2015; Ishikawa et al., 2017, 2018; Lin & Hwang, 2018; McLaughlin et al., 2014; Nouri, 2016). Vitta and Al-Hoorie (2020) conducted a meta-analysis of 56 language-learning reports involving 61 unique samples and 4,220 participants and demonstrated that flipped classrooms outperformed traditional classrooms. Vitta and Al-Hoorie (2020) also claimed that FL clearly and substantially affected most language outcomes and recommended that future research investigate whether FL is effective and when and how its effectiveness is maximized. Researchers have investigated whether adopting innovative strategies or technologies can make FL more effective (Hwang et al., 2019). Therefore, it is of great importance to redesign lessons to make FL even more effective.

Purpose of the Study

The research project reported in this chapter aimed to investigate approaches that redesign lessons to make FL more effective in a way that promotes student engagement in learning, and results in improving students’ presentation skills in English through sustained study both in and out of class, in university-level English as a foreign language (EFL) course in Japan. To achieve this goal the following steps were completed:

1. a team-based FL framework was created;
2. an application that allows students to repeatedly practice their presentations with a text-to-speech (TTS) synthesis engine was developed;
3. a function that simultaneously visualizes self-, peer, and teacher assessments using a rubric was created;
4. another function for small-group discussions outside the classroom was added to the application.

During the development of stages 1–4 above, the ADDIE model, a practical model for the design and development of instructional materials that centers all component processes in a holistic systematic planning environment, was applied. The ADDIE model has five phases: analysis, design, development, implementation, and evaluation. The outcome of each phase naturally leads to the subsequent step (Gagne et al., 2005).

To evaluate the application and the functions mentioned in stages 2–4, the following hypotheses were set:

1. The implementation of an application that allows students to repeatedly practice their presentations with a TTS synthesis engine, would decrease anxiety regarding presentations among students and increase their self-efficacy for presentations.
2. Peer assessment, rather than self-assessment, would be effective in raising the achievement level of presentations as a learning outcome.
3. If students engaged in discussions in small groups outside the classroom, they would externalize the cognitive processes during these discussions.

Needs Analysis

Student Needs Analysis: A Questionnaire on Students’ Feelings toward Delivering Presentations in English

A questionnaire on students’ feelings toward delivering class presentations in English was administered in 2019. One hundred and sixty-nine students from a university in Japan volunteered to participate in the questionnaire.

The questionnaire consisted of four parts: questions concerning presentation anxiety, presentation self-efficacy, the value of studying at a university, and the number of presentations that the students had made in the past. Regarding the presentation anxiety scale, five items were created referencing the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993) and State-Trait Anxiety Inventory for Children (STAIC; Spielberger et al., 1973). As for the presentation self-efficacy scale, five items were created referencing the MSLQ (Pintrich et al., 1993). Table 6.1 presents the questionnaire items. A 5-point Likert scale (ranging from 1 [strongly disagree] to 5 [strongly agree]) was used for the responses to allow an adequate range of expression for participants’ feelings regarding peer assessments, motivations for continuous learning, and course evaluations.

Table 6.1

Questionnaire Items

To investigate the reliability of the results, alpha (α) coefficients were calculated for each subscale, and the values were sufficient. Therefore, the mean and standard deviations were calculated for each variable. The mean, standard deviation, α coefficient, and the number of people analyzed for each subscale are presented in Table 6.2.

Table 6.2

Mean, Standard Deviation, and α Coefficient of Each Subscale

Correlation analysis was conducted to investigate the relationship between each variable. First, a negative correlation was identified between presentation anxiety and self-efficacy. This implies that the more anxious the students were regarding presenting in English, the lower their confidence. This result was consistent with that of Pintrich et al. (1993), who revealed a negative association between self-efficacy for learning and test anxiety. Second, no correlation was identified between presentation anxiety and the value of studying at a university. This result is consistent with Pintrich and De Groot’s (1990) findings. Finally, self-efficacy did not correlate with the value of studying at a university. The expectation was that self-efficacy would be positively correlated with the value of studying at a university, as in previous studies (Pintrich & De Groot, 1990; Pintrich et al., 1993); however, the results were inconsistent. Thus, the reliability and validity of the scale measuring presentation anxiety and self-efficacy were confirmed.

A positive correlation was identified between presentation self-efficacy and the number of presentations delivered. This revealed that the more presentations the students delivered in English, the more confident they were. Delivering more presentations in English can lead to an increase in confidence. However, there was no correlation between presentation anxiety and the number of presentations. This indicates that the number of presentations the students have delivered in English does not automatically contribute to lowering their anxiety. Therefore, to address the need to help students prepare for presentations and to lower their anxiety, it was deemed necessary to develop an application that would allow students to practice their presentations as much as possible, both inside and outside the classroom.

Student Needs Analysis: A Questionnaire on the Usefulness of Rubric-Based Self- and Peer Assessment

Another questionnaire was administered in 2019 regarding the usefulness of rubric-based self- and peer assessment to develop our project. The participants were 18 out of 21 first-year students who were registered for a collaborative learning course to improve their presentation skills in English at a university in Japan. The questionnaire consisted of three structured questions and one open-ended question. A 4-point Likert scale (ranging from 1 [strongly disagree] to 4 [strongly agree]) was used for the responses.

Among the three structured question items, the item regarding self-assessment had a lower mean and a higher SD than the other two did (Table 3); thus, interviews with five randomly selected students were conducted to ask them how they felt about self-assessment. All five students who participated in the interviews had a negative view of self-assessment. The following are representative student comments on self-assessment: “I am too nervous during the presentation to remember much, so I cannot evaluate objectively,” “I am too positive and sometimes inaccurate, so I cannot rely on self-assessment,” and “I evaluate without properly understanding what points were not good enough”. Many students had a negative view of self-evaluation.

As for peer assessment, the mean value was the highest among the three question items (Table 6.3); many students had a positive view of peer assessment. Representative comments on peer assessment from the open-ended questions were as follows: “I thought it was good to be able to see my presentation and have it evaluated because I can learn in detail what I need to improve that I am not aware of,” and “The audience listens to my presentation, and thus, opinions from such an objective point of view will help me to improve my presentation in the future.”

The above two results demonstrate that peer rather than self-assessment is effective in raising the achievement level of presentations as a learning outcome. The students felt that their peers pointed out exactly how their presentations should be revised. In this study, such needs led to the development of an application that simultaneously visualizes self-, peer, and teacher assessments using a rubric.

Table 6.3

Mean and Standard Deviation of Each Question

Teacher Needs Analysis

In August 2012, the Japanese Ministry of Education, Culture, Sports, Science, and Technology (MEXT) referenced the importance of the role that university education plays in human resource development and stated that a shift to active learning (AL) is necessary to develop human resources who can think independently and solve problems. Mizokami (2018) defined AL as follows:

Active learning includes all kinds of learning beyond the mere one-way transmission of knowledge in lecture-style classes (passive learning). It requires engagement in activities (writing, discussion, and presentation) and externalizing cognitive processes in activities. (p. 79)

Furthermore, Mizokami explained “cognitive processes” as follows:

Cognitive processes refer to the processes of information processing on mental representations using cognition, such as perception, memory, language, and thinking (logical/critical/creative thinking, reasoning, judging, decision-making, problem-solving, etc.). We understand that information processing is performed in process activities such as writing, discussion, and presentation. (p. 79)

At the university with which many of the authors of this chapter are affiliated, it is stated that students should develop presentation skills to summarize and present their ideas while incorporating the opinions of third parties, and the educational method for this purpose is to adopt AL effectively. Moreover, FL is recommended as one of the methods that are capable of transforming the classroom from passive to active; in FL, engaging in activities (writing, discussion, and presentation) to externalize cognitive processes is encouraged. Therefore, the need to combine an original FL framework with a tool for small-group discussions outside the classroom emerged.

Design: Team-Based Flipped Learning Framework

In an attempt to maximize the effectiveness of FL and to meet the teacher’s needs described in the section above, an original team-based FL framework was created. The framework consists of three phases: 1) pre-class, 2) class, and 3) post-class sessions. First, within small groups, students collaborate in learning tasks outside the classroom. Second, after rubric-based self- and peer assessments of their out-of-class activities, students set goals and collaborate in learning tasks within small groups to achieve the goal. Before the class ends, following the self- and peer assessments of their class activities, the students set a new goal for their next out-of-class activities. Third, out of class, students collaborate in learning tasks within small groups to achieve the new goal set at the end of the class session. Figure 6.1 illustrates this team-based FL framework.

Figure 6.1

Team-Based Flipped Learning Framework

Development

Development of the Application

To decrease students’ anxiety regarding presentations and increase their presentation self-efficacy, an application that allows them to repeatedly practice their presentations with a TTS synthesis engine was developed. A function that simultaneously visualizes self-, peer, and teacher assessments using a rubric was created and added to the application to meet the needs stated in the “Needs Analysis” section. Furthermore, another function for small-group discussions outside the classroom was added to the application so that students could discuss in small groups and externalize cognitive processes during these discussions.

The application acts as a learning environment for students and teachers’ mobile devices, such as laptops, tablets, and smartphones. All functions (Figure 6.2) were delivered using an online server. Various data were stored in a MySQL database, including texts, audio files, attachments, and students’ performance data (Figure 6.3).

Figure 6.2

Diagram of the Functional Components in the Application

Figure 6.3

MySQL System Block Diagram

Development of the Course Design

As mentioned earlier, in this study, an application was developed that: 1) supports students’ speaking practice using TTS technology; 2) synchronously visualizes rubric-based assessments made by teachers and students on mobile devices; and 3) enhances engagement in tasks with peers outside of class. The application was used within the team-based FL framework developed for the university EFL course. Figure 6.4 presents where the above three functions are used in the team-based FL framework.

Figure 6.4

Functions Used in the Team-Based Flipped Learning Framework

Note: The three functions in the application, described in the text, are marked on the diagram as 1) = TTS, 2) = rubric, and 3) = discussion board.

The course consisted of four session units, with sessions occurring once per week. The following outline indicates the flow of the learning tasks in the four sessions of one unit:

Session 1

Out-of-class learning task:

1. Students engage in small-group collaborative learning to understand a course text and perform a reading comprehension test to ensure they have it.
2. The teacher sends needs-based messages via an online discussion board during the collaboration process.

In-class learning task:

1. Students reflect on what they did during out-of-class learning tasks using the rubric visualization function of the application.
2. The students collaborate in small groups to write a summary of the text to completely understand it and brainstorm the text’s key topics using a mind map.
3. Each student selects one of the main topics, or a topic relevant to the text, for a presentation in Session 4. They reflect on in-class learning tasks using the rubric visualization function of the application and set a new goal.

Session 2

Out-of-class learning task:

1. Students engage in small-group collaborative learning to share their research findings on each group member’s selected topic on the online discussion board.
2. The students exchange opinions and outline how to present their research findings on the online discussion board.
3. The teacher sends needs-based messages via a discussion board during the collaboration process.

In-class learning task:

1. Students reflect on what they have accomplished in out-of-class learning tasks using the rubric visualization function of the application.
2. The students collaborate in small groups to finalize the outline of their presentation and create drafts of the slide show to be used in their presentation.
3. The students reflect on in-class learning tasks using the rubric visualization function of the application and set a new goal.

Session 3

Out-of-class learning task:

1. Students engage in small-group collaborative learning to revise the slide show and write a script for their presentation on the online discussion board.
2. The teacher sends needs-based messages via a discussion board during the collaboration process.

In-class learning task:

1. Students reflect on what they have accomplished in out-of-class learning tasks using the rubric visualization function of the application.
2. The students collaborate in small groups to finalize the slide show, revise the script for their presentation, and practice their presentation using the TTS function of the application.
3. The students reflect on in-class learning tasks using the rubric visualization function of the application and set a new goal.

Session 4

Out-of-class learning task:

1. Students engage in small groups to practice their presentations using the TTS function of the application.
2. The teacher sends needs-based messages via a discussion board during the collaboration process.

In-class learning task:

1. The students in their groups use the slide show to present their research findings, and after the presentation, they conduct self-assessment using the rubric visualization function of the application.
2. All the other students in the classroom assess their presentation, and the teacher supports the students by giving them comments and advice using the rubric visualization function of the application.

The aforementioned application and course design within the team-based FL framework were implemented in an EFL course at a university in Japan, as described in the “Purpose of the Study” section.

Evaluation

Within the team-based FL framework that was created, 1) an application that allows students to repeatedly practice their presentations with the TTS synthesis engine was developed; 2) a function that simultaneously visualizes self-, peer, and teacher assessments using a rubric was created and added to the application; and 3) another function for small-group discussions outside the classroom was created and added to the application (see Figure 4 in the “Development” section). Following this procedure, validation was conducted for stages 1-3.

(1) Presentation Practice with the TTS Synthesis Engine

It was hypothesized that the implementation of this function would decrease anxiety regarding presentations among students and increase their self-efficacy for presentations, as is indicated in the “Purpose of the Study” section.

To validate the above hypothesis, experimental and control groups were created. Thirty out of the 39 students enrolled in EFL FL courses at a university in Japan in 2020 voluntarily participated in this study. Of the 30 students, 16 were in the experimental group and 14 were in the control group. Pre- and post-questionnaires on students’ anxiety and self-efficacy regarding delivering class presentations in English, which was the same as Table 1 in the “Needs Analysis” section, were conducted. A two-factor analysis of variance (two-way ANOVA) with a mixed design of time (pre and post), group (experimental [n = 16] and control [n = 14] groups), and multiple comparisons (Holm) was conducted.

There was a difference between the groups at the time of the pre-questionnaire, with the control group having a higher mean value than the experimental group (p < 0.05). Additionally, in the experimental group, there was an increase in mean values from the pre- to post-questionnaires (p < 0.05), and the mean value of the control group decreased from pre- to post-questionnaires (p = 0.08; see Table 6.4).

Table 6.4

Mean, Standard Deviation, α Coefficient of Each Subscale, and Results of Two-Way ANOVA

Note: * p < 0.05, ** p < 0.01

A positive correlation was identified between the difference score of presentation self-efficacy and practice time per session (in seconds; r = 0.45, p = 0.08). This result implies that the longer the students practiced with the TTS synthesis engine per session, the higher their self-efficacy tended to be from pre- to post-questionnaires, which supported the above hypothesis (see Ishikawa et al. (2021a) for details).

(2) Rubric-Based Self-, Peer, and Teacher Assessments

This study hypothesized that peer assessment, rather than self-assessment, would be effective in raising the achievement level of presentations as the learning outcome described in the “Purpose of the Study” section.

Fifty-one out of the 57 first- and second-year students enrolled in EFL collaborative learning courses at a university in Japan in 2020 voluntarily participated in this study to improve their English presentation skills. A questionnaire was administered to investigate students’ feelings regarding peer assessments, students’ motivation for continuous learning, and students’ course evaluations (n = 51).

A correlation analysis was conducted to examine the relationships among trust in peer assessments, the usefulness of peer assessments, course evaluations, and motivation for continuous learning. There was a positive correlation between trust in peer assessments and the usefulness of peer assessments (p < 0.001). There was also a positive correlation between the usefulness of peer assessments and course evaluations (p < 0.001) and between the usefulness of peer assessments and motivations for continuous learning (p < 0.001). Moreover, course evaluations were strongly correlated with motivation for continuous learning (p < 0.001).

The results of the correlation analyses revealed that 1) the students who have trust in the peer assessments have higher usefulness of peer assessments; 2) the students who find peer assessments highly useful value the course and are inclined to continue taking the course; and 3) the students who valued the course highly were relatively more motivated to continue taking the course. Thus, the hypothesis was validated by the above three findings (see Ishikawa et al. (2021b) for details).

(3) Discussion Board for Small-Group Discussions

The hypothesis was that if students discussed in small groups outside the classroom, they would externalize cognitive processes during the discussions indicated in the “Purpose of the Study” section.

Twenty-four first-year students enrolled in EFL collaborative learning courses at a university in Japan voluntarily participated in this study. Pre- and post-questionnaires measuring the quality of students’ AL and AL externalization (Mizokami et al., 2016) were conducted at the beginning of April and at the end of July 2017.

As for AL externalization, there was an increase in mean values from pre- to post-questionnaire (p < 0.05). As for AL, there was also an increase in mean values from pre- to post-questionnaire (p < 0.10).

The pre- and post-questionnaire results and analyses of AL and AL externalization indicate that the students increased their ability to externalize their opinions to others, deepened their understanding through the discussion, and noticed that their ideas were different from those of their classmates. Thus, the quality of AL experienced by students was enhanced through discussions.

Spearman’s rank correlation analysis for the frequency of category appearances and the indicators for social and cognitive presence, which was guided by the Community of Inquiry framework (Garrison, 2017) concerning out-of-class student discussions and the pre- and post-questionnaire gains of AL and AL externalization, revealed that the students who were cognitively present during the discussions increased AL and AL externalization, which entailed the externalization of cognitive processes (Mizokami, 2018; Table 6.5). Thus, this hypothesis was supported (see Ishikawa et al. (2020) for details).

Table 6.5

Results of Spearman’s Rank Correlation Analysis

Summary of the Findings

The three hypotheses of this study— 1) the implementation of the application that allows students to repeatedly practice their presentations with a TTS synthesis engine would decrease anxiety regarding presentations among students and increase their self-efficacy for presentations, 2) peer assessment, rather than self-assessment, would be effective in raising the achievement level of presentations as the learning outcome, and 3) if students discussed in small groups outside the classroom, they would externalize cognitive processes during these discussions—were answered positively.

1. The results of the two-way ANOVA and correlation analysis between the difference score of presentation self-efficacy and the practice time per session revealed that practicing presentations using the TTS synthesis engine decreased students’ anxiety regarding presentations and increased their self-efficacy.
2. The results of the correlation analyses indicated that peer assessment, rather than self-assessment, would be effective in raising the achievement level of presentations as a learning outcome.
3. The results of the Spearman’s rank correlation analysis for the frequency of category appearances and the indicators of social and cognitive presence concerning out-of-class student discussions and the pre- and post-questionnaire gains of AL and AL externalization indicated that small-group discussions through the discussion board increased engagement outside the classroom and led to externalization of cognitive processes.

Conclusion

This chapter reports an empirical research project on the design, development, implementation, and evaluation of an original team-based FL framework that aims to achieve students’ engagement in both in-class and out-of-class learning activities. To achieve student engagement in the learning process, a team-based FL framework was created. Furthermore, an application was developed that allows students to repeatedly practice their presentations with a TTS synthesis engine, simultaneously visualizing self-, peer, and teacher assessments using a rubric, and allows students to discuss outside the classroom. Based on the analyses of the questionnaire results and students’ performance data, the team-based FL framework through the application engaged the students in learning both in- and out-of-class and can lead to achieving the desired learning outcomes.

The team-based FL framework presented in this chapter (see Figure 1 in the “Design” section) is highly generalizable. This framework can be implemented not only in EFL teaching/learning but also in other disciplines. For example, in project-based learning and problem-solving learning, by adopting this framework for small-group student activities, students will be encouraged to efficiently and effectively fulfill projects and identify methods to solve problems through discussions among themselves. Further research across disciplines is needed to validate this framework.

Recently, various strategies and technologies have been adopted to make FL more effective and efficient. In this context, the following points should be considered when implementing FL-based learning environments. The first is to clarify the purpose of the FL. FL is a very efficient and effective teaching method. However, this is only one method of teaching. If students do not clearly understand the skills they will acquire through FL, then it will be a mere facade. The second point is that the key to success in flipped classroom teaching lies in students’ attitudes toward learning. The required attitude is displayed by students who can take part in learning opportunities through effective learning processes based on taking responsibility for their choices; this attitude will lead to students’ engagement in learning.

References

Boekaerts, M. (2016). Engagement as an inherent aspect of the learning process. Learning and Instruction, 43, 76-83. https://doi.org/10.1016/j.learninstruc.2016.02.001

Bond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., Kerres, M. (2020). Mapping research in student engagement and educational technology in higher education: A systematic evidence map. International Journal of Educational Technology in Higher Education, 17(2). https://doi.org/10.1186/s41239-019-0176-8

Dziuban, C., Graham, C., Picciano, A. G. (2013). Research Perspectives in Blended Learning, (2nd ed.). Routledge.

Gagne, R. M., Wager, W. W., Golas, K. C., & Keller, J. M. (2005). Principles of instructional design (5th ed.). Wadsworth Publishing.

Garrison, D. R. (2017). E-learning in the 21st century: A framework for research and practice, (3rd ed.). Routledge.

Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109-114.

Halverson, L.R., & Graham, C.R. (2019). Learner engagement in blended learning environments: A conceptual framework. Online Learning, 23(2), 145-178. doi:10.24059/olj.v23i2.1481

Hsieh, J. S. C., Wu, Wen-Chi V., Marek, M. W. (2017). Using the flipped classroom to enhance EFL learning. Computer Assisted Language Learning, 30(1-2), 1-12.

Hung, Hsiu-Ting. (2015). Flipping the classroom for English language learners to foster active learning. Computer Assisted Language Learning, 28(1), 81-96.

Hwang, G.-J., Yin, C., & Chu, H.-C. (2019). The era of flipped learning: Promoting active learning and higher-order thinking with innovative flipped learning strategies and supporting systems. Interactive Learning Environments, 27(8), 991-994.

Ishikawa, Y., Akahane-Yamada, R., Smith, C., Murakami, M., Kondo, M., Kitamura, M., Tsubota, Y., & Dantsuji, M. (2017). A flipped learning approach to university EFL courses. In M. Khosrow-Pour (Ed.), Encyclopedia of information science and technology (4th ed.), (pp. 3850-3860). Information Science Reference.

Ishikawa, Y., Tsubota, Y., Smith, C., Murakami, M., Kondo, M., & Tsuda, M. (2018). Integrating online and offline student collaboration in EFL flipped learning courses. In A. Palalas (Ed.), Blended language learning: International perspectives on innovative practice, (pp. 303-328). China Central Radio & TV University Press.

Ishikawa, Y., Tsuboto, Y., Umemoto, T., Murakami, M., Kondo, M., Suto, A., & Nishiyama, K. (2020). Building student interactions outside the classroom: Utilizing a web-based application in a university flipped learning course for EFL learners. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies: Designing, developing, and deploying learning experiences, (pp. 326-338). Springer Nature Switzerland AG.

Ishikawa, Y., Takamichi, S., Umemoto, T., Aikawa, M., Sakamoto, K., Yui, K., Fujiwara, S., Suto, A., & Nishiyama, K. (2021a). Japanese EFL learners’ speaking practice utilizing text-to-speech technology within a team-based flipped learning framework. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies: New challenges and learning experiences, (pp. 283-291). Springer Nature Switzerland AG.

Ishikawa, Y., Umemoto, T., Tsuboto, Y., Fujiwara, S., Suto, A., & Nishiyama, K. (2021b). Achieving student engagement in learning: Utilizing a rubric-based assessment system for visualizing learners’ self-, peer, and teacher assessments. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies: New challenges and learning experiences, (pp. 292-300). Springer Nature Switzerland AG.

Mizokami, S. (2018) Deep active learning from the perspective of active learning theory. In K. Matsushita (Ed.) Deep active learning: Toward greater depth in university education, (pp. 79-91). Springer Nature.

Mizokami, S., Mori, T., Konda, H., Kawai, T., Miho, N., Honda, S., & Yamada, Y. (2016). Bifactor model niyoru active learning (externalization) shakudo no kaihatsu [Developing the active learning (externalization) scale by bifactor model], Kyoto University Studies in Higher Education, 22, 151-162.

Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813.

Spielberger, C. D., Edward, C. D., Lushene, R. E., Montuori, J., & Platzek, D. (1973). STAIC preliminary manual for the state-trait anxiety inventory for children (“How I feel questionnaire”). Consulting Psychologists Press.

Stuntz, D. F. (2013). Flipped classrooms and CALL sustainability: A rationale for the development of flipped classrooms for sustainable CALL. Global perspectives on Computer-Assisted Language Learning Gloasgow, 10-13 July 2013 Papers, 323-326.

Vaughan, N. D., Cleveland-Innes, M., & Garrison, D. R. (2013) Teaching in blended learning environments: Creating and sustaining communities of inquiry. Athabasca University Press.

Vitta, J. P. & Al-Hoorie, A, H. (2020). The flipped classroom in second language learning: A meta-analysis, Language Teaching Research, 1-25. https://doi.org/10.1177/1362168820981403

Yarbro, J., Arfstrom, K. M., McKnight, K., & McKnight, P. (2014). Extension of a review of flipped learning. [PDF file]. https://flippedlearning.org/wp-content/uploads/2016/07/Extension-of-FLipped-Learning-LIt-Review-June-2014.pdf