7.2 Method

L-SONG Instrument

Likert-type statements were generated from sources used in teaching the course to create an empirical measure of student learning in the Listening to the SONG of Life course.^[1] Fifteen items representing the four contexts in the SONG of Life are represented in the L-SONG instrument. Three to four items for each context are included for assessing internal reliability. The same six-point Likert-type scale^[2] is used for all fifteen items. Items are grouped ^[3] by the four SONG contexts of self (see items 1 through 4 in Appendix B), others (5 through 7), nature (8 through 11), and Goddess-God-the Divine (12 through 15).^[4]

Expert Validity of L-SONG

Expertise in an academic field is partly determined by credibility. McCroskey and Young’s three-decade summary of research on credibility finds competence (education and experience) the most important credibility factor. Accordingly, a credible expert evaluator for this study must have earned their PhD in Communication (educational competence) and published at least one academic journal article or book on a listening topic (experiential competence). Four experts in the field of listening were contacted through e-mail and asked to rate the appropriateness of the fifteen L-SONG items for inclusion in each of the SONG contexts.^[5]

All experts agreed to participate. Experts used a six-point Likert scale^[6] to rate each item on “appropriateness for inclusion” in a given listening context. Average mean scores for each SONG context are 5.0 for the listening to self context, 4.2 for other, 5.5 for nature, and 4.9 for the listening to Goddess-God-the Divine context. An average mean score of “4” or greater indicates experts agree that the items are appropriate for inclusion in a given listening context. All four contexts show average mean scores greater than “4,” indicating that experts agree that the items are appropriate for inclusion. In sum, the four SONG contexts demonstrate a degree of expert validity.

Design and Procedures

Using a pre-test and post-test design, the L-SONG instrument was made available to students in the Author’s Listening to the SONG of Life courses. Students were informed that the L-SONG instrument was not part of their academic grade and that the data would be used for student feedback and research. There was an option for students to opt out of including their data for research purposes and still receive feedback. The College of Arts and Letters Human Subject’s Committee at the Author’s home institution approved the research.

Students completed L-SONG on the first day of class before any instruction as a pre-test, and then a second time on the last day of regular class as a post-test. In principle, potential student learning would be indicated by positive gain scores, measured from pre- to post-test. The instructor left the room while students completed the L-SONG instrument, and a graduate student collected the surveys and turned them in to the department administrative staff, who returned them to the instructor after grades were posted at the end of the term.

Students identified themselves on the surveys with self-selected code names.^[7] In addition, students wrote their code names on a separate page which the graduate student forwarded to the instructor. This sheet of code names was provided to students during the post-test to ensure accurate matching of pre-test and post-test codenames. For pedagogical purposes, all students were e-mailed a summary of the results of pre-post data organized by code name as feedback after posting semester grades.

Students enrolled in the Listening to SONG of Life courses completed pre-test and post-test measures for the L-SONG instrument for five consecutive fall semesters taught by the Author using a similar curriculum. The Listening to the SONG of Life class is cross-listed as an upper-level undergraduate and lower-level graduate course with two to four graduate students and fifteen to twenty undergraduate students enrolled each fall semester. In total, there are sixty-nine matched pairs (pre and post-test) of student responses over five years and twenty-four unmatched surveys (missing either the pre-test or post-test). The high attrition rate of twenty-six percent could be due to many factors. Possible attrition factors are, students adding the class after the pre-testing, students withdrawing from the class before the post-test, and student absenteeism on the first or last day of class.