Assessing Listening to the SONG of Life
7.3 Results
Data analysis is based on sixty-nine matched pairs of students who completed the pre-test and post-test L-SONG instrument. Data were analyzed using IBM SPSS software.[1] The results section reports the reliability for each of the four contexts (sub-scales) of L-SONG and the predictive validity of each sub-scale by comparing pre-test and post-test mean scores.
Reliability of L-SONG
Reliability for each of the four sub-scales of L-SONG was computed using Cronbach’s inter-item alpha reliability coefficient. Reliabilities are .71 for the self sub-scale (items 1 through 4 in Appendix B), .76 for others (items 5 through 7), .81 for nature (items 8 through 11), and .91 for Goddess-God-the Divine sub-scale (items 12 through 15). All reliabilities for each of the four subscales of L-SONG exceed the .70 criterion for “respectable” Cronbach alpha reliability” according to standards set forth by Wrench, Thomas-Maddox, Richmond, and McCroskey.[2] None of the items were deleted from the subscales to improve reliability.
Predictive Validity of L-SONG
Four paired sample statistical t-tests between pre-test and post-test mean scores for each L-SONG subscale were computed to determine if the presumed positive gains in student learning are statistically significant at the probability level of less than or equal to .01.[3] See Table 1 for subscale average mean scores for each listening context, individual item means within each subscale, t-values, and Cohen’s d effect sizes.
Table 1*
Pre- and Post-test Means, Gain Scores, t-values, and Effect Sizes for Listening-Song (L-SONG) Subscales, and Pretest and Post-test Means for Individual Items Within Each Subscale.
| Pre-test Mean (SD) | Post-test Mean (SD) | Gain Score Mean (Post – Pre) | t-value (Cohen’s d) | |
|---|---|---|---|---|
| Self | 3.1 (.93) | 4.5 (.86) | 1.38 | 11.35* (1.36) |
| 1 | 3.8 (1.3) | 5.1 (1.1) | 1.3 | |
| 2 | 3.1 (1.4) | 4.7 (1.2) | 1.6 | |
| 3 | 3.0 (1.6) | 4.1 (1.3) | 1.1 | |
| 4 | 2.5 (1.2) | 4.1 (1.0) | 1.6 | |
| Other |
4.5 (.95) |
5.1 (.80) |
0.5 | 5.49* (.66) |
| 5 | 4.2 (1.2) | 4.8 (1.2) | 0.6 | |
| 6 | 4.5 (1.2) | 5.1 (0.8) | 0.6 | |
| 7 | 4.9 (1.2) | 5.4 (0.9) | 0.5 | |
| Nature | 3.74 (1.06) | 4.60 (.79) | .86 | 8.25* (.99) |
| 8 | 4.2 (1.6) | 4.9 (1.0) | 0.7 | |
| 9 | 2.7 (1.2) | 3.7 (1.1) | 1.0 | |
| 10 | 4.1 (1.4) | 4.8 (1.0) | 0.7 | |
| 11 | 3.8 (1.4) | 4.9 (1.0) | 1.1 | |
| Goddess-God-
the Divine |
3.2 (1.28) |
4.2 (1.0) |
1.07 |
9.82* (1.18) |
| 12 | 3.1 (1.4) | 4.2 (1.3) | 1.1 | |
| 13 | 2.7 (1.5) | 3.9 (1.3) | 1.2 | |
| 14 | 3.5 (1.5) | 4.2 (1.2) | 0.7 | |
| 15 | 3.3 (1.4) | 4.5 (0.7) | 1.2
|
|
*Probability level is less than or equal to .01.
Note. The sample size is sixty-nine matched pairs of participants. Individual items are measured on Likert scales (1 is “Very Strongly Disagree” through 6 is “Very Strongly Agree”). SD represents the standard deviation. The t-values are based on paired sample t-tests. Cohen’s d is the effect size of the t-value. Subscales represent the four contexts in listening to the SONG of life (self, others, nature, and Goddess-God-the Divine), and the numbered items are individual statements in the L-SONG measure (see Appendix B for the content of individual items).
Results demonstrate that all four subscales predict gains in student learning from pre-test to post-test. Statistically significant t-values ranged from 5.49 to 11.35 at the .01 probability level. Average positive gains in student learning as measured by the L-SONG subscales were .56 for listening to others, .86 for listening to nature, 1.07 for listening to Goddess-God-the Divine, and a high of 1.38 for listening to self. The magnitude of these gains is calculated using Cohen’s d for effect size and ranged from .66 to 1.36. Wrench, Thomas-Maddox, Richmond, and McCroskey interpret these effect sizes as medium to large in magnitude.[4] Overall, all four L-SONG subscales show acceptable predictive validity.
- SPSS is an abbreviation for "Statistical Package for the Social Sciences." ↵
- Jason S. Wrench, Candice Thomas-Maddox, Virginia P. Richmond, and James C. McCroskey, Quantitative Research Methods for Communication (Oxford: Oxford University Press, 2008), 195. Reliability standards are as follows. A range of .65 through .70 is "minimally acceptable," .70 through .80 is "respectable," and a range of .80 through .90 is considered "good." ↵
- As an exploratory study with four paired t-tests, a stricter (more conservative) alpha level of .01 was used instead of the standard probability level of .05 to reduce the possible number of false positives. ↵
- Jason S. Wrench, et al., Quantitative Research Methods for Communication (Oxford: Oxford University Press, 2008), 349. The standards for the magnitude of effect size are as follows. An effect size of 0.2 is considered small, 0.5 is a medium effect, and over 0.9 is a large effect size. ↵
