4.1 Results
Internal reliability was confirmed using Cronbach’s Alpha with the first survey. As shown in Table 1, acceptable reliability levels were achieved, with α = 0.71 for PCS items (n = 4) and α = 0.81 for IMI items (n = 6). This allowed us to compute composite PCS and IMI scores for each participant by using the median for each respective scale.
A between-group analysis of the initial survey (Time 1) confirmed no significant differences in perceived competence between the two groups before the lectures (see Table 2). Group 1 had a rank of 30.81, and Group 2 had a rank of 35.72 for the PCS score. For the IMI score, Group 1 had a rank of 32.19, and Group 2 had a rank of 34.00. Mann-Whitney U results revealed no significant difference between the groups for PCS (p = 0.28) or IMI (p = 0.69).
After the lectures (Time 2), both PCS and IMI scores increased in both groups, suggesting the lectures had a positive impact on participants’ perceived competence. Mann-Whitney U tests revealed an insignificant difference between the groups: PCS Mean rank for Group 1 was 37.36, and for Group 2, 31.47. IMI ranks were 36.81 for Group 1 and 32.05 for Group 2, with p-values of 0.20 (PCS) and 0.30 (IMI). These results indicate that the use of clickers (Group 2) did not significantly impact perceived competence compared to the traditional lecture format (Group 1) during the first phase of the GAS.
However, a significant difference was observed at the end of the course (Time 3) for IMI (p = 0.05) but not for PCS (p = 0.06). A post-hoc effect size analysis revealed a small effect size (r = 0.24) between the groups. For PCS, Group 1 had a Mean rank of 37.48, while Group 2 had a Mean rank of 28.91. For IMI, Group 1’s Mean rank was 37.65, and Group 2’s was 28.76. These results suggest that the learning experience, rather than the lectures alone, may have had a greater influence on Group 2's perceived competence (IMI) compared to Group 1.
Within-group analyses using Wilcoxon Signed-Ranks tests showed differing results between the groups (see Tables 3 and 4). For Group 1, Time 2 PCS scores were significantly higher than Time 1 scores (Z = 2.08, p = 0.01), but no significant difference was found for IMI (Z = 1.89, p = 0.06). An effect size of r = 0.48 suggests a moderate impact of the lectures on perceived competence. For Group 2, no significant difference was found between Time 1 and Time 2 for PCS (Z = 1.80, p = 0.07) or IMI (Z = 1.30, p = 0.19).
At the end of the semester (Time 3), the within-group analysis showed significant increases in both PCS and IMI scores for Group 1. Time 3 PCS ranks were significantly higher than Time 2 (Z = 3.13, p = 0.00), and IMI scores also increased significantly (Z = 3.51, p = 0.00). Effect sizes were medium for PCS (r = 0.53) and large for IMI (r = 0.63), suggesting that the language learning experience in the GAS had a substantial impact on Group 1’s perceived competence.
For Group 2, significant increases were also observed from Time 2 to Time 3 for both PCS (p = 0.04) and IMI (p = 0.00), with medium effect sizes for PCS (r = 0.36) and IMI (r = 0.58). These results suggest that for both groups, it was most likely the language learning experience in the GAS, rather than the lectures themselves, contributed to the increases in perceived competence.
However, to further explore the relationship between perceived competence and retention of lecture content, as shown in Table 5, Levene’s Test for equality of variances was administered. It then confirmed homogeneity of variance for both Group 1 (F(1,93) = 0.91, p = 0.41) and Group 2 (F(1,92) = 0.18, p = 0.83).
Despite an increase in the Mean scores for both groups at Time 2, t-tests revealed no significant difference in retention between the groups. Group 1’s Mean score was 3.86 (SD = 1.52), while Group 2’s was 4.00 (SD = 1.39) after the lectures (t(63) = 0.04, p = 0.71). Similarly, by Time 3, Group 1’s Mean score decreased slightly to 4.59 (SD = 1.59), while Group 2’s increased to 5.00 (SD = 1.59), but again, no significant difference was found between the groups (t(63) = 1.29, p = 0.20).
Overall, it was found that the clicker intervention did not significantly enhance retention rates compared to the traditional lecture format. Paired-sample t-tests conducted within groups confirmed no significant differences in test scores between Time 1 and Time 2 or Time 2 and Time 3.
Finally, an examination of whether perceived competence (PCS and IMI) correlated with retention of lecture content using Spearman’s Rho was conducted (Table 6). No significant relationships were found between test scores and perceived competence at Time 3 for either group. These findings suggest that perceived competence was more likely maintained through the overall learning experience in the GAS rather than through retention of lecture content as discussed in Hiromori (2006) in which the limitations of the school or class in the EFL context where the students most likely gain a sense of being competent in the learning as they progress, hence internalizing their motivation to learn.
4.2 Discussion
This study sought to explore the effects of lectures aimed at providing students with a deeper understanding of the language learning process to measure its influence on satisfying the need of competence within the SDT framework. Also, the role of clickers in content retention was examined to explore whether retention of content contributes to satisfaction of this need. While the results showed that lectures alone did not significantly increase perceived competence, a more nuanced examination of the data suggested that the overall learning experience in an autonomy-supportive learning environment the GAS aims to provide had a stronger influence on satisfying the need for competence.
In our examination of RQ1, findings indicate that lectures, by themselves, did not result in a significant increase in perceived competence for either group. For the science and math education majors, or Rikei students, perceived competence increased significantly after the lectures, whereas Japanese language education majors, or Bunkei students, showed no significant increase until later in the course, during the more autonomous phase of the GAS.
To explain this difference, it is essential to clarify the nature of the learning experience within the GAS. In the GAS, after the initial lectures, students engaged in a self-directed learning process. This process involved setting their own learning goals, creating personalized study plans, and reflecting on their progress (Fukuda, Sakata, Pope, 2019). Rikei students may have benefited more from the language learning lectures because of their pre-existing negative self-perceptions regarding language learning. The lectures may have helped them understand that language learning is not solely based on innate talent, but rather on structured practice and the development of learning strategies, thus a deeper understanding of this may have fostered their perceived competence in language learning after the lectures. For Bunkei students, who already had stronger language backgrounds, data suggested that it was the actual experience of applying their learning in the GAS that most likely led to fostering their perceived competence in the later phases of the GAS. In sum, data suggests that while lectures aim to deepen students’ understanding of the language learning process, they need to be complemented by practical, autonomy-supportive learning experiences to have a meaningful impact on students’ perception of their own competence in language learning.
In interpreting data to answer RQ2, the use of clickers did not result in a significant improvement in content retention compared to the non-clicker group. This finding contradicts some prior research on the benefits of digital tools for retention. However, a deeper analysis suggests that the interactive nature of the lectures themselves may have already been sufficient to promote engagement, reducing the added value of the clickers. Moreover, it is possible that the novelty of using clickers, as noted by the participants, might have initially engaged students but did not translate into long-term retention benefits including the testing effect. Future studies could explore how integrating clickers more systematically across the course, rather than in isolated lectures, might produce more lasting effects on retention.
In the lectures, students were encouraged to engage in peer discussions, process their answers, and receive immediate feedback on their understanding. These activities involved elements of productive failure, where students were intentionally led to make mistakes in their initial answers, which were then corrected through class discussions. This peer-driven, active learning approach likely provided enough cognitive stimulation to enhance engagement and learning, regardless of the use of clickers. The clickers, while useful for anonymous participation, may not have significantly added to the already interactive lecture format and better learning outcomes as discussed in the literature review.
To answer RQ3, data showed no significant relationship between retention of lecture content and perceived competence. These results suggested that retaining knowledge from the lectures in language learning did not directly influence students' perceived competence in language learning. Instead, perceived competence, again, seemed to be more strongly tied to the broader learning experience in the GAS.
In the GAS, students were given the opportunity to take ownership of their learning by setting personalized goals and engaging in independent study. This autonomy-supportive learning environment likely had a greater influence on their perceived competence than the specific content of the lectures. The process of making decisions about their learning paths, reflecting on their progress, and experiencing success in achieving self-determined goals may have contributed more to their confidence as language learners than simply retaining lecture content or the lecture itself.
Overall, these findings align with previous research in GAS, which emphasizes the importance of autonomy and relatedness in fostering intrinsic motivation (e.g., Fukuda et al., 2011, Fukuda et al., 2016). In this study, while the lectures in language learning may have provided important foundational knowledge, it was most likely the active, autonomous learning experience that had the most profound impact on maintaining perceived competence over time.
Regardless, this study should be interpreted considering several limitations. Future research should further investigate how different types of learning experiences emphasizing autonomy or foundational subject knowledge influence the basic psychological needs, and whether integrating digital tools more holistically throughout a course might yield more consistent benefits. Additionally, examining the role of student understanding of the language learning process over a longer timeframe, opposed to two lectures in the beginning of the course, could provide insights into how lectures in language learning might set the stage for successful learning outcomes in EFL contexts.
Expanding the number of items in both the PCS and IMI could provide a more comprehensive assessment of students’ perceptions and motivation. Future iterations of this study may incorporate the full version of these scales to ensure a broader representation of intrinsic motivation and all three basic needs. Furthermore, the novelty effect of using clickers for the first time, as well as the Hawthorne effect should not be overlooked. These factors may have contributed to increased engagement or inflated PCS scores immediately after the lectures.
Additionally, the tests used to measure retention relied on a 0- or 1-point scale, which may not have fully captured participants' retention of lecture content. Binary Yes/No answers can be difficult to interpret, as they may reflect opinion rather than true knowledge retention. Future studies could use more nuanced assessment methods, such as open-ended questions or scaled responses, to provide a clearer understanding of retention and knowledge acquisition. Finally, this study should also be viewed as exploratory, given the specific context of education majors and the relatively small sample size. Research involving larger, more diverse cohorts from different academic disciplines would be necessary before any broad generalizations can be made. Measuring perceived competence and retention a year later, or even by the time students graduate, would offer insights into the long-term effects of the GAS.