Muammer
Çalik
*a,
Neslihan
Ültay
b,
Hasan
Bağ
c and
Alipaşa
Ayas
d
aDepartment of Elementary Teacher Education, Fatih Faculty of Education, Trabzon University, Trabzon, Türkiye. E-mail: muammer38@hotmail.com
bDepartment of Elementary Teacher Education, Faculty of Education, Giresun University, 28200 Giresun, Türkiye. E-mail: neslihanultay@gmail.com
cDepartment of Elementary Teacher Education, Recep Tayyip Erdoğan University, Rize, Türkiye. E-mail: hsnbag@gmail.com
dFaculty of Education, Bilkent University, Ankara, Türkiye. E-mail: apayas@bilkent.edu.tr
First published on 5th January 2024
The purpose of this study is to meta-analytically evaluate research that used chemical bonding-based interventions to improve academic performance. Through meta-analysis, the present study used several keyword patterns (e.g., chemical bonding, experimental, chemistry education, science education) via relevant databases (e.g., ERIC, Springer Link, Taylor & Francis, Wiley Online Library Full Collection, and Scopus) to find chemical bonding-intervention studies. Thus, it included 50 chemical bonding-based intervention papers (15 dissertations, 32 articles, and 3 proceedings). The current meta-analysis found that the overall effect-size of chemical bonding-based intervention studies was 1.007, which shows a large effect. Findings regarding moderator analysis displayed non-significant differences between educational levels and a statistically significant difference between the intervention types. This meta-analysis reveals that the chemical bonding-based intervention studies are effective at improving the participants’ academic performance in terms of chemical bonding. Further, it denotes that when the abstract nature of chemical bonding is overlapped with the features of the intervention type, the interventions (e.g., cooperative learning and enriched learning environment with different methods) result in better academic performance. Since this study, like all meta-analyses, points out consistent and inconsistent findings among published research, further meta-analysis studies should be undertaken to resolve any contradictory findings.
However, previous studies have reported that students have difficulties understanding the concept of chemical bonding or hold superficial understanding of this topic because of its abstract nature (Özmen, 2004; Ünal et al., 2006; Nahum et al., 2010; Hunter et al., 2022; van Dulmen et al., 2023). Moreover, the concept also includes other complex concepts such as covalent bonds, molecules, ions, giant lattices, and hydrogen bonds that necessitate using the interplay amongst macroscopic, sub-microscopic and symbolic aspects of chemistry/science (Ünal et al., 2006). Further, fully understanding these concepts requires students to be familiar with mathematical and physical concepts and laws that are associated with such key bonding concepts as orbital, electronegativity, electron repulsions, polarity, and Coulomb's law (Nahum et al., 2010). Given its importance in chemistry learning and teaching, the Chemical Bond Approach (CBA), which was released in the USA in the 1950s, has handled chemical bonds as a central topic in teaching chemistry.
The adverse effect(s) of singular and simplistic approaches on students’ conceptual understanding and alternative conceptions of chemical bonding makes teaching chemical bonding difficult (Nahum et al., 2007; van Dulmen et al., 2023). Thus, teachers need to challenge these adverse effects, difficulties and alternative conceptions to enrich their science teaching of chemical bonding and enhance student understanding of chemical bonding (Ünal et al., 2006). Therefore, students’ difficulties or alternative conceptions of chemical bonding have drawn science/chemistry educators’ and teachers’ attention to how to improve chemistry learning and overcome them.
Studies report that the traditional approach to teaching bonding is problematic and often result in misconceptions (Özmen, 2004; Ünal et al., 2006; Nahum et al., 2010; Hunter et al., 2022; van Dulmen et al., 2023). What is more, it directs students to memorize key phrases and facts instead of developing a deeper conceptual understanding of the topic (Nahum et al., 2010; Hunter et al., 2022). Further, it does not help students visualize the abstract nature of chemical bonding, link sub-microscopic, macroscopic, and symbolic aspects of chemistry with each other and meaningfully construct the interplay and interconnections between core concepts (e.g., the particulate nature of matter, atomic structure, and chemical bonding) (e.g., Coll and Treagust, 2002; Ünal et al., 2006; Hunter et al., 2022). Also, when teachers’ content knowledge and pedagogical content knowledge of chemical bonding are limited or superficial, they generally prefer the traditional approach instead of alternative pedagogical ones taking students’ alternative conceptions and difficulties into account and may cause the development of normative ideas about bonding and bonding models (Hunter et al., 2022). For example, when teachers oversimplify octet rule by using improper language that atoms “want” or “need” an octet, their students use similar statements to explain chemical bonding (Hunter et al., 2022). Therefore, chemistry educators have attempted to develop more effective, pedagogically and scientifically sound strategies to teach the concepts of chemical bonding (Teichert and Stacy, 2002; Nahum et al., 2010). Notably, intervention studies have also tested varied methods, strategies, models, and approaches to enable students to grasp a deep conceptual understanding of chemical bonding (Özmen, 2004; Ünal et al., 2006; Nahum et al., 2010; Hunter et al., 2022; van Dulmen et al., 2023). Examples of these interventions include computer-assisted instruction, concept maps, conceptual change, constructivist learning environments, cooperative learning, enriched learning environments with different methods, inquiry-based learning, multiple representations, context-based learning, and problem-based learning. These interventions point how pedagogical tools or preferences of chemical bonding depend on different contexts (e.g., philosophy or theoretical framework of national chemistry curriculum, students’ preparedness, and familiarity with the preferred intervention) (van Dulmen et al., 2023). To gain insights into these variations and implications for practice, a few researchers have systematically reviewed past studies to look for patterns and procedures. The next section provides information about these prior systematic reviews of chemical bonding studies.
The aforementioned systematic reviews provided a rich corpus of chemical bonding education that not only incorporates invaluable insights, arguments, and frameworks on how to teach chemical bonding but also illuminates future and further research. However, none of them has used a standard measurement value (e.g., effect-size or Hedges’ g or Cohen d) to explore the effectiveness of chemical bonding-based intervention studies and compare the studies with each other. For example, van Dulmen et al. (2023)'s emphasis on the positive effects of cooperative learning and group work on students’ conceptual understanding/achievement of chemical bonding calls for a meta-analysis study that examines the extent to which these intervention types improve students’ academic performance. Therefore, the current study aims to address this shortcoming with a supplemental and enriched approach to conducting a meta-analysis.
As chemical bonding is a milestone for learning advanced and related science/chemistry concepts, it is generally taught at the high school and university levels (Fensham, 1975; Nahum et al., 2010; Hunter et al., 2022). Meanwhile, some science curricula (e.g., the Turkish science curriculum) introduce fundamental concepts of chemical bonding at middle school (Ministry of National Education, 2018). As a matter of fact, previous studies have studied with different samples from middle school to university. This also appears the question “Which educational level is better for teaching it?”, which directs the current meta-analysis to handle educational level as a moderator variable.
Luckily, the related literature has incorporated five systematic reviews of chemical bonding (Özmen, 2004; Ünal et al., 2006; Nahum et al., 2010; Hunter et al., 2022; van Dulmen et al., 2023). However, none of them has recruited a standard measurement value (e.g., Hedges’ g) along with a meta-analysis. For this reason, the current study fills in an important gap in the related literature by focusing on meta-analysis, different databases, updated year coverage, and various moderator variables. Thus, it provides invaluable results for chemistry educators, chemistry teachers, curriculum developers, and decision makers on the practical significance of chemical bonding-based intervention studies via evidence with a standard measurement value (e.g., Hedges’ g). For example, it would illuminate the extreme values in chemical bonding-based interventions that can be deeply investigated by future research. Also, given the consistency and efficiency of the chemical bonding-based interventions, it would inform all stakeholders about how to effectively use time, effort, and budget for chemistry education and learning chemistry. For instance, researchers and teachers may seek alternative pedagogical approaches to better teach ‘chemical bonding’ in place of the tested ones. Similarly, chemistry/science educators, teachers, and curriculum developers may have an opportunity to easily access a pedagogical repertoire of chemical bonding and decide which of the intervention types matches better with its nature. Handling the ‘educational level’ as a moderator variable may also give some insights about relative efficacy of interventions in different educational levels (middle school, high school and university). To sum up, this study may shed more light on future decisions and discussions about the effectiveness of chemical bonding-based interventions and the development of guidelines and frameworks for teaching chemical bonding.
● How effective are chemical bonding-based intervention studies on academic performance?
● How do moderator variables (namely, educational level and type of intervention) affect the participants’ academic performance?
● Pattern 1: chemical bonding, experimental and chemistry education or science education
● Pattern 2: chemical bonding, treatment and chemistry education or science education
● Pattern 3: chemical bonding, intervention and chemistry education or science education
● Pattern 4: intra-molecular forces, experimental and chemistry education or science education
● Pattern 5: intra-molecular forces, treatment and chemistry education or science education
● Pattern 6: intra-molecular forces, intervention and chemistry education or science education
● Pattern 7: intermolecular forces, experimental and chemistry education or science education
● Pattern 8: intermolecular forces, treatment and chemistry education or science education
● Pattern 9: intermolecular forces, intervention and chemistry education or science education
Also, they manually searched the related journals and dissertations by examining the references of recently published papers. As a result of these searches, the authors found 146 potential studies for the meta-analysis. The next step was to critique the papers to ensure they complied with the purpose of the study (see Fig. 1 for a flow chart of this selection process).
The critique of the original set of the papers began with a review to organize the selection process and avoid duplicates. For this review, the authors labelled each study as an in-text citation format. This labelling also helped to determine some papers that were indexed in more than one database or some dissertations that were published in journals as research papers. Therefore, the authors removed 30 papers because of duplications in databases, and research papers from the dissertations.
After the duplicates were removed, the authors assessed eligibility of the papers by carefully reading the full-texts of the papers and applying the inclusion criteria (chemical bonding-based intervention, learning outcomes—conceptual understanding and achievement, quasi- and true-experimental designs, and publication language—Turkish and English). Thereby, the authors excluded 55 papers with pre-experimental design (e.g., Wang, 2007; Zohar; 2020; Lutfi et al., 2021), different dependent variables (e.g., scientific attitude and meanings/definitions of modelling and models) (for example, Barnea and Dori, 1999; Astuti et al., 2019) and different languages (e.g., Heprew) (for example, Frailich, 2007; Fridman, 2020).
Eligibility was also assessed by examining the statistical data of each paper to calculate any effect-size(s). This examination eliminated 11 papers, that lacked sufficient data (e.g., mean, standard deviation, sample size, paired p-value, paired t-value) for meta-analysis. That is, some of the papers only provided either the findings of non-parametric analysis (e.g., mean rank, sum of ranks, and U-value) or qualitative data from pre- and post-interview or limited descriptive statistical results (e.g., frequency and percentage) from open-ended questions. For example, given unclear sample sizes of the experimental and control groups, the authors eliminated Salyani et al. (2020)'s research from the corpus studies.
Nine of the papers contained:
● multiple experimental groups with different cohorts (Ekinci, 2010; Singh and Moono, 2015; Sunyono and Meristin, 2018);
● different modules (subtopics) for home and jigsaw groups (Doymus, 2008);
● different instructional designs (Ulusoy, 2011; Karacop and Doymus, 2013; Korkman, 2018);
● different instructional stages (beginning and end of the course) (Sevim, 2007);
● different parts of the assessment instrument (Tsaparlis et al., 2018).
Since six studies (Sevim, 2007; Doymus, 2008; Ekinci, 2010; Singh and Moono, 2015; Sunyono and Meristin, 2018; Tsaparlis et al., 2018) carried out the same teaching design for their experimental groups, the authors calculated combined effect-sizes for them by using ‘the study as the unit of analysis’ and ‘subgroups within the study’ options via Comprehensive Meta-Analysis (CMA) statistics software. However, because three of them (Ulusoy, 2011; Karacop and Doymus, 2013; Korkman, 2018) carried out different intervention types for their experimental groups, they were viewed as individual studies. For example, the authors handled Karacop and Doymus (2013)'s study as two individual studies in that it included cooperative learning and computer-assisted instruction for their experimental groups as types of intervention. Overall, the current meta-analysis attempted to minimally decrease the number of individual studies to deal with the inflating impacts of multiple experimental studies on the overall effect-size.
● 43 individual studies (from 41 papers, some of which included several experimental groups and were imported as individual studies) contained mean scores, standard deviations, and sample sizes for the experimental and control groups.
● Four studies (four papers) included the mean scores and sample sizes of the experimental and control groups and independent groups p-values.
● Three studies (three papers) supplied mean scores and sample sizes for the experimental and control groups and independent groups t-values.
● Three studies (two papers, one of which incorporated several experimental groups and was imported as individual study) reported differences in means, sample sizes for the experimental and control groups, and independent group p-values.
Model | Number of studies | Effect-size and 95% confidence interval | Test of null (2-tail) | Heterogeneity | Tau-squared | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Point estimate | Standard error | Variance | Lower limit | Upper limit | Z-Value | P-Value | Q-Value | df (Q) | p-Value | I-Squared | Tau-squared | Standard error | Variance | Tau | ||
Fixed | 53 | 0.727 | 0.026 | 0.001 | 0.677 | 0.777 | 28.466 | 0.000 | 606.536 | 52 | 0.000 | 91.427 | 0.409 | 0.167 | 0.028 | 0.639 |
Random | 53 | 1.007 | 0.094 | 0.009 | 0.822 | 1.192 | 10.660 | 0.000 |
The second research question called for two moderator variables: educational level and type of intervention. The former includes the following grades: K-8 (kindergarten to grade 8), high school (grades 9–12) and university (undergraduate or bachelor). The studies mentioned ‘types of intervention’ such as computer-assisted instruction, concept maps, conceptual change, constructivist learning environments, cooperative learning, and enriched learning environments with different methods, inquiry-based learning, and multiple representation (see Appendix).
The results of the funnel plot (see Fig. 2) display an asymmetrical structure between standard error and effect-size (e.g., Karadag, 2020). This means that the papers with a larger standard error (smaller sample size) might have had larger effects (Rahman and Lewis, 2020). This could be considered as evidence of publication bias. Likewise, the findings of the Duval and Tweedie's trim and fill test (see Table 2) pointed to a difference between the observed (1.007 at 95% confidence interval of 0.822–1.192) and adjusted values (1.219 at 95% confidence interval of 0.997–1.441) that supports the evidence of publication bias in the funnel plot. As seen from Fig. 3 (black dots in the funnel plot), ten studies (as a counterpart data) are needed to offset this asymmetry or cope with the effect of observed asymmetry on the overall effect-size.
Studies trimmed | Point estimate | Confidence interval (CI) | Q value | ||
---|---|---|---|---|---|
Lower limit | Upper limit | ||||
Observed values | 1.007 | 0.822 | 1.192 | 606.536 | |
Adjusted values | 10 | 1.219 | 0.997 | 1.441 | 1575.476 |
Fortunately, the difference between the observed and adjusted estimates was 17.39%, which falls into the negligible range (20%) reported by Kepes et al. (2012), Vevea et al. (2019) and Chang et al. (2022). This means that the publication bias could be tolerable for the corpus of data even though some evidence of publication bias was found in the relevant data. However, prior to making a decision about it, the Classic fail-safe N (N = 3013 for p-value of zero and alpha of 0.05) (see Table 3) and Orwin's fail-safe N (N = 333 for the trivial effect size of 0.10 and the mean effect size in missing studies of zero) values were also calculated. The Classic fail-safe N value means that 2960 additional papers with non-significant findings would be necessary to nullify the effect of chemical bonding-based intervention studies on academic performance. Similarly, Orwin's fail-safe N value (see Table 4) indicates that 280 additional papers with an effect-size of zero would be necessary to make the mean effect of this meta-analysis as trivial (Üstün and Eryılmaz, 2014). Further, the failsafe formula [N/(5k + 10)] (k means the total number of the studies in the meta-analysis) appeared to have a high ratio (10.956) for the current meta-analysis, which is higher than the cut-off point (1.00) offered by Mullen et al. (2001). This ratio also complies with the other results of the publication bias analysis. The foregoing values affirmed that the current meta-analysis has the power to sufficiently tolerate future null results. Therefore, its data selection process was robust and not influenced by publication bias.
Z-Value for observed studies | 30.773 |
P-Value for observed studies | 0.000 |
Alpha | 0.050 |
Tails | 2.000 |
Z for alpha | 1.960 |
Number of observed studies | 53 |
Number of missing studies that would bring p value to >alpha | 3013 |
Hedges’ g in observed studies | 0.727 |
Criterion for a ‘trivial’ Hedges’ g | 0.100 |
Mean Hedges’ g in missing studies | 0.000 |
Number missing studies needed to bring Hedges’ g under 0.1 | 333 |
Frequency | Stem | Leaf |
---|---|---|
1 | −0 | 0 |
12 | 0 | 000112233334 |
19 | 0 | 5556666677777899999 |
9 | 1 | 000012223 |
6 | 1 | 567779 |
1 | 2 | 2 |
4 | 2 | 6678 |
1 | 4 | 1 |
Studies | Hedges' g | Standard error | P-Value | |
---|---|---|---|---|
Acar and Tarhan (2008) | 2.698 | 0.364 | 0.000 | |
Adane (2020) | 1.324 | 0.223 | 0.000 | |
Adherr et al. (2019) | 1.051 | 0.310 | 0.001 | |
Anekwe and Opara (2021) | 2.663 | 0.193 | 0.000 | |
Atasoy et al. (2003) | 0.049 | 0.221 | 0.826 | |
Bayrak (2005) | 0.606 | 0.261 | 0.020 | |
Chan (2016) | 0.021 | 0.323 | 0.948 | |
da Silva et al. (2020) | 0.031 | 0.216 | 0.885 | |
Doymus (2008) | 0.910 | 0.180 | 0.000 | |
Ekinci (2010) | 1.180 | 0.190 | 0.000 | |
Eymur and Geban (2017) | 2.817 | 0.331 | 0.000 | |
Frailich et al. (2009) | 0.758 | 0.139 | 0.000 | |
Genel (2008) | 1.686 | 0.147 | 0.000 | |
Ginting and Juniar (2022) | 0.544 | 0.260 | 0.036 | |
Gongden et al. (2020) | 2.787 | 0.326 | 0.000 | |
Ikenna (2014) | 0.658 | 0.227 | 0.004 | |
İnal (2013) | 0.626 | 0.190 | 0.001 | |
Iryani et al. (2021) | 0.693 | 0.244 | 0.004 | |
Karacop and Doymus (2013) | Subgroup 1 | 0.762 | 0.231 | 0.001 |
Subgroup 2 | 0.779 | 0.236 | 0.001 | |
Kılıç (2007) | 1.266 | 0.312 | 0.000 | |
Kırılmazkaya et al. (2014) | 0.740 | 0.268 | 0.006 | |
Korkman (2018) | Subgroup 1 | −0.077 | 0.247 | 0.756 |
Subgroup 2 | 0.600 | 0.180 | 0.001 | |
Kuit and Osman (2021) | 0.515 | 0.233 | 0.027 | |
Mercy et al. (2019) | 0.296 | 0.182 | 0.105 | |
Mondal (2012) | 0.817 | 0.231 | 0.000 | |
Munawarah et al. (2020) | 4.189 | 0.456 | 0.000 | |
Okorie (2015) | 0.277 | 0.114 | 0.015 | |
Özmen (2008) | 2.225 | 0.356 | 0.000 | |
Özmen et al. (2009) | 0.578 | 0.265 | 0.029 | |
Pabuçcu and Geban (2006) | 0.990 | 0.325 | 0.002 | |
Pabuçcu and Geban (2012) | 0.162 | 0.307 | 0.597 | |
Pamuk (2018) | 1.261 | 0.280 | 0.000 | |
Sarı (2013) | 1.774 | 0.271 | 0.000 | |
Sentongo et al. (2013) | 1.751 | 0.218 | 0.000 | |
Sevim (2007) | 1.700 | 0.160 | 0.000 | |
Sharma (2015) | 0.379 | 0.147 | 0.010 | |
Side et al. (2020) | 0.400 | 0.249 | 0.108 | |
Singh and Moono (2015) | 1.520 | 0.310 | 0.000 | |
Sunyono and Meristin (2018) | 0.350 | 0.160 | 0.029 | |
Suyanti and Sormin (2016) | 0.905 | 0.268 | 0.001 | |
Şeker (2012) | 1.947 | 0.340 | 0.000 | |
Tarhan et al. (2008) | 1.000 | 0.238 | 0.000 | |
Teichert and Stacy (2002) | 0.103 | 0.295 | 0.726 | |
Tsaparlis et al. (2018) | 0.346 | 0.045 | 0.000 | |
Ulusoy (2011) | Subgroup 1 | 0.929 | 0.208 | 0.000 |
Subgroup 2 | 0.960 | 0.208 | 0.000 | |
Uzuntiryaki (2003) | 1.061 | 0.321 | 0.001 | |
Ültay (2015) | 0.374 | 0.296 | 0.207 | |
Widarti et al. (2019) | 1.220 | 0.288 | 0.000 | |
Yıldırım et al. (2018) | 0.779 | 0.298 | 0.009 | |
Zorluoğlu and Sözbilir (2016) | 1.020 | 0.252 | 0.000 | |
Random | 1.007 | 0.094 | 0.000 |
Educational level | k | Point estimate | Standard error | Confidence interval (95%) | Z-Value | p-Value | Q-Value | df (Q) | p-Value | |
---|---|---|---|---|---|---|---|---|---|---|
Lower limit | Upper limit | |||||||||
K-8 | 9 | 0.783 | 0.175 | 0.440 | 1.126 | 4.475 | 0.000 | |||
High school | 34 | 1.084 | 0.125 | 0.839 | 1.329 | 8.677 | 0.000 | |||
University | 10 | 0.945 | 0.195 | 0.562 | 1.328 | 4.833 | 0.000 | |||
Total between | 1.996 | 2 | 0.369 |
Type of intervention | k | Point estimate | Standard error | Confidence interval (95%) | Z-Value | p-Value | Q-Value | df (Q) | p-Value | |
---|---|---|---|---|---|---|---|---|---|---|
Lower limit | Upper limit | |||||||||
Computer-assisted instruction | 14 | 1.165 | 0.222 | 0.730 | 1.600 | 5.249 | 0.000 | |||
Concept map | 2 | 1.391 | 0.181 | 1.036 | 1.746 | 7.689 | 0.000 | |||
Conceptual change | 7 | 1.175 | 0.188 | 0.808 | 1.543 | 6.265 | 0.000 | |||
Constructivist learning environment | 3 | 0.415 | 0.251 | −0.077 | 0.908 | 1.652 | 0.099 | |||
Cooperative learning | 4 | 1.766 | 0.510 | 0.766 | 2.767 | 3.461 | 0.001 | |||
Enriched learning environment with different methods | 6 | 1.472 | 0.417 | 0.653 | 2.290 | 3.525 | 0.000 | |||
Inquiry-based learning | 5 | 0.541 | 0.164 | 0.219 | 0.863 | 3.292 | 0.001 | |||
Multiple representation | 2 | 0.752 | 0.434 | −0.098 | 1.603 | 1.734 | 0.083 | |||
Total between | 23.141 | 7 | 0.002 |
It is important to point out that some of the intervention studies outperformed others. For example, Munawarah et al. (2020), who implemented the enriched learning environment with different methods (e.g., problem-based learning and worksheets) for teaching chemical bonding, produced the highest effect-size (perfectly huge). Their study was crucial for illustrating how the scope, content, and nature of the teaching intervention are important in addressing the purpose of one's research (Pacaci et al., 2023). Moreover, cultural contexts can affect how the interventions fit with the learning goals of the national science/chemistry curriculum or educational priorities. To emphasize this finding, the studies conducted in the following countries, which have high-stakes testing or performance exams in science/chemistry education, had effect-sizes that were within the perfectly huge range: Türkiye (Sevim, 2007; Acar and Tarhan, 2008; Genel, 2008; Özmen, 2008; Şeker, 2012; Sarı, 2013; Eymur and Geban, 2017), Nigeria (Anekwe and Opara, 2021; Gongden et al., 2020), Indonesia (Munawarah et al., 2020), Uganda (Sentongo et al., 2013), and Zambia (Singh and Moono, 2015).
Nonetheless, it is important to note that high school (k = 34) had the highest mean effect-size while K-8 (k = 9) was the lowest. These values align with the effect-sizes of conceptual change strategies reported by Pacaci et al. (2023). Also, the mean effect-size of the studies with high school is very close to the value (Cohen d = 1.06) found by Ahmad et al. (2023). This may result from the nature of the course involving chemical bonding topic. For instance, high school and university include separate chemistry courses, whereas K-8 contains an integrated science course (e.g., biology, chemistry, physics, earth science, and astronomy). Another consideration is that as chemistry concepts become more advanced, more time is needed to study them. Consequently, the duration of the time spent studying chemistry is longer for students in higher grades. It follows that the intervention studies conducted within K-8 took less time than those with older students. For example, the implementation duration of the studies with K-8 (k = 9) generally took 2–3 weeks (e.g., Bayrak, 2005; Ültay, 2015; Pamuk, 2018; Yıldırım et al., 2018), while that for high school (k = 34) and university (k = 10) mostly lasted 4–8 weeks (e.g., Pabuçcu and Geban, 2006; Sevim, 2007; Karacop and Doymus, 2013; Okorie, 2015; Chan, 2016; Adane, 2020).
Although the findings indicate that differences in mean effect-size among educational levels were non-significant, it is worthwhile explaining the n-shaped developmental curve regarding academic performance that was found in the meta-analysis. This curve means that there were notable effects of the interventions at the educational levels, but the effects were more muted in K-8 (k = 9). One reason for this pattern may be that chemistry, as part of an integrated science course, is new for younger learners, so its logical changes would be dramatic. For older students, the pattern may be explained by the fact that their chemistry studies and academic performance are more advanced as a result of epistemological and ontological growth. In other words, the student performance outcomes of the studies’ findings may come from the type of chemical bonding-based assessment implemented at each level (Langbeheim et al., 2023; Pacaci et al., 2023). For example, although multiple choice questions were very common at all educational levels (e.g., Bayrak, 2005; Karacop and Doymus, 2013; Korkman, 2018; Adherr et al., 2019; Ginting and Juniar, 2022), the studies conducted at the university level had the students respond to open-ended questions or two-tier diagnostic questions where they expressed their responses in writing and diagrammed chemical formulas (e.g., Teichert and Stacy, 2002; Karacop and Doymus, 2013; Sari, 2013). Therefore, the academic performance of university students (k = 10) may have been affected by their ability to state, draw, and describe their conceptual understanding of chemical bonding.
The mean effect-size for cooperative learning (k = 4) had the highest value (Hedges’ g = 1.766) amongst the intervention types. This may result from matching the nature of the chemical bonding with the common features of cooperative learning, which requires students to work in groups on a common task. Therefore, aspects of cooperative learning (e.g., positive interdependence, accountability, promotive interactions, teaching interpersonal skills, and group processing) not only encourage students to have intellectual conversations about problem-solving skills and creative thinking, but also facilitate their own learning (e.g., achievement and conceptual understanding) through group interactions (Johnson and Johnson, 1987; Johnson et al., 1998; Rahman and Lewis, 2020; Akkuş and Doymuş, 2022). Furthermore, a socially situated learning environment may have contributed to participants’ meaningful learning of the ‘chemical bonding’ topic (McManus and Gettinger, 1996). Phrased differently, a well-designed cooperative learning, seems to have improved participants’ academic performance (Acar and Tarhan, 2008; Eymur and Geban, 2017) and resulted in a perfectly huge effect.
Another intervention that had a strong effect-size (classified under the perfectly huge effect), was an enriched learning environment with different methods (k = 6). This outcome may come from the power of different learning methods or strategies (e.g., computer-assisted presentation, three-dimensional materials, concept maps, concept networks, evidence-based learning, problem-based learning, worksheets, conceptual change texts, animations, and analogies) that meet various learning styles and pose students’ capacities for learning to increase the efficiency of any intervention (Genel, 2008; Munawarah et al., 2020). Moreover, the studies that merged methods within an enriched learning environment (k = 6) resulted in better learning outcomes than when methods were used in isolation (e.g., Çalik et al., 2023). Therefore, using a variety of teaching methods provides students with a broader sense of learning chemical bonding and increases the probability of them achieving the course objectives (e.g., Zhang, 2014; Çalik et al., 2015).
The mean effect-sizes of intervention studies that used computer-assisted instruction (k = 14), concept map (k = 2), and conceptual change (k = 7) had a very large effect (see Table 8). This effect may result from the nature of these intervention types. For example, computer-assisted instruction models scientific phenomena at the sub-microscopic level and includes enriched technological tools/components (e.g., audio, sound, visualization, interaction, feedback, and hypertext) (Bayraktar, 2001). This modelling may have enhanced students’ capacities for learning and made unfamiliar and abstract concepts (such as chemical bonding) more concrete and familiar (Özmen, 2008; Sentongo et al., 2013; Pamuk, 2018; Gongden et al., 2020; Anekwe and Opara, 2021). Further, computer-assisted instruction may have afforded students the opportunity to bridge their own macroscopic observations to the sub-microscopic level (Barak and Dori, 2005; Frailich et al., 2009).
Concept mapping (k = 2) may provide students with a different way to envision key chemical bonding concepts, using words or short phrases to link and organize the concepts (Novak, 1990, 1998; Singh and Moono, 2015; Adane, 2020). The pedagogical/meta-cognitive engagement germane to concept mapping seems to solidify and facilitate students’ conceptual understanding of the ‘chemical bonding’ topic (Chevron, 2014; Adane, 2020).
Teaching strategies that promote conceptual change (k = 7) may challenge students to consider alternative conceptions and encourage them to replace insufficient concepts with more fruitful and scientifically accepted ones (Hewson and Hewson, 1983; Çalik et al., 2009; Hafizhah Putri et al., 2022). Thus, the use of cognitive and knowledge-based interpretations/processes of conceptual change seems to have supported students’ meaningful learning and sound understanding of chemical bonding concepts (Guzzetti et al., 1993; Çalik et al., 2023). Also, this may result from matching the nature of students’ alternative conceptions of chemical bonding with possible ways and/or strategies of conceptual change (e.g., cognitive conflict, cognitive bridging, and ontological category shift) (Pacaci et al., 2023). Moreover, the mean effect-size (Hedges’ g = 1.175) of conceptual change studies on chemical bonding advocates the findings of Pacaci et al. (2023), who meta-analytically evaluated 218 conceptual change studies in science education (Hedges’ g = 1.10).
The intervention studies that used multiple representations (k = 2) had a mean effect-size value of large (Hedges’ g = 0.752). This means that this type of intervention has some success at assisting students to learn abstract or complicated concepts/knowledge of chemical bonding (Özmen, 2004; Ünal et al., 2006; Nahum et al., 2010; Hunter et al., 2022; van Dulmen et al., 2023). Multiple representation gives an opportunity for students to construct, re-consider and synthesise the same knowledge from multiple perspectives via the use of models, pictures, and a combination of them, or various visual tools (Tsui and Treagust, 2004; Adadan et al., 2010; Namdar and Shen, 2018). It was surprising that this intervention was somewhat effective given that it entails students using abstract thinking to understand abstract concepts of chemical bonding. This means that a theoretical match between multiple representation and chemical bonding seems to have somewhat reflected in practicum. Indeed, there were a couple of intervention studies using multiple representation that had different effect-sizes (Sunyono and Meristin, 2018 [Hedges’ g = 0.350]; and Widarti et al., 2019 [Hedges’ g = 1.220]). It is possible that the overall large effect found in the current meta-analysis resulted from the limited number of intervention studies included in the sample.
Finally, the types of intervention with a medium effect on improving students’ academic performance were constructivist learning environments (k = 3) and inquiry-based learning (k = 5) (see Table 8). These interventions are typically known for being effective because they are student-centred, similar to cooperative learning. The findings of the current study indicate that the topic of chemical bonding may not be suitable for these types of interventions; perhaps the concept is too abstract for constructivist learning and inquiry. Another consideration is the extent to which these types of interventions are incorporated into different cultural contexts. For example, the Turkish science curriculum released in 2000 initially included some elements of constructivist learning theory (Çalık and Ayas, 2008). This means that this type of intervention may be very new for students and authors (Atasoy et al., 2003; Uzuntiryaki, 2003; Mercy et al., 2019). They may have had difficulties involving essential features of constructivism (e.g., eliciting prior knowledge, creating cognitive dissonance, applying new knowledge with feedback, and reflecting on learning) in the learning process or developing instructional materials or lesson plans (Baviskar et al., 2009). Overall, there were inconsistent effect-sizes for constructivist learning environments that may have affected the outcome of the analysis (Atasoy et al., 2003 [Hedges’ g = 0.049]; Uzuntiryaki, 2003 [Hedges’ g = 1.061]; Mercy et al., 2019 [Hedges’ g = 0.296]).
Cultural contexts may have affected the outcomes of intervention studies using inquiry-based learning as well. Inquiry-based learning incorporates a process of enquiry that requests students to answer a question or solve a problem by conducting experiments, collecting and analysing data, and drawing conclusions (Orosz et al., 2022; Ma, 2023). Another important component of inquiry involves asking questions and criticising answers. These actions may differ from earlier learning habits and cultural behaviours or characteristics (Halim et al., 2023). For example, in some cultures students may prefer not to question the knowledge of their teachers and parents. Cultural norms endorse being respectful and polite to people in positions of authority; therefore, it is important to avoid making challenges or posing arguments (e.g., Nusirjan and Fensham, 1987; Halim et al., 2023; Wiyarsi et al., 2023). Therefore, these habits and cultural norms may have caused some adaptation and orientation issues for intervention studies that used inquiry-based learning in different context(s) or countries (Indonesia, Nigeria, and Türkiye) since inquiry-based learning was initially released for Western countries.
Given the significant role of scope, content, and nature of the teaching intervention in improving academic performance, future research should carefully handle these issues within their chemical bonding-based interventions. Since some extrinsic factors (e.g., cultural context) can influence the effectiveness of interventions, future research should consider them while selecting the type of intervention and carefully portray how they fit with the learning goals of the national science/chemistry curriculum. Also, because educational level indicates the n-shaped developmental curve in terms of chemical bonding-based academic performance, further research should explore the reasons and factors affecting this issue.
A key finding of this study is that the effect sizes of some interventions varied greatly (e.g., Acar and Tarhan, 2008; Özmen, 2008; Eymur and Geban, 2017; Gongden et al., 2020; Munawarah et al., 2020; Anekwe and Opara, 2021). Therefore, future researchers should explore the effectiveness of these intervention types on academic performance in different contexts, cultures, and grades. Furthermore, there were some interventions that were used by limited studies and therefore it was impossible to analyse overall effect size across multiple studies. When more researchers consider interventions such as context-based learning, modelling, multiple intelligences, problem-based learning, and enriched texts, more information about the effectiveness of these interventions can be investigated. Finally, this study, like all meta-analyses, points out consistent and inconsistent findings among published research. Therefore, further meta-analysis studies should be undertaken to resolve any contradictory findings.
Type of intervention | Descriptions |
---|---|
Computer-assisted instruction | Refers to the use of computers and computer-based applications (e.g., animation, simulation, educational games, web-based learning environment) in the learning-teaching process to facilitate student learning (Bayraktar, 2001; Tang and Abraham, 2016). Thus, it serves the goals of science education by modelling any scientific phenomenon at sub-microscopic level and enhancing students’ capacities of learning via enriched technological tools/components (e.g., audio, sound, visualization, interaction, feedback and hypertext) (Özmen, 2008; Kırılmazkaya et al., 2014; Adherr et al., 2019; da Silva et al., 2020; Anekwe and Opara, 2021). |
Concept map | Provides a visual road map showing pathways and relationships between concepts, propositions and phrases (Novak and Gowin, 1984; Novak, 1990). Thus, it not only helps students meaningfully learn concepts, relationships and hierarchies but also clarify or refine their cognitive structures (Singh and Moono, 2015; Adane, 2020; Huynh and Yang, 2023). Therefore, it promotes students’ conceptual understanding and facilitates their abilities of problem-solving and synthesizing concepts (Singh and Moono, 2015). |
Conceptual change | Intends to replace alternative conceptions or misconceptions with scientifically accepted ones and integrate new conceptions with pre-existing ones (Hewson and Hewson, 1983; Çalık et al., 2009). Thus, it supports meaningful learning and a sound understanding of science concepts that play a generative or organizing role in thought (Guzzetti et al., 1993; Pabuçcu and Geban, 2006; Ültay, 2015). |
Constructivist learning environment | Claims that learning is an interaction between pre-existing knowledge and new knowledge. Further, students, who actively participate in learning process and take their own responsibility for learning, construct new knowledge in light of their prior conceptions (Guzzetti et al., 1993). To involve essential features of constructivism (e.g., eliciting prior knowledge, creating cognitive dissonance, application of new knowledge with feedback, and reflection on learning) in learning process, such instructional models as learning cycle, generative learning model, 5E learning model are offered (Baviskar et al., 2009). |
Cooperative learning | Requires students to work in small cooperative groups on a common task, which is in harmony with the goals of science curriculum. Thus, an effective cooperative learning includes such essential features as positive interdependence, accountability, promotive interactions, teaching interpersonal skills and group processing (Johnson et al., 1998; Rahman and Lewis, 2019). Phrased differently, it not only fosters students to have intellectual conversations about problem-solving skills and creative thinking, but also increases group interaction to enhance their own learning (e.g., achievement and conceptual understanding) (Johnson and Johnson, 1990). |
Enriched learning environment with different methods | Refers to the use of some combinations of different methods (e.g., cooperative learning, animation, worksheet, problem-based learning, animation, analogies and conceptual change text) to increase the effectiveness of any intervention (Özmen et al., 2009; Pabuçcu and Geban, 2012; Karacop and Doymus, 2013; Munawarah et al., 2020). Hence, it merges the advantages of different methods into an enriched teaching intervention to meet various learning styles and pose students’ capacities of learning. For this reason, it purposes to result in better learning outcomes, e.g., achievement and conceptual understanding (Bağ and Çalık, 2022; Er Nas et al., 2022; Türkoguz and Ercan, 2022). |
Inquiry-based learning | Refers to a process of enquiry that asks students to answer a question or solve a problem by conducting experiments, collecting and analysing data, and drawing conclusions (Orosz et al., 2022; Ma, 2023). Thus, they are able to acquire new knowledge, inquiry skills, and develop attitudes towards science and understand the nature of science as the outcomes of a student-centred method (Ikenna, 2014; Suyanti and Sormin, 2016; Iryani et al., 2021; Ginting and Juniar, 2022). |
Multiple representation | Involves the use of models, pictures, and a combination of them or various visual tools that enable people to communicate ideas or concepts (Tsui and Treagust, 2004; Adadan et al., 2010). By capturing students’ attention to the concepts to be taught, it improves students’ conceptual understanding (Ainsworth, 1999; Adadan et al., 2010). Since it provides diverse opportunities to students in constructing the same knowledge from multiple perspectives, any weaknesses dealt with one particular representation might be replaced by another one (Adadan et al., 2010). Thus, it facilitates the interpretation of abstract or complicated concepts/knowledge and helps students achieve meaningful learning (Adadan et al., 2010; Sunyono and Meristin, 2018; Widarti et al., 2019; Adadan and Ataman, 2021). |
Other | Covers different intervention types (e.g., context-based learning, discovery learning model equipped re-lyric songs, drama-assisted instruction, enriched text, explain and integration ideas, learning styles, modelling, multiple intelligence, problem-based learning and remedial learning system) reported by only one paper. |
Footnote |
† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d3rp00258f |
This journal is © The Royal Society of Chemistry 2024 |