1887

Using a Concept Inventory to Assess the Reasoning Component of Citizen-Level Science Literacy: Results from a 17,000-Student Study

    Authors: Edward B. Nuhfer1,*, Christopher B. Cogan2, Carl Kloock3, Gregory G. Wood4, Anya Goodman5, Natalie Zayas Delgado6, Christopher W. Wheeler7
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Retired Professor of Geology and Director of Faculty Development & Assessment, Niwot, CO 80503; 2: Independent Consultant, Environmental Science, Geography, and Geographic Information Systems, Camarillo, CA 93012; 3: Department of Biology, California State University Bakersfield, Bakersfield, CA 93311; 4: Department of Physics, California State University Channel Islands, Camarillo CA 93012; 5: Department of Chemistry and Biochemistry, California Polytechnic State University, San Luis Obispo, CA 93407; 6: Department of Science & Environmental Policy, California State University Monterey Bay, Seaside, CA 93955; 7: Geology Program, California State University Channel Islands, Camarillo, CA 93012
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 01 March 2016
    • ©2016 Author(s). Published by the American Society for Microbiology.
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://jmbe.asm.org
    • *Corresponding author. Mailing address: Retired Professor of Geology and Director of Faculty Development & Assessment, 7147 Overbrook Drive, Niwot, CO 80503. Phone: 208-241-5029. E-mail: enuhfer@earthlink.net.
    Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    94.68 Kb
  • PDF
    816.83 Kb
  • XML

    Abstract:

    After articulating 12 concepts for the reasoning component of citizen-level science literacy and restating these as assessable student learning outcomes (SLOs), we developed a valid and reliable assessment instrument for addressing the outcomes with a brief 25-item science literacy concept inventory (SLCI). In this paper, we report the results that we obtained from assessing the citizen-level science literacy of 17,382 undergraduate students, 149 graduate students, and 181 professors. We address only findings at or above the 99.9% confidence level. We found that general education (GE) science courses do not significantly advance understanding of science as a way of knowing. However, the understanding of science’s way of knowing does increase through academic ranks, indicating that the extended overall academic experience better accounts for increasing such thinking capacity than do science courses alone. Higher mean institutional SLCI scores correlate closely with increased institutional selectivity, as measured by the institutions’ higher mean SAT and ACT scores. Socioeconomic factors of a) first-generation student, b) English as a native language, and c) interest in commitment to a science major are unequally distributed across ethnic groups. These factors proved powerful in accounting for the variations in SLCI scores across ethnicities and genders.

Key Concept Ranking

Lead
0.925
Coffee
0.5848656
Infection
0.48125
Stems
0.4577538
0.925

References & Citations

1. American Association for the Advancement of Science1989Science for all AmericansOxford University PressNew York, NY
2. American Association for the Advancement of Science1993Benchmarks for science literacyOxford University PressNew York, NY
3. Antonellis J, Buxner S, Impey C, Sugarman H2012Surveying science literacy among undergraduates: insights from open-ended responsesJ Coll Sci Teach4138290
4. Brookfield S2012Teaching for critical thinkingJohn Wiley and SonsSan Francisco, CA
5. Chamberlin TC1897The method of multiple working hypothesesJ Geol5837848Reprinted: 1965. Science 148:754–75910.1086/607980 http://dx.doi.org/10.1086/607980
6. Coil D, Wenderoth MP, Cunningham M, Dirks C2010Teaching the process of science: faculty perceptions and an effective methodologyCBE Life Sci Educ952453510.1187/cbe.10-01-0005211236992995770 http://dx.doi.org/10.1187/cbe.10-01-0005
7. Duit R2009Students’ and teachers’ conceptions and science education: full version bibliography[Online.] http://archiv.ipn.uni-kiel.de/stcse/download_stcse.htmlAccessed 22 August 2015
8. Flavell JH1976Metacognitive aspects of problem solving231236 Resnick LBThe nature of intelligenceErlbaumHillsdale, NJ
9. Flavell JH1979Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiryAmer Psychol341090691110.1037/0003-066X.34.10.906 http://dx.doi.org/10.1037/0003-066X.34.10.906
10. Gaze E2014Teaching quantitative reasoning: a better context for algebraNumeracy71Article 1. [Online.] http://scholarcommons.usf.edu/numeracy/vol7/iss1/art1/Accessed 22 August 201510.5038/1936-4660.7.1.1 http://dx.doi.org/10.5038/1936-4660.7.1.1
11. Gaze EC, Montgomery A, Kilic-Bahi S, Leoni D, Misener L, Taylor C2014Towards developing a quantitative literacy/reasoning assessment instrumentNumeracy72Article 4. [Online.] http://scholarcommons.usf.edu/numeracy/vol7/iss2/art4/Accessed 22 August 201510.5038/1936-4660.7.2.4 http://dx.doi.org/10.5038/1936-4660.7.2.4
12. Gill RA, Burke IC1999Using an environmental science course to promote scientific literacyJ Coll Sci Teach25105110
13. Glazer FS2000Journal clubs—a successful vehicle to science literacyJ Coll Sci Teach295320324
14. Gormally C, Brickman P, Lutz M2012Developing a test of scientific literacy skills TOSLS: measuring undergraduates’ evaluation of scientific information and argumentsCBE Life Sci Educ1136437710.1187/cbe.12-03-0026232228323516792 http://dx.doi.org/10.1187/cbe.12-03-0026
15. Hallowell C, Holland MJ1998Journalism as a path to scientific literacyJ Coll Sci Teach2812932
16. Hattie J2009Visible Learning: A synthesis of over 800 meta-analyses relating to achievementRoutledgeNew York, NY61636871
17. Henderson C2002Common concerns about the Force Concept InventoryPhys Teach4054254710.1119/1.1534822 http://dx.doi.org/10.1119/1.1534822
18. Hestenes D, Wells M, Swackhamer G1992Force Concept InventoryPhys Teach3014115810.1119/1.2343497 http://dx.doi.org/10.1119/1.2343497
19. Holton G19991948The new imperative for science literacyJ Coll Sci Teach283181185
20. Impey C, Buxner S, Antonellis J, Johnson E, King C2011A twenty-year survey of science literacy among college undergraduatesJ Coll Sci Teach4043137
21. Johnson V2013Revised standards for statistical evidenceProc Natl Acad Sci USA110193131931710.1073/pnas.1313476110242185813845140 http://dx.doi.org/10.1073/pnas.1313476110
22. Journal of Adult Development2004Special volume of nine papers on the Perry legacy of cognitive developmentJ Adult Dev11259161Periodicals Service CoGermantown, NY
23. King PM, Kitchener KS1994Developing reflective judgment: understanding and promoting intellectual growth and critical thinking in adolescents and adultsJossey BassSan Francisco, CA
24. Laugksch RC, Spargo PE1996Construction of a paper-and-pencil test of basic scientific literacy based on selected literacy goals recommended by the American Association for the Advancement of SciencePublic Underst Sci533135910.1088/0963-6625/5/4/003 http://dx.doi.org/10.1088/0963-6625/5/4/003
25. Lawson AE1978Development and validation of the classroom test of formal reasoningJ Res Sci Teach151112410.1002/tea.3660150103 http://dx.doi.org/10.1002/tea.3660150103
26. Libarkin J2008Geoscience concept inventory: suggestions for question review and development: Notes Provided from Geological Society of America Annual Meeting Short Course # 523Writing and Evaluating Geoscience Concept Inventory Questions
27. Maskey CL2011An evaluation of the relationship between reflective judgment and critical thinking in senior associate-degree nursing studentsDoctoral dissertationIndiana UniversityIndianapolis, IN
28. McComas WF1998The principal elements of the nature of science: dispelling the myths5370 McComas WFThe Nature of Science in Science EducationKluwer Academic PublishersNetherlands
29. Miller JD1983Scientific literacy: a conceptual and empirical reviewDaedalus11222948
30. Miller JD1998The measurement of civic scientific literacyPublic Underst Sci7320322310.1088/0963-6625/7/3/001 http://dx.doi.org/10.1088/0963-6625/7/3/001
31. National Academy of Sciences and Institute of Medicine2008Science, evolution, and creationismThe National Academies PressWashington, DC
32. National Center for Education Statistics2012Institute of Education Sciences. Digest of Education Statistics 2013, Table 318.20Bachelor’s, master’s, and doctor’s degrees conferred by postsecondary institutions, by field of study: Selected years, 1970–71 through 2011–12[Online.] http://nces.ed.gov/programs/digest/d13/tables/dt13_318.20.aspAccessed 23 August 2015
33. NGSS Lead States2013Next generation science standards: for states, by statesThe National Academies PressWashington, DC
34. Nuhfer EB2013Understanding Cinderella—expanding an essential conversation on the metadiscipline of technology: educating in fractal patterns XXXVIINatl Teach Learn Forum222710
35. Nuhfer EB, Gaze E2014Benefits of assessing citizen literacy in science and quantitative reasoning: educating in fractal patterns XLIINatl Teach Learn Forum24181110.1002/ntlf.30012 http://dx.doi.org/10.1002/ntlf.30012
36. Nuhfer EB, et al2010Multi-campus project: promoting and assessing science literacy in general education science coursesCalifornia State University, Institute for Teaching and Learning Connections34[Online.] https://www.calstate.edu/itl/newsletter/10-summer.shtmlAccessed 18 August 2015
37. Pavelich MJ, Moore WS1996Measuring the effect of experiential education using the Perry modelJ Engineer Educ85428729210.1002/j.2168-9830.1996.tb00247.x http://dx.doi.org/10.1002/j.2168-9830.1996.tb00247.x
38. Perry WGJr1999Forms of intellectual and ethical development in the college yearsReprint of the original 1968 1st edition with introduction by L. KnefelkampJossey-BassSan Francisco, CA
39. Seckam A, Cooper R2013Understanding how honey impacts on wounds: an update on recent research findingsWounds Internatl412024
40. Selby C2006What makes it science?: A modern look at scientific inquiryJ Coll Sci Teach357811
41. Specter M2010Denialism: how irrational thinking harms the planet and threatens our livesDuckworth OverlookLondon, UK
42. Steele CM, Aronson J1995Stereotype threat and the intellectual test performance of African AmericansJ Personality Social Psych69579781110.1037/0022-3514.69.5.797 http://dx.doi.org/10.1037/0022-3514.69.5.797
43. Strahler A1992Understanding science: an introduction to concepts and issuesPrometheus BooksBuffalo, NY
44. Strimaitis AM, Schellinger J, Jones A, Grooms J, Sampson V2014Development of an instrument to assess student knowledge necessary to critically evaluate scientific claims in the popular mediaJ Coll Sci Teach43556810.2505/4/jcst14_043_05_55 http://dx.doi.org/10.2505/4/jcst14_043_05_55
45. Terry DR2012Assessing critical thinking skills using articles from the popular pressJ Coll Sci Teach426670
46. Trefil J1992Science matters: achieving scientific literacyAnchor BooksNew York, NY
47. Trefil J2008Why science?Teachers College Press, National Science Teachers AssociationArlington, VA
48. Wenning CJ2006Assessing nature-of-science literacy as one component of scientific literacyJ Phys Teach Educ Online34314
49. White B, et al2011A novel instrument for assessing students’ critical thinking abilitiesJ Coll Sci Teach405102107
50. Wise SL, DeMars CE2005Examinee motivation in low-stakes assessment: problems and potential solutionsEduc Assess10111710.1207/s15326977ea1001_1 http://dx.doi.org/10.1207/s15326977ea1001_1
51. Wolpert L1992The unnatural nature of scienceHarvard University PressCambridge, MA
jmbe.v17i1.1036.citations
jmbe/17/1
content/journal/jmbe/10.1128/jmbe.v17i1.1036
Loading

Citations loading...

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v17i1.1036
2016-03-01
2017-09-24

Abstract:

After articulating 12 concepts for the reasoning component of citizen-level science literacy and restating these as assessable student learning outcomes (SLOs), we developed a valid and reliable assessment instrument for addressing the outcomes with a brief 25-item science literacy concept inventory (SLCI). In this paper, we report the results that we obtained from assessing the citizen-level science literacy of 17,382 undergraduate students, 149 graduate students, and 181 professors. We address only findings at or above the 99.9% confidence level. We found that general education (GE) science courses do not significantly advance understanding of science as a way of knowing. However, the understanding of science’s way of knowing does increase through academic ranks, indicating that the extended overall academic experience better accounts for increasing such thinking capacity than do science courses alone. Higher mean institutional SLCI scores correlate closely with increased institutional selectivity, as measured by the institutions’ higher mean SAT and ACT scores. Socioeconomic factors of a) first-generation student, b) English as a native language, and c) interest in commitment to a science major are unequally distributed across ethnic groups. These factors proved powerful in accounting for the variations in SLCI scores across ethnicities and genders.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/17/1/jmbe-17-143.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v17i1.1036&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Actual undergraduate student performance ( = 17,362) compared with performance through random guessing. Mean overall score = 68.33%. Distribution reveals little total ignorance of science’s way of knowing among respondents, but room exists for significant improvement. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

SLCI mean scores for 17,072 undergraduates based on number of college science courses completed. Little mean difference exists between none, one, and two courses (red font). The height of the diamonds denotes the 99.9% confidence interval, which is the same as the diameters of the circles in the Student’s -test box to the right. Space separating the circles denotes significant differences between course categories. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

SLCI mean scores arranged in increasing order of self-reported ranks for undergraduates, graduate students, and professors. Highly significant differences exist between successively higher academic ranks except for an absence of significant difference between sophomores and juniors (red font). SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 4

Click to view

FIGURE 4

Gains in SLCI mean scores of freshmen and senior ranks for 12 institutions of varied selectivity. One school with a mean ACT of about 22 showed a decrease, but all seniors sampled at that school came from introductory courses and were not likely representative of the institution’s actual seniors. To do meaningful plots requires a school to obtain a good representative sampling of upper- and lower-division courses. As shown by the slopes of the line fits and their convergence, a ceiling effect makes it more difficult for selective schools to achieve large gains. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error