1887

Using a Concept Inventory to Assess the Reasoning Component of Citizen-Level Science Literacy: Results from a 17,000-Student Study

    Authors: Edward B. Nuhfer1,*, Christopher B. Cogan2, Carl Kloock3, Gregory G. Wood4, Anya Goodman5, Natalie Zayas Delgado6, Christopher W. Wheeler7
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Retired Professor of Geology and Director of Faculty Development & Assessment, Niwot, CO 80503; 2: Independent Consultant, Environmental Science, Geography, and Geographic Information Systems, Camarillo, CA 93012; 3: Department of Biology, California State University Bakersfield, Bakersfield, CA 93311; 4: Department of Physics, California State University Channel Islands, Camarillo CA 93012; 5: Department of Chemistry and Biochemistry, California Polytechnic State University, San Luis Obispo, CA 93407; 6: Department of Science & Environmental Policy, California State University Monterey Bay, Seaside, CA 93955; 7: Geology Program, California State University Channel Islands, Camarillo, CA 93012
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 01 March 2016
    • ©2016 Author(s). Published by the American Society for Microbiology.
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://jmbe.asm.org
    • *Corresponding author. Mailing address: Retired Professor of Geology and Director of Faculty Development & Assessment, 7147 Overbrook Drive, Niwot, CO 80503. Phone: 208-241-5029. E-mail: [email protected].
    Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    94.68 Kb
  • PDF
    816.83 Kb
  • XML

    Abstract:

    After articulating 12 concepts for the reasoning component of citizen-level science literacy and restating these as assessable student learning outcomes (SLOs), we developed a valid and reliable assessment instrument for addressing the outcomes with a brief 25-item science literacy concept inventory (SLCI). In this paper, we report the results that we obtained from assessing the citizen-level science literacy of 17,382 undergraduate students, 149 graduate students, and 181 professors. We address only findings at or above the 99.9% confidence level. We found that general education (GE) science courses do not significantly advance understanding of science as a way of knowing. However, the understanding of science’s way of knowing does increase through academic ranks, indicating that the extended overall academic experience better accounts for increasing such thinking capacity than do science courses alone. Higher mean institutional SLCI scores correlate closely with increased institutional selectivity, as measured by the institutions’ higher mean SAT and ACT scores. Socioeconomic factors of a) first-generation student, b) English as a native language, and c) interest in commitment to a science major are unequally distributed across ethnic groups. These factors proved powerful in accounting for the variations in SLCI scores across ethnicities and genders.

Key Concept Ranking

Lead
0.925
Coffee
0.5848656
Infection
0.48125
Stems
0.4577538
0.925

References & Citations

1. American Association for the Advancement of Science 1989 Science for all Americans Oxford University Press New York, NY
2. American Association for the Advancement of Science 1993 Benchmarks for science literacy Oxford University Press New York, NY
3. Antonellis J, Buxner S, Impey C, Sugarman H 2012 Surveying science literacy among undergraduates: insights from open-ended responses J Coll Sci Teach 413 82 90
4. Brookfield S 2012 Teaching for critical thinking John Wiley and Sons San Francisco, CA
5. Chamberlin TC 1897 The method of multiple working hypotheses J Geol 5 837 848 Reprinted: 1965. Science 148:754–759 10.1086/607980 http://dx.doi.org/10.1086/607980
6. Coil D, Wenderoth MP, Cunningham M, Dirks C 2010 Teaching the process of science: faculty perceptions and an effective methodology CBE Life Sci Educ 9 524 535 10.1187/cbe.10-01-0005 21123699 2995770 http://dx.doi.org/10.1187/cbe.10-01-0005
7. Duit R 2009 Students’ and teachers’ conceptions and science education: full version bibliography [Online.] http://archiv.ipn.uni-kiel.de/stcse/download_stcse.html Accessed 22 August 2015
8. Flavell JH 1976 Metacognitive aspects of problem solving 231 236 Resnick LB The nature of intelligence Erlbaum Hillsdale, NJ
9. Flavell JH 1979 Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry Amer Psychol 34 10 906 911 10.1037/0003-066X.34.10.906 http://dx.doi.org/10.1037/0003-066X.34.10.906
10. Gaze E 2014 Teaching quantitative reasoning: a better context for algebra Numeracy 7 1 Article 1. [Online.] http://scholarcommons.usf.edu/numeracy/vol7/iss1/art1/ Accessed 22 August 2015 10.5038/1936-4660.7.1.1 http://dx.doi.org/10.5038/1936-4660.7.1.1
11. Gaze EC, Montgomery A, Kilic-Bahi S, Leoni D, Misener L, Taylor C 2014 Towards developing a quantitative literacy/reasoning assessment instrument Numeracy 7 2 Article 4. [Online.] http://scholarcommons.usf.edu/numeracy/vol7/iss2/art4/ Accessed 22 August 2015 10.5038/1936-4660.7.2.4 http://dx.doi.org/10.5038/1936-4660.7.2.4
12. Gill RA, Burke IC 1999 Using an environmental science course to promote scientific literacy J Coll Sci Teach 25 105 110
13. Glazer FS 2000 Journal clubs—a successful vehicle to science literacy J Coll Sci Teach 29 5 320 324
14. Gormally C, Brickman P, Lutz M 2012 Developing a test of scientific literacy skills TOSLS: measuring undergraduates’ evaluation of scientific information and arguments CBE Life Sci Educ 11 364 377 10.1187/cbe.12-03-0026 23222832 3516792 http://dx.doi.org/10.1187/cbe.12-03-0026
15. Hallowell C, Holland MJ 1998 Journalism as a path to scientific literacy J Coll Sci Teach 28 1 29 32
16. Hattie J 2009 Visible Learning: A synthesis of over 800 meta-analyses relating to achievement Routledge New York, NY 61 63 68 71
17. Henderson C 2002 Common concerns about the Force Concept Inventory Phys Teach 40 542 547 10.1119/1.1534822 http://dx.doi.org/10.1119/1.1534822
18. Hestenes D, Wells M, Swackhamer G 1992 Force Concept Inventory Phys Teach 30 141 158 10.1119/1.2343497 http://dx.doi.org/10.1119/1.2343497
19. Holton G 1999 1948 The new imperative for science literacy J Coll Sci Teach 28 3 181 185
20. Impey C, Buxner S, Antonellis J, Johnson E, King C 2011 A twenty-year survey of science literacy among college undergraduates J Coll Sci Teach 404 31 37
21. Johnson V 2013 Revised standards for statistical evidence Proc Natl Acad Sci USA 110 19313 19317 10.1073/pnas.1313476110 24218581 3845140 http://dx.doi.org/10.1073/pnas.1313476110
22. Journal of Adult Development 2004 Special volume of nine papers on the Perry legacy of cognitive development J Adult Dev 11 2 59 161 Periodicals Service Co Germantown, NY
23. King PM, Kitchener KS 1994 Developing reflective judgment: understanding and promoting intellectual growth and critical thinking in adolescents and adults Jossey Bass San Francisco, CA
24. Laugksch RC, Spargo PE 1996 Construction of a paper-and-pencil test of basic scientific literacy based on selected literacy goals recommended by the American Association for the Advancement of Science Public Underst Sci 5 331 359 10.1088/0963-6625/5/4/003 http://dx.doi.org/10.1088/0963-6625/5/4/003
25. Lawson AE 1978 Development and validation of the classroom test of formal reasoning J Res Sci Teach 15 1 11 24 10.1002/tea.3660150103 http://dx.doi.org/10.1002/tea.3660150103
26. Libarkin J 2008 Geoscience concept inventory: suggestions for question review and development: Notes Provided from Geological Society of America Annual Meeting Short Course # 523 Writing and Evaluating Geoscience Concept Inventory Questions
27. Maskey CL 2011 An evaluation of the relationship between reflective judgment and critical thinking in senior associate-degree nursing students Doctoral dissertation Indiana University Indianapolis, IN
28. McComas WF 1998 The principal elements of the nature of science: dispelling the myths 53 70 McComas WF The Nature of Science in Science Education Kluwer Academic Publishers Netherlands
29. Miller JD 1983 Scientific literacy: a conceptual and empirical review Daedalus 1122 29 48
30. Miller JD 1998 The measurement of civic scientific literacy Public Underst Sci 7 3 203 223 10.1088/0963-6625/7/3/001 http://dx.doi.org/10.1088/0963-6625/7/3/001
31. National Academy of Sciences and Institute of Medicine 2008 Science, evolution, and creationism The National Academies Press Washington, DC
32. National Center for Education Statistics 2012 Institute of Education Sciences. Digest of Education Statistics 2013, Table 318.20 Bachelor’s, master’s, and doctor’s degrees conferred by postsecondary institutions, by field of study: Selected years, 1970–71 through 2011–12 [Online.] http://nces.ed.gov/programs/digest/d13/tables/dt13_318.20.asp Accessed 23 August 2015
33. NGSS Lead States 2013 Next generation science standards: for states, by states The National Academies Press Washington, DC
34. Nuhfer EB 2013 Understanding Cinderella—expanding an essential conversation on the metadiscipline of technology: educating in fractal patterns XXXVII Natl Teach Learn Forum 22 2 7 10
35. Nuhfer EB, Gaze E 2014 Benefits of assessing citizen literacy in science and quantitative reasoning: educating in fractal patterns XLII Natl Teach Learn Forum 24 1 8 11 10.1002/ntlf.30012 http://dx.doi.org/10.1002/ntlf.30012
36. Nuhfer EB, et al 2010 Multi-campus project: promoting and assessing science literacy in general education science courses California State University, Institute for Teaching and Learning Connections 3 4 [Online.] https://www.calstate.edu/itl/newsletter/10-summer.shtml Accessed 18 August 2015
37. Pavelich MJ, Moore WS 1996 Measuring the effect of experiential education using the Perry model J Engineer Educ 85 4 287 292 10.1002/j.2168-9830.1996.tb00247.x http://dx.doi.org/10.1002/j.2168-9830.1996.tb00247.x
38. Perry WG Jr 1999 Forms of intellectual and ethical development in the college years Reprint of the original 1968 1st edition with introduction by L. Knefelkamp Jossey-Bass San Francisco, CA
39. Seckam A, Cooper R 2013 Understanding how honey impacts on wounds: an update on recent research findings Wounds Internatl 4 1 20 24
40. Selby C 2006 What makes it science?: A modern look at scientific inquiry J Coll Sci Teach 35 7 8 11
41. Specter M 2010 Denialism: how irrational thinking harms the planet and threatens our lives Duckworth Overlook London, UK
42. Steele CM, Aronson J 1995 Stereotype threat and the intellectual test performance of African Americans J Personality Social Psych 69 5 797 811 10.1037/0022-3514.69.5.797 http://dx.doi.org/10.1037/0022-3514.69.5.797
43. Strahler A 1992 Understanding science: an introduction to concepts and issues Prometheus Books Buffalo, NY
44. Strimaitis AM, Schellinger J, Jones A, Grooms J, Sampson V 2014 Development of an instrument to assess student knowledge necessary to critically evaluate scientific claims in the popular media J Coll Sci Teach 43 55 68 10.2505/4/jcst14_043_05_55 http://dx.doi.org/10.2505/4/jcst14_043_05_55
45. Terry DR 2012 Assessing critical thinking skills using articles from the popular press J Coll Sci Teach 42 66 70
46. Trefil J 1992 Science matters: achieving scientific literacy Anchor Books New York, NY
47. Trefil J 2008 Why science? Teachers College Press, National Science Teachers Association Arlington, VA
48. Wenning CJ 2006 Assessing nature-of-science literacy as one component of scientific literacy J Phys Teach Educ Online 34 3 14
49. White B, et al 2011 A novel instrument for assessing students’ critical thinking abilities J Coll Sci Teach 405 102 107
50. Wise SL, DeMars CE 2005 Examinee motivation in low-stakes assessment: problems and potential solutions Educ Assess 10 1 1 17 10.1207/s15326977ea1001_1 http://dx.doi.org/10.1207/s15326977ea1001_1
51. Wolpert L 1992 The unnatural nature of science Harvard University Press Cambridge, MA

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v17i1.1036
2016-03-01
2019-06-20

Abstract:

After articulating 12 concepts for the reasoning component of citizen-level science literacy and restating these as assessable student learning outcomes (SLOs), we developed a valid and reliable assessment instrument for addressing the outcomes with a brief 25-item science literacy concept inventory (SLCI). In this paper, we report the results that we obtained from assessing the citizen-level science literacy of 17,382 undergraduate students, 149 graduate students, and 181 professors. We address only findings at or above the 99.9% confidence level. We found that general education (GE) science courses do not significantly advance understanding of science as a way of knowing. However, the understanding of science’s way of knowing does increase through academic ranks, indicating that the extended overall academic experience better accounts for increasing such thinking capacity than do science courses alone. Higher mean institutional SLCI scores correlate closely with increased institutional selectivity, as measured by the institutions’ higher mean SAT and ACT scores. Socioeconomic factors of a) first-generation student, b) English as a native language, and c) interest in commitment to a science major are unequally distributed across ethnic groups. These factors proved powerful in accounting for the variations in SLCI scores across ethnicities and genders.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/17/1/jmbe-17-143.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v17i1.1036&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Actual undergraduate student performance ( = 17,362) compared with performance through random guessing. Mean overall score = 68.33%. Distribution reveals little total ignorance of science’s way of knowing among respondents, but room exists for significant improvement. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

SLCI mean scores for 17,072 undergraduates based on number of college science courses completed. Little mean difference exists between none, one, and two courses (red font). The height of the diamonds denotes the 99.9% confidence interval, which is the same as the diameters of the circles in the Student’s -test box to the right. Space separating the circles denotes significant differences between course categories. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

SLCI mean scores arranged in increasing order of self-reported ranks for undergraduates, graduate students, and professors. Highly significant differences exist between successively higher academic ranks except for an absence of significant difference between sophomores and juniors (red font). SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint
Image of FIGURE 4

Click to view

FIGURE 4

Gains in SLCI mean scores of freshmen and senior ranks for 12 institutions of varied selectivity. One school with a mean ACT of about 22 showed a decrease, but all seniors sampled at that school came from introductory courses and were not likely representative of the institution’s actual seniors. To do meaningful plots requires a school to obtain a good representative sampling of upper- and lower-division courses. As shown by the slopes of the line fits and their convergence, a ceiling effect makes it more difficult for selective schools to achieve large gains. SLCI = science literacy concept inventory.

Source: J. Microbiol. Biol. Educ. March 2016 vol. 17 no. 1 143-155. doi:10.1128/jmbe.v17i1.1036
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error