1887

Development, Validation, and Application of the Microbiology Concept Inventory

    Authors: Timothy D. Paustian1,*, Amy G. Briggs2, Robert E. Brennan3, Nancy Boury4, John Buchner5, Shannon Harris1, Rachel E. A. Horak6, Lee E. Hughes7, D. Sue Katz-Amburn8, Maria J. Massimelli9, Ann H. McDonald10, Todd P. Primm11, Ann C. Smith5, Ann M. Stevens12, Sunny B. Yung11
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: University of Wisconsin-Madison, Madison, WI 53706; 2: Beloit College, Beloit, WI 53511; 3: University of Central Oklahoma, Edmond, OK 73034; 4: Iowa State University, Ames, IA 50011; 5: University of Maryland, College Park, MD 20742; 6: American Society of Microbiology, Washington, DC 20036; 7: University of North Texas, Denton, TX 76203; 8: Rogers State University, Claremore, OK 74017; 9: University of California – Irvine, Irvine, CA 92697; 10: Concordia University Wisconsin, Mequon, WI 53097; 11: Sam Houston State University, Huntsville, TX 77340; 12: Virginia Tech, Blacksburg, VA, 24061
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Received 13 March 2017 Accepted 28 July 2017 Published 05 October 2017
    • ©2017 Author(s). Published by the American Society for Microbiology
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://asmscience.org/jmbe
    • *Corresponding author: Mailing address: Microbial Sciences Building, Room 2517, University of Wisconsin-Madison, 1550 Linden Drive, Madison, WI 53706. Phone: 608-263-4921. E-mail: paustian@wisc.edu.
    Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • PDF
    577.79 Kb
  • XML
    84.60 Kb
  • HTML
    67.70 Kb

    Abstract:

    If we are to teach effectively, tools are needed to measure student learning. A widely used method for quickly measuring student understanding of core concepts in a discipline is the concept inventory (CI). Using the American Society for Microbiology Curriculum Guidelines (ASMCG) for microbiology, faculty from 11 academic institutions created and validated a new microbiology concept inventory (MCI). The MCI was developed in three phases. In phase one, learning outcomes and fundamental statements from the ASMCG were used to create T/F questions coupled with open responses. In phase two, the 743 responses to MCI 1.0 were examined to find the most common misconceptions, which were used to create distractors for multiple-choice questions. MCI 2.0 was then administered to 1,043 students. The responses of these students were used to create MCI 3.0, a 23-question CI that measures students’ understanding of all 27 fundamental statements. MCI 3.0 was found to be reliable, with a Cronbach’s alpha score of 0.705 and Ferguson’s delta of 0.97. Test item analysis demonstrated good validity and discriminatory power as judged by item difficulty, item discrimination, and point-biserial correlation coefficient. Comparison of pre- and posttest scores showed that microbiology students at 10 institutions showed an increase in understanding of concepts after instruction, except for questions probing metabolism (average normalized learning gain was 0.15). The MCI will enable quantitative analysis of student learning gains in understanding microbiology, help to identify misconceptions, and point toward areas where efforts should be made to develop teaching approaches to overcome them.

Key Concept Ranking

Horizontal Gene Transfer
0.44602397
0.44602397

References & Citations

1. President’s Council of Advisors on Science and Technology (PCAST)2012Engage to excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematicsExecutive Office of the PresidentWashington, DC
2. Handelsman J, Ebert-May D, Beichner R, Bruns P, Chang A, Dehaan R, Gentile J, Lauffer S, Stewart J, Tilghman SM, Wood WB2004Scientific teachingSource Sci New Ser304521522
3. Dirks C, Wenderoth MP, Withers M2014Assessment in the college science classroomW. H. FreemanUnited States
4. Brown PC, Roediger HLIII, McDaniel MA2014Make it stick: the science of successful learningBelknap PressUnited States10.4159/9780674419377 http://dx.doi.org/10.4159/9780674419377
5. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP2014Active learning increases student performance in science, engineering, and mathematicsProc Natl Acad Sci USA1118410841510.1073/pnas.1319030111248217564060654 http://dx.doi.org/10.1073/pnas.1319030111
6. Hestenes D, Wells M, Swackhamer G1992Force concept inventoryPhys Teach3014110.1119/1.2343497 http://dx.doi.org/10.1119/1.2343497
7. Savinainen A, Scott P2002The force concept inventory: a tool for monitoring student learningPhys Educ37455210.1088/0031-9120/37/1/306 http://dx.doi.org/10.1088/0031-9120/37/1/306
8. Sadler PM, Coyle H, Miller JL, Cook-Smith N, Dussault M, Gould RR2009The astronomy and space science concept inventory: development and validation of assessment instruments aligned with the K–12 national science standardsAstron Educ Rev81011110.3847/AER2009024 http://dx.doi.org/10.3847/AER2009024
9. Aslanides JS, Savage CM2013Relativity concept inventory: development, analysis, and resultsPhys Rev Spec Top Phys Educ Res9
10. Herman GL, Loui M, Zilles C2010Creating the digital logic concept inventory102106Proceedings of the 41st ACM Technical Symposium on Computer Science Education
11. Stone A, Allen K, Rhoads TR, Murphy TJ, Shehab RL, Saha C2003The statistics concept inventory: a pilot studyT3D1T3D6Proceedings – Frontiers in Education Conference, FIE
12. Epstein J2007Development and validation of the calculus concept inventory165170Proceedings of the Ninth International Conference on Mathematics Education in a Global Community
13. Almstrum VL, Henderson PB, Harvey V, Heeren C, Marion W, Riedesel C, Soh L-K, Tew AE2006Concept inventories in computer science for the topic discrete mathematicsACM SIGCSE Bull3813210.1145/1189136.1189182 http://dx.doi.org/10.1145/1189136.1189182
14. Krause S, Birk J, Bauer R, Jenkins B, Pavelich MJ2004Development, testing, and application of a chemistry concept inventory10310734th Annual Frontiers in Education, 2004. FIE 2004
15. Knudson D2006Biomechanics concept inventoryPercept Mot Skills103818210.2466/pms.103.1.81-8217037646 http://dx.doi.org/10.2466/pms.103.1.81-82
16. D’Avanzo C2008Biology concept inventories: overview, status, and next stepsBioscience58107910.1641/B581111 http://dx.doi.org/10.1641/B581111
17. Newman DL, Snyder CW, Fisk JN, Wright LK2016Development of the central dogma concept inventory (CDCI) assessment toolCBE Life Sci Educ152ar910.1187/cbe.15-06-0124270557754909347 http://dx.doi.org/10.1187/cbe.15-06-0124
18. Wang JR2004Development and validation of a two-tier instrument to examine understanding of internal transport in plants and the human circulatory systemInt J Sci Math Educ213115710.1007/s10763-004-9323-2 http://dx.doi.org/10.1007/s10763-004-9323-2
19. Deane T, Nomme K, Jeffery E, Pollock C, Birol G2014Development of the biological experimental design concept inventory (BEDCI)CBE Life Sci Educ1354055110.1187/cbe.13-11-0218251852364152214 http://dx.doi.org/10.1187/cbe.13-11-0218
20. Lin SW2004Development and application of a two-tier diagnostic test for high school students’ understanding of flowering plant growth and developmentInt J Sci Math Educ217519910.1007/s10763-004-6484-y http://dx.doi.org/10.1007/s10763-004-6484-y
21. Odom AL, Barrow LH1995Development and application of a two-tier diagnostic test measuring college biology students’ understanding of diffusion and osmosis after a course of instructionJ Res Sci Teach32456110.1002/tea.3660320106 http://dx.doi.org/10.1002/tea.3660320106
22. Smith MK, Wood WB, Knight JK2008The genetics concept assessment: a new concept inventory for gauging student understanding of geneticsCBE Life Sci Educ742243010.1187/cbe.08-08-0045190474282592048 http://dx.doi.org/10.1187/cbe.08-08-0045
23. Kalas P, O’Neill A, Pollock C, Birol G2013Development of a meiosis concept inventoryCBE Life Sci Educ1265566410.1187/cbe.12-10-0174242972923846516 http://dx.doi.org/10.1187/cbe.12-10-0174
24. Couch BA, Wood WB, Knight JK2015The molecular biology capstone assessment: a concept assessment for upper-division molecular biology studentsCBE Life Sci Educ14ar1010.1187/cbe.14-04-0071257130984353076 http://dx.doi.org/10.1187/cbe.14-04-0071
25. Stefanski KM, Gardner GE, Seipelt-Thiemann RL2016Development of a lac operon concept inventory (LOCI)CBE Life Sci Educ15ar2410.1187/cbe.15-07-0162272523004909346 http://dx.doi.org/10.1187/cbe.15-07-0162
26. Marbach-Ad G, Briken V, El-Sayed NM, Frauwirth K, Fredericksen B, Hutcheson S, Gao LY, Joseph SW, Lee V, McIver KS, Mosser D, Booth Quimby B, Shields P, Song W, Stein DC, Yuan RT, Smith AC2009Assessing student understanding of host-pathogen interactions using a concept inventoryJ Microbiol Biol Educ10435010.1128/jmbe.v10.98 http://dx.doi.org/10.1128/jmbe.v10.98
27. Horak REA, Merkel S, Chang A2015The ASM curriculum guidelines for undergraduate microbiology: a case study of the advocacy role of societies in reform effortsJ Microbiol Biol Educ1610010410.1128/jmbe.v16i1.915259497694416496 http://dx.doi.org/10.1128/jmbe.v16i1.915
28. Stevens A, Liao M-K, Merkel S, Chang A2014ASM task force for learning outcomes – ASM general microbiology learning outcome exampleshttps://www.asm.org/images/Education/FINAL_Learning_Outcomes_w_title_page.pdf
29. Seitz HM, Horak REA, Howard MW, Kluckhohn Jones LW, Muth T, Parker C, Rediske AP, Whitehurst MM2017Development and validation of the microbiology for health sciences concept inventoryJ Microbiol Biol Educ183110
30. Briggs AG, Hughes LE, Brennan RE, Buchner J, Horak REA, Katz-Amburn DS, McDonald AH, Primm TP, Smith AC, Stevens AM, Yung SB, Paustian TD2017Concept inventory development reveals common student misconceptions about microbiologyJ Microbiol Biol Educ18319
31. Adams WK, Wieman CE2011Development and validation of instruments to measure learning of expert-like thinkingInt J Sci Educ331289131210.1080/09500693.2010.512369 http://dx.doi.org/10.1080/09500693.2010.512369
32. Merkel SASM Task Force on Curriculum Guidelines for Undergraduate Biology2012The development of curricular guidelines for introductory microbiology that focus on understandingJ Microbiol Biol Educ13323810.1128/jmbe.v13i1.363236537793577306 http://dx.doi.org/10.1128/jmbe.v13i1.363
33. Meltzer DE2002The relationship between mathematics preparation and conceptual learning gains in physics: a possible “hidden variable” in diagnostic pretest scoresAm J Phys70125910.1119/1.1514215 http://dx.doi.org/10.1119/1.1514215
34. Crocker LM, Algina J1986Introduction to classical and modern test theoryHolt, Rinehart, and WinstonNew York
35. Ebel RL1954Procedures for the analysis of classroom testsEduc Psychol Meas1435236410.1177/001316445401400215 http://dx.doi.org/10.1177/001316445401400215
36. Bardar EM, Prather EE, Brecher K, Slater TF2006Development and validation of the light and spectroscopy concept inventoryAstron Educ Rev510311310.3847/AER2006020 http://dx.doi.org/10.3847/AER2006020
37. Ding L, Chabay R, Sherwood B, Beichner R2006Evaluating an electricity and magnetism assessment tool: brief electricity and magnetism assessmentPhys Rev Spec Top Phys Educ Res2
38. Ferguson GA1949On the theory of test discriminationPsychometrika14616810.1007/BF0229014118113205 http://dx.doi.org/10.1007/BF02290141
39. Anderson DL, Fisher KM, Norman GJ2002Development and evaluation of the conceptual inventory of natural selectionJ Res Sci Teach3995297810.1002/tea.10053 http://dx.doi.org/10.1002/tea.10053
jmbe.v18i3.1320.citations
jmbe/18/3
content/journal/jmbe/10.1128/jmbe.v18i3.1320
Loading

Citations loading...

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v18i3.1320
2017-10-05
2017-12-17

Abstract:

If we are to teach effectively, tools are needed to measure student learning. A widely used method for quickly measuring student understanding of core concepts in a discipline is the concept inventory (CI). Using the American Society for Microbiology Curriculum Guidelines (ASMCG) for microbiology, faculty from 11 academic institutions created and validated a new microbiology concept inventory (MCI). The MCI was developed in three phases. In phase one, learning outcomes and fundamental statements from the ASMCG were used to create T/F questions coupled with open responses. In phase two, the 743 responses to MCI 1.0 were examined to find the most common misconceptions, which were used to create distractors for multiple-choice questions. MCI 2.0 was then administered to 1,043 students. The responses of these students were used to create MCI 3.0, a 23-question CI that measures students’ understanding of all 27 fundamental statements. MCI 3.0 was found to be reliable, with a Cronbach’s alpha score of 0.705 and Ferguson’s delta of 0.97. Test item analysis demonstrated good validity and discriminatory power as judged by item difficulty, item discrimination, and point-biserial correlation coefficient. Comparison of pre- and posttest scores showed that microbiology students at 10 institutions showed an increase in understanding of concepts after instruction, except for questions probing metabolism (average normalized learning gain was 0.15). The MCI will enable quantitative analysis of student learning gains in understanding microbiology, help to identify misconceptions, and point toward areas where efforts should be made to develop teaching approaches to overcome them.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/18/3/jmbe-18-49.html?itemId=/content/journal/jmbe/10.1128/jmbe.v18i3.1320&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Item difficulty and Item discrimination pre- vs. posttest. A total of 1,161 student surveys were used to determine question difficulty and discrimination. The dashed line indicates where each question would land if there was no change in difficulty or discrimination. Measured difficulty of each question decreased after instruction (the difficulty score increased). The discriminatory power of each question increased in the posttest.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

Performance by question, pre- vs. posttest. Comparison of the average number of students answering correctly in the Pre-Test ( ) vs. the Post-Test ( ). Students showed improvement in all but questions 11, 12, and 13.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

Normalized learning gains pre- vs. posttest. The normalized learning gains for each student by question. A total of 1,161 pre- and post- surveys from 10 colleges were analyzed per question. Positive learning gains were found for all but questions 11, 12, and 13.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error