1887

Correlating Student Knowledge and Confidence Using a Graded Knowledge Survey to Assess Student Learning in a General Microbiology Classroom

    Authors: Lacey Favazzo1, John D. Willford2, Rachel M. Watson2,*
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Department of Microbiology and Immunology, University of Rochester School of Medicine and Dentistry, Rochester, NY 14642; 2: Department of Molecular Biology, University of Wyoming, Laramie, WY 82071
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 15 December 2014
    • Supplemental materials available at http://jmbe.asm.org
    • *Corresponding author. Mailing address: Department of Molecular Biology, 1000 E. University Ave., Laramie, WY 82071. Phone: 307-766-3524. Fax: 307-766-5098. E-mail: rwatson@uwyo.edu.
    • ©2014 Author(s). Published by the American Society for Microbiology.
    Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    64.89 Kb
  • PDF
    343.71 Kb
  • XML

    Abstract:

    Knowledge surveys are a type of confidence survey in which students rate their confidence in their ability to answer questions rather than answering the questions. These surveys have been discussed as a tool to evaluate student in-class or curriculum-wide learning. However, disagreement exists as to whether confidence is actually an accurate measure of knowledge. With the concomitant goals of assessing content-based learning objectives and addressing this disagreement, we present herein a pretest/posttest knowledge survey study that demonstrates a significant difference correctness on graded test questions at different levels of reported confidence in a multi-semester timeframe. Questions were organized into Bloom’s taxonomy, allowing for the data collected to further provide statistical analyses on strengths and deficits in various levels of Bloom’s reasoning with regard to mean correctness. Collectively, students showed increasing confidence and correctness in all levels of thought but struggled with synthesis-level questions. However, when students were only asked to rate confidence and not answer the accompanying test questions, they reported significantly higher confidence than the control group which was asked to do both. This indicates that when students do not attempt to answer questions, they have significantly greater confidence in their ability to answer those questions. Additionally, when students rate only confidence without answering the question, resolution across Bloom’s levels of reasoning is lost. Based upon our findings, knowledge surveys can be an effective tool for assessment of both breadth and depth of knowledge, but may require students to answer questions in addition to rating confidence to provide the most accurate data.

Key Concept Ranking

Spring
0.6052632
Lead
0.5263158
Canning
0.4221587
0.6052632

References & Citations

1. Anderson L, Krathwohl D2000A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectivesPearsonLondon
2. Ary D, Jacobs LC, Razavieh A, Sorensen C2006Introduction to research in education7th ed1Thomson-WadsworthBelmont, CA
3. Bandura A1989Regulation of cognitive processes through perceived self-efficacyDev Psychol2572973510.1037/0012-1649.25.5.729 http://dx.doi.org/10.1037/0012-1649.25.5.729
4. Bandura A1977Self-efficacy: towards a unifying theory of behavioral changePsychol Rev8419121510.1037/0033-295X.84.2.191847061 http://dx.doi.org/10.1037/0033-295X.84.2.191
5. Bandura A1986Social foundation of thought and actionEnglewood CliffsLondon, UK
6. Bell P, Volckmann D2001Knowledge surveys in general chemistry: confidence, overconfidence, and performanceJ Chem Educ881469147610.1021/ed100328c http://dx.doi.org/10.1021/ed100328c
7. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR1956Taxonomy of educational objectives: The classification of educational goalsHandbook I: Cognitive domainDavid McKay CompanyNew York, NY
8. Bouffard-Bouchard T, Parent S, Larivee S1991Influence of self-efficacy on self-regulation and performance among junior and senior high-school aged studentsInt J Behav Develop1415316410.1177/016502549101400203 http://dx.doi.org/10.1177/016502549101400203
9. Bowers N, Brandon M2006Response: re: the use of a knowledge survey as an indicator of student learning in an introductory biology courseCell Biol Educ531510.1187/cbe.06-07-0173 http://dx.doi.org/10.1187/cbe.06-07-0173
10. Bowers N, Brandon M, Hill CD2005The use of a knowledge survey as an indicator of student learning in an introductory biology courseCell Biol Educ431132210.1187/cbe.04-11-0056163412581305893 http://dx.doi.org/10.1187/cbe.04-11-0056
11. Clauss J, Geedey K2010Knowledge surveys: students’ ability to self-assessJ Sch Teach Learn101424
12. Collins JL1982Self-efficacy and ability in achievement behaviorAnnual Meeting of the American Educational Research AssociationNew York, NY
13. Crozier R1997Individual learners: personality differences in educationRoutledgeLondon
14. Gardiner L1994Redesigning Higher Education: Producing Dramatic Gains in Student Learning, Higher Education Report No. 7ASHE-ERICWashington, DC
15. Koriat A, Lichtenstein S, Fischhov B1980Reasons for confidenceJ Exper Psych Human Learn Mem610711810.1037/0278-7393.6.2.107 http://dx.doi.org/10.1037/0278-7393.6.2.107
16. Lichtenstein S, Fischhoff B, Philips LD1982Calibration of probabilities: the state of the art in 1980 Kahneman D, Slavic P, Tversky AJudgement Under Uncertainty: Heuristics and BiasesCambridge University PressCambridge, UK10.1017/CBO9780511809477.023 http://dx.doi.org/10.1017/CBO9780511809477.023
17. Nuhfer E, Knipp D2003The knowledge survey: a tool for all reasonsTo Improve the Academy215978
18. Nuhfer E, Knipp D2006Re: The use of a knowledge survey as an indicator of student learning in an introductory biology courseCell Biol Educ531331410.1187/cbe.06-05-0166 http://dx.doi.org/10.1187/cbe.06-05-0166
19. O’Neill G, Murphy F2010Guide to taxonomies of learning. UCD Teaching and Learning Assessment Resources. [Online.]Accessed May 2014 at http://www.ucd.ie/t4cms/ucdtla0034.pdf
20. Overbaugh RC, Schultz Ln.d.Bloom’s Taxonomy. [Online.]Accessed August 2009 at http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm
21. Pajares F2002Gender and perceived self-efficacy in self-regulated learningTheory Pract4111612510.1207/s15430421tip4102_8 http://dx.doi.org/10.1207/s15430421tip4102_8
22. Ronis DL, Yates JF1987Components of probability judgement accuracy: individual consistency and effects of subject matter and assessment methodOrganiz Behav Human Decision Proc4019321810.1016/0749-5978(87)90012-4 http://dx.doi.org/10.1016/0749-5978(87)90012-4
23. Sander P, Sanders L2007Measuring confidence in academic study: a summary reportJ Res Educ Psych Ped1117
24. Sieck WR, Merkle EC, Van Zandt T2007Option fixation: a cognitive contributor to overconfidenceOrganiz Behav Human Decision Proc103688310.1016/j.obhdp.2006.11.001 http://dx.doi.org/10.1016/j.obhdp.2006.11.001
25. Sniezek JA, Paese PW, Switzer FSIII1990The effect of choosing on confidence in choiceOrganiz Behav Human Decision Proc4626428210.1016/0749-5978(90)90032-5 http://dx.doi.org/10.1016/0749-5978(90)90032-5
26. Warner RM2008Applied Statistics: from bivariate through multivariate techniquesSage Publications, IncThousand Oaks, CA
27. Wright G1982Changes in realism of probability assessments as a function of question typeActa Psychologica5216517410.1016/0001-6918(82)90033-6 http://dx.doi.org/10.1016/0001-6918(82)90033-6
jmbe.v15i2.693.citations
jmbe/15/2
content/journal/jmbe/10.1128/jmbe.v15i2.693
Loading

Citations loading...

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v15i2.693
2014-12-15
2017-03-25

Abstract:

Knowledge surveys are a type of confidence survey in which students rate their confidence in their ability to answer questions rather than answering the questions. These surveys have been discussed as a tool to evaluate student in-class or curriculum-wide learning. However, disagreement exists as to whether confidence is actually an accurate measure of knowledge. With the concomitant goals of assessing content-based learning objectives and addressing this disagreement, we present herein a pretest/posttest knowledge survey study that demonstrates a significant difference correctness on graded test questions at different levels of reported confidence in a multi-semester timeframe. Questions were organized into Bloom’s taxonomy, allowing for the data collected to further provide statistical analyses on strengths and deficits in various levels of Bloom’s reasoning with regard to mean correctness. Collectively, students showed increasing confidence and correctness in all levels of thought but struggled with synthesis-level questions. However, when students were only asked to rate confidence and not answer the accompanying test questions, they reported significantly higher confidence than the control group which was asked to do both. This indicates that when students do not attempt to answer questions, they have significantly greater confidence in their ability to answer those questions. Additionally, when students rate only confidence without answering the question, resolution across Bloom’s levels of reasoning is lost. Based upon our findings, knowledge surveys can be an effective tool for assessment of both breadth and depth of knowledge, but may require students to answer questions in addition to rating confidence to provide the most accurate data.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/15/2/jmbe-15-251.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v15i2.693&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1.

Click to view

FIGURE 1.

Average graded correctness for each reported confidence level. a, b, c, d, and e define significantly different subsets within the pretest (dark gray bars) and posttest (light gray bars), respectively. Reported confidence level is depicted on the X-axis with mean graded correctness on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 2.

Click to view

FIGURE 2.

Correlation of student average confidence to average graded correctness on the pretest. Linear trend line depicted. = 0.615; R = 0.378.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 3.

Click to view

FIGURE 3.

Correlation of student average confidence to average graded correctness on the posttest. Linear trend line depicted. = 0.612; R = 0.375.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 4.

Click to view

FIGURE 4.

Average reported confidence and comparison of final course grade by test group. Test group is depicted on the X-axis with the average reported confidence on the Y-axis. The pretest data (dark gray bars) and posttest data (light gray bars) for both the A group (students both answering the question and rating confidence) and B group (students only rating confidence) are shown along with the average final course grade (flat line above each group) which was utilized to compare the knowledge of each group. Final grades for each group are the average final course grade percentage. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 5.

Click to view

FIGURE 5.

Average reported confidence by Bloom’s level of reasoning for test group A. a, b, c, d define significantly different subsets within the pretest (dark gray bars) and posttest (light gray bars), respectively. Bloom’s level of reasoning is depicted on the X-axis with average reported confidence on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 6.

Click to view

FIGURE 6.

Posttest average confidence for Bloom’s reasoning level by test group. a, b, c, and d define significantly different subsets within the A group (students both answering the question and rating confidence – dark gray bars) and B group (students only rating confidence – light gray bars), respectively. Bloom’s level of reasoning is depicted on the X-axis with average reported posttest confidence on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error