1887

Correlating Student Knowledge and Confidence Using a Graded Knowledge Survey to Assess Student Learning in a General Microbiology Classroom

    Authors: Lacey Favazzo1, John D. Willford2, Rachel M. Watson2,*
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Department of Microbiology and Immunology, University of Rochester School of Medicine and Dentistry, Rochester, NY 14642; 2: Department of Molecular Biology, University of Wyoming, Laramie, WY 82071
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 15 December 2014
    • Supplemental materials available at http://jmbe.asm.org
    • *Corresponding author. Mailing address: Department of Molecular Biology, 1000 E. University Ave., Laramie, WY 82071. Phone: 307-766-3524. Fax: 307-766-5098. E-mail: [email protected].
    • ©2014 Author(s). Published by the American Society for Microbiology.
    Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    64.89 Kb
  • PDF
    343.71 Kb
  • XML

    Abstract:

    Knowledge surveys are a type of confidence survey in which students rate their confidence in their ability to answer questions rather than answering the questions. These surveys have been discussed as a tool to evaluate student in-class or curriculum-wide learning. However, disagreement exists as to whether confidence is actually an accurate measure of knowledge. With the concomitant goals of assessing content-based learning objectives and addressing this disagreement, we present herein a pretest/posttest knowledge survey study that demonstrates a significant difference correctness on graded test questions at different levels of reported confidence in a multi-semester timeframe. Questions were organized into Bloom’s taxonomy, allowing for the data collected to further provide statistical analyses on strengths and deficits in various levels of Bloom’s reasoning with regard to mean correctness. Collectively, students showed increasing confidence and correctness in all levels of thought but struggled with synthesis-level questions. However, when students were only asked to rate confidence and not answer the accompanying test questions, they reported significantly higher confidence than the control group which was asked to do both. This indicates that when students do not attempt to answer questions, they have significantly greater confidence in their ability to answer those questions. Additionally, when students rate only confidence without answering the question, resolution across Bloom’s levels of reasoning is lost. Based upon our findings, knowledge surveys can be an effective tool for assessment of both breadth and depth of knowledge, but may require students to answer questions in addition to rating confidence to provide the most accurate data.

Key Concept Ranking

Spring
0.6052632
Lead
0.5263158
Canning
0.4221587
0.6052632

References & Citations

1. Anderson L, Krathwohl D 2000 A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives Pearson London
2. Ary D, Jacobs LC, Razavieh A, Sorensen C 2006 Introduction to research in education 7th ed 1 Thomson-Wadsworth Belmont, CA
3. Bandura A 1989 Regulation of cognitive processes through perceived self-efficacy Dev Psychol 25 729 735 10.1037/0012-1649.25.5.729 http://dx.doi.org/10.1037/0012-1649.25.5.729
4. Bandura A 1977 Self-efficacy: towards a unifying theory of behavioral change Psychol Rev 84 191 215 10.1037/0033-295X.84.2.191 847061 http://dx.doi.org/10.1037/0033-295X.84.2.191
5. Bandura A 1986 Social foundation of thought and action Englewood Cliffs London, UK
6. Bell P, Volckmann D 2001 Knowledge surveys in general chemistry: confidence, overconfidence, and performance J Chem Educ 88 1469 1476 10.1021/ed100328c http://dx.doi.org/10.1021/ed100328c
7. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR 1956 Taxonomy of educational objectives: The classification of educational goals Handbook I: Cognitive domain David McKay Company New York, NY
8. Bouffard-Bouchard T, Parent S, Larivee S 1991 Influence of self-efficacy on self-regulation and performance among junior and senior high-school aged students Int J Behav Develop 14 153 164 10.1177/016502549101400203 http://dx.doi.org/10.1177/016502549101400203
9. Bowers N, Brandon M 2006 Response: re: the use of a knowledge survey as an indicator of student learning in an introductory biology course Cell Biol Educ 5 315 10.1187/cbe.06-07-0173 http://dx.doi.org/10.1187/cbe.06-07-0173
10. Bowers N, Brandon M, Hill CD 2005 The use of a knowledge survey as an indicator of student learning in an introductory biology course Cell Biol Educ 4 311 322 10.1187/cbe.04-11-0056 16341258 1305893 http://dx.doi.org/10.1187/cbe.04-11-0056
11. Clauss J, Geedey K 2010 Knowledge surveys: students’ ability to self-assess J Sch Teach Learn 10 14 24
12. Collins JL 1982 Self-efficacy and ability in achievement behavior Annual Meeting of the American Educational Research Association New York, NY
13. Crozier R 1997 Individual learners: personality differences in education Routledge London
14. Gardiner L 1994 Redesigning Higher Education: Producing Dramatic Gains in Student Learning, Higher Education Report No. 7 ASHE-ERIC Washington, DC
15. Koriat A, Lichtenstein S, Fischhov B 1980 Reasons for confidence J Exper Psych Human Learn Mem 6 107 118 10.1037/0278-7393.6.2.107 http://dx.doi.org/10.1037/0278-7393.6.2.107
16. Lichtenstein S, Fischhoff B, Philips LD 1982 Calibration of probabilities: the state of the art in 1980 Kahneman D, Slavic P, Tversky A Judgement Under Uncertainty: Heuristics and Biases Cambridge University Press Cambridge, UK 10.1017/CBO9780511809477.023 http://dx.doi.org/10.1017/CBO9780511809477.023
17. Nuhfer E, Knipp D 2003 The knowledge survey: a tool for all reasons To Improve the Academy 21 59 78
18. Nuhfer E, Knipp D 2006 Re: The use of a knowledge survey as an indicator of student learning in an introductory biology course Cell Biol Educ 5 313 314 10.1187/cbe.06-05-0166 http://dx.doi.org/10.1187/cbe.06-05-0166
19. O’Neill G, Murphy F 2010 Guide to taxonomies of learning. UCD Teaching and Learning Assessment Resources. [Online.] Accessed May 2014 at http://www.ucd.ie/t4cms/ucdtla0034.pdf
20. Overbaugh RC, Schultz L n.d. Bloom’s Taxonomy. [Online.] Accessed August 2009 at http://ww2.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm
21. Pajares F 2002 Gender and perceived self-efficacy in self-regulated learning Theory Pract 41 116 125 10.1207/s15430421tip4102_8 http://dx.doi.org/10.1207/s15430421tip4102_8
22. Ronis DL, Yates JF 1987 Components of probability judgement accuracy: individual consistency and effects of subject matter and assessment method Organiz Behav Human Decision Proc 40 193 218 10.1016/0749-5978(87)90012-4 http://dx.doi.org/10.1016/0749-5978(87)90012-4
23. Sander P, Sanders L 2007 Measuring confidence in academic study: a summary report J Res Educ Psych Ped 1 1 17
24. Sieck WR, Merkle EC, Van Zandt T 2007 Option fixation: a cognitive contributor to overconfidence Organiz Behav Human Decision Proc 103 68 83 10.1016/j.obhdp.2006.11.001 http://dx.doi.org/10.1016/j.obhdp.2006.11.001
25. Sniezek JA, Paese PW, Switzer FS III 1990 The effect of choosing on confidence in choice Organiz Behav Human Decision Proc 46 264 282 10.1016/0749-5978(90)90032-5 http://dx.doi.org/10.1016/0749-5978(90)90032-5
26. Warner RM 2008 Applied Statistics: from bivariate through multivariate techniques Sage Publications, Inc Thousand Oaks, CA
27. Wright G 1982 Changes in realism of probability assessments as a function of question type Acta Psychologica 52 165 174 10.1016/0001-6918(82)90033-6 http://dx.doi.org/10.1016/0001-6918(82)90033-6

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v15i2.693
2014-12-15
2019-06-16

Abstract:

Knowledge surveys are a type of confidence survey in which students rate their confidence in their ability to answer questions rather than answering the questions. These surveys have been discussed as a tool to evaluate student in-class or curriculum-wide learning. However, disagreement exists as to whether confidence is actually an accurate measure of knowledge. With the concomitant goals of assessing content-based learning objectives and addressing this disagreement, we present herein a pretest/posttest knowledge survey study that demonstrates a significant difference correctness on graded test questions at different levels of reported confidence in a multi-semester timeframe. Questions were organized into Bloom’s taxonomy, allowing for the data collected to further provide statistical analyses on strengths and deficits in various levels of Bloom’s reasoning with regard to mean correctness. Collectively, students showed increasing confidence and correctness in all levels of thought but struggled with synthesis-level questions. However, when students were only asked to rate confidence and not answer the accompanying test questions, they reported significantly higher confidence than the control group which was asked to do both. This indicates that when students do not attempt to answer questions, they have significantly greater confidence in their ability to answer those questions. Additionally, when students rate only confidence without answering the question, resolution across Bloom’s levels of reasoning is lost. Based upon our findings, knowledge surveys can be an effective tool for assessment of both breadth and depth of knowledge, but may require students to answer questions in addition to rating confidence to provide the most accurate data.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/15/2/jmbe-15-251.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v15i2.693&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1.

Click to view

FIGURE 1.

Average graded correctness for each reported confidence level. a, b, c, d, and e define significantly different subsets within the pretest (dark gray bars) and posttest (light gray bars), respectively. Reported confidence level is depicted on the X-axis with mean graded correctness on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 2.

Click to view

FIGURE 2.

Correlation of student average confidence to average graded correctness on the pretest. Linear trend line depicted. = 0.615; R = 0.378.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 3.

Click to view

FIGURE 3.

Correlation of student average confidence to average graded correctness on the posttest. Linear trend line depicted. = 0.612; R = 0.375.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 4.

Click to view

FIGURE 4.

Average reported confidence and comparison of final course grade by test group. Test group is depicted on the X-axis with the average reported confidence on the Y-axis. The pretest data (dark gray bars) and posttest data (light gray bars) for both the A group (students both answering the question and rating confidence) and B group (students only rating confidence) are shown along with the average final course grade (flat line above each group) which was utilized to compare the knowledge of each group. Final grades for each group are the average final course grade percentage. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 5.

Click to view

FIGURE 5.

Average reported confidence by Bloom’s level of reasoning for test group A. a, b, c, d define significantly different subsets within the pretest (dark gray bars) and posttest (light gray bars), respectively. Bloom’s level of reasoning is depicted on the X-axis with average reported confidence on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint
Image of FIGURE 6.

Click to view

FIGURE 6.

Posttest average confidence for Bloom’s reasoning level by test group. a, b, c, and d define significantly different subsets within the A group (students both answering the question and rating confidence – dark gray bars) and B group (students only rating confidence – light gray bars), respectively. Bloom’s level of reasoning is depicted on the X-axis with average reported posttest confidence on the Y-axis. Error bars define a 95% confidence interval.

Source: J. Microbiol. Biol. Educ. December 2014 vol. 15 no. 2 251-258. doi:10.1128/jmbe.v15i2.693
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error