1887

Problem-Solving Skills Among Precollege Students in Clinical Immunology and Microbiology: Classifying Strategies with a Rubric and Artificial Neural Network Technology

    Authors: SUSAN KANOWITH-KLEIN1,*, MEL STAVE2, RON STEVENS1, ADRIAN M. CASILLAS3
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Department of Microbiology and Immunology and; 3: Department of Medicine, UCLA School of Medicine, Los Angeles, California 9009; 2: Ulysses S. Grant High School, Valley Glen, California 91401
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • *Corresponding author. Mailing address: Department of Medicine, UCLA School of Medicine, Box 957144, Los Angeles, CA 90095-7144. Phone: (310)206-7067. Fax: (310)267-0395. E-mail: [email protected].
    • Copyright © 2001, American Society for Microbiology. All Rights Reserved.
    Source: J. Microbiol. Biol. Educ. May 2001 vol. 2 no. 1 25-33. doi:10.1128/154288101X14285805896158
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • PDF
    191.40 Kb
  • XML
  • HTML
    65.57 Kb

    Abstract:

    Educators emphasize the importance of problem solving that enables students to apply current knowledge and understanding in new ways to previously unencountered situations. Yet few methods are available to visualize and then assess such skills in a rapid and efficient way. Using a software system that can generate a picture (i.e., map) of students’ strategies in solving problems, we investigated methods to classify problem-solving strategies of high school students who were studying infectious and noninfectious diseases. Using maps that indicated items students accessed to solve a software simulation as well as the sequence in which items were accessed, we developed a rubric to score the quality of the student performances and also applied artificial neural network technology to cluster student performances into groups of related strategies. Furthermore, we established that a relationship existed between the rubric and neural network results, suggesting that the quality of a problem-solving strategy could be predicted from the cluster of performances in which it was assigned by the network. Using artificial neural networks to assess students’ problem-solving strategies has the potential to permit the investigation of the problem-solving performances of hundreds of students at a time and provide teachers with a valuable intervention tool capable of identifying content areas in which students have specific misunderstandings, gaps in learning, or misconceptions.

Key Concept Ranking

Enzyme-Linked Immunosorbent Assay
0.50179094
Hepatitis A
0.42857143
Infectious Diseases
0.40278968
0.50179094

References & Citations

1. Arocha JF, Patel VL, Patel YC 1993 Hypothesisgeneration and the coordination of theory and evidence in novice diagnostic reasoning Med Decis Making 13 198 211 10.1177/0272989X9301300305 8412548 http://dx.doi.org/10.1177/0272989X9301300305
2. Bransford JD, Brown AL, Cocking RR 1999 How experts differ from novices 19 38 How people learn National Academy Press Washington, D.C.
3. Casillas AM, Clyman SG, Stephen G, Fan YV, Stevens RH 2000 Exploring alternative models of complex patient management with artificial neural networks Adv. in Health Sci. Educ. 5 23 41 10.1023/A:1009802528071 http://dx.doi.org/10.1023/A:1009802528071
4. Dhillon AS 1998 Individual differences within problem-solving strategies used in physics Sci Educ 82 379 405 10.1002/(SICI)1098-237X(199806)82:3<379::AID-SCE5>3.0.CO;2-9 http://dx.doi.org/10.1002/(SICI)1098-237X(199806)82:3<379::AID-SCE5>3.0.CO;2-9
5. Fatemi E 1999 Building the digital curriculum Education Week/Technology Counts ‘99 19 23 3437049
6. Lawton M 2000 Making the most of assessments Technol. Counts ‘98 18 59 62 3495945
7. National Research Council 1996 National science education standards National Academy Press Washington, D.C.
8. Palacio-Cayetano J 1997 Ph.D.Thesis. Problem-solving skills in high school biology: the effectiveness of the IMMEX problem-solving assessment software. University of Southern California, Los Angeles.
9. Palacio-Cayetano J, Allen RD, Stevens RH 1999 Computer-assisted evaluation—the next generation The Am Biol Teacher 61 514 522 10.2307/4450754 http://dx.doi.org/10.2307/4450754
10. Palacio-Cayetano J, Kanowith-Klein S, Stevens R 1999 UCLA’s outreach program of science education in the Los Angeles schools Acad Med 74 348 351 10.1097/00001888-199904000-00021 10219207 http://dx.doi.org/10.1097/00001888-199904000-00021
11. Refenes AN, Zapranis N, Francis G 1994 Stock performance modeling using neural networks: a comparative study with regression models Neural Networks 7 375 388 10.1016/0893-6080(94)90030-2 http://dx.doi.org/10.1016/0893-6080(94)90030-2
12. Reggie J 2000 Neural computation in medicine Artificial Intelligence in Med. 5 143 158 10.1016/0933-3657(93)90014-T http://dx.doi.org/10.1016/0933-3657(93)90014-T
13. Roth WM 2000 Artificial neural networks for modeling knowing and learning in science J Staff Dev 37 63 80
14. Rumelhart DE, McClelland JL 1986 Parallel distributed processing: explorations in the microstructure of cognition MIT Press Cambridge, Mass
15. Sivaramakrishnan M, Arocha JF, Patel VL 1998 Cognitive assessment and health education in children from two different cultures Soc Sci Med 47 697 712 10.1016/S0277-9536(98)00094-X 9690818 http://dx.doi.org/10.1016/S0277-9536(98)00094-X
16. Stevens R, Ikeda J, Casillas A, Palacio-Cayetano J, Clyman S 1999 Artificial neural network-based performance assessments Computers in Hum. Behavior 15 295 313 10.1016/S0747-5632(99)00025-4 http://dx.doi.org/10.1016/S0747-5632(99)00025-4
17. Stevens RH 1991 Search path mapping: a versatile approach for visualizing problem-solving behavior Acad Med 66 S73 S75 10.1097/00001888-199109000-00046 1930536 http://dx.doi.org/10.1097/00001888-199109000-00046
18. Stevens RH, Kwak AR, McCoy JM 1989 Evaluating preclinical medical students by using computer-based problem-solving examinations Acad Med 64 685 687 10.1097/00001888-198911000-00018 2679616 http://dx.doi.org/10.1097/00001888-198911000-00018
19. Stevens RH, Lopo AC 1994 Artificial neural network comparison of expert and novice problem-solving strategies Proc Annu Symp Comput Appl Med Care 1 Suppl. 64
20. Stevens RH, Lopo AC, Wang P 1996 Artificial neural networks can distinguish novice and expert strategies during complex problem solving J Am Med Inform Assoc 3 131 138 10.1136/jamia.1996.96236281 8653449 http://dx.doi.org/10.1136/jamia.1996.96236281
21. Stevens RH, McCoy JM, Kwak AR 1991 Solving the problem of how medical students solve problems MD Comput 8 13 20 2011052
22. Stevens RH, Vendlinski T, Palacio-Cayetano J, Underdahl J, Paek P, Sprang M, Simpson E 2000 Developing and implementing a K-12 technology program of case-based reasoning Manuscript in preparation.
23. Trotter A 1998 Putting school technology to the test. Education Week/Technology Counts ‘98 18 1
24. Van Melle E, Tomalty L 2000 Using computer technology to foster learning for understanding Microbiol Educ 1 7 13

Supplemental Material

No supplementary material available for this content.

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/154288101X14285805896158
2001-05-01
2019-09-21

Abstract:

Educators emphasize the importance of problem solving that enables students to apply current knowledge and understanding in new ways to previously unencountered situations. Yet few methods are available to visualize and then assess such skills in a rapid and efficient way. Using a software system that can generate a picture (i.e., map) of students’ strategies in solving problems, we investigated methods to classify problem-solving strategies of high school students who were studying infectious and noninfectious diseases. Using maps that indicated items students accessed to solve a software simulation as well as the sequence in which items were accessed, we developed a rubric to score the quality of the student performances and also applied artificial neural network technology to cluster student performances into groups of related strategies. Furthermore, we established that a relationship existed between the rubric and neural network results, suggesting that the quality of a problem-solving strategy could be predicted from the cluster of performances in which it was assigned by the network. Using artificial neural networks to assess students’ problem-solving strategies has the potential to permit the investigation of the problem-solving performances of hundreds of students at a time and provide teachers with a valuable intervention tool capable of identifying content areas in which students have specific misunderstandings, gaps in learning, or misconceptions.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/2/1/jmbe-2-1-25.xml.a.html?itemId=/content/journal/jmbe/10.1128/154288101X14285805896158&mimeType=html&fmt=ahah

Figures

Image of FIG. 1

Click to view

FIG. 1

Classification with a rubric: alternative approaches to solving the chemical (latex) allergy case. (A) Rubric score 1: student accessed relevant data (circled) and moved directly to the solution. The search-path map is created by overlaying the student performance or sequence of student selections on the template and by clearing the menu items not selected. The line moves sequentially from the upper left-hand corner of the first menu item to the bottom-middle border of the next menu item. The different shades of the boxed menu items indicate these selections are from different content domains available in the main menu of the problem. Additional details may be found at the IMMEX website (http://www.immex.ucla.edu/HomeMenuItems.htm). (B) Rubric score 2: student accessed relevant data (circled) and strayed before solving the problem. (C) Rubric score 3: students accessed irrelevant data and still solved the problem. (D) Rubric score 4: student accessed relevant data (circled) but did not solve problem. (E) Rubric score 5: student accessed irrelevant data and did not solve problem.

Source: J. Microbiol. Biol. Educ. May 2001 vol. 2 no. 1 25-33. doi:10.1128/154288101X14285805896158
Download as Powerpoint
Image of FIG. 2

Click to view

FIG. 2

Major nodes. (A) Student performances clustered by the ANN were assigned to specific nodes. Nodes 3, 6, 7, 15, 17, and 19 were composed of at least 75% allergy cases. Nodes 18 and 24 contained a significant number of cases relating to viral diseases. Node 8, the largest node (71 performances), contained many unsolved, incomplete, and guessed (direct from start to solution) performances. (B) Nodes with 10 or more performances showing the ratio of allergy to nonallergy cases attempted at each node. Nodes with 75% or more allergy cases are indicated with an arrow.

Source: J. Microbiol. Biol. Educ. May 2001 vol. 2 no. 1 25-33. doi:10.1128/154288101X14285805896158
Download as Powerpoint
Image of FIG. 3

Click to view

FIG. 3

Student performances at node 3, search-path map vs. menu item usage. (A) A group search-path map showing 26 student performances clustered at node 3. The thickness of the lines indicates the relative number of students who selected the same sequence of menu item usage. The two most accessed menu items at this node (circled) were the SPT and ELISA for food. All 14 food allergy cases at this node were solved in contrast to only two of the remaining 12 cases. (B) More than 95% of the students selected the menu item with the SPT results for food, and more than 50% selected the menu item with the ELISA results for food. All students automatically selected item 1, the case scenario from where the problem begins.

Source: J. Microbiol. Biol. Educ. May 2001 vol. 2 no. 1 25-33. doi:10.1128/154288101X14285805896158
Download as Powerpoint
Image of FIG. 4

Click to view

FIG. 4

Menu item usage for student performances at Nodes 19, 15, and 6. (A) Menu item selection versus percentage of student performances at node 19. More than 90% of the students selected the menu item for pollen SPT results and about 65%, the menu item for pollen ELISA results. (B) Menu item selection versus percentage of student performances at node 15. Four menu items were selected by more than 60% of the students and all of them were ELISA results. More than 65% of the students selected menu items for ELISA results for both a chemical (latex) and pollen, while close to 95% selected the menu item for ELISA dust mite results and 100% for ELISA food results. (C) Menu item selection versus percentage of student performances at node 6. All students conferred with the allergist while 60% looked up “allergen” and “skin prick test” in the library. Close to 90% of the students selected menu items with SPT results for dust mite and food and with ELISAs for dust mite and food, while 80% accessed ELISA results for the chemical latex and for the pollen count.

Source: J. Microbiol. Biol. Educ. May 2001 vol. 2 no. 1 25-33. doi:10.1128/154288101X14285805896158
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error