Created by : Regina McEntee, Cherie Gibson & Megan Puls
1. Norm-referenced testing refers to standardize tests that are designed to rank and compare test takers in relation to one another (Billings & Halstead, 2015). A norm-referenced test utilizes the 'bell curve' and will provide information as to whether a student did better or worse than a hypothetical, average student (norm group). For example, if a student received a percentile rank score on the total test of 39, this means that the student performed as well as or better than 39% of the students in the norm group. This could be useful in determining if a student needs remediation or is a candidate for acceptance into a gifted program. Though some formats incorporate open-ended short answer questions, norm-referenced tests are often multiple choice and are based on national standards (Norm-Referenced Test, 2015). An example of norm-referenced testing would SAT or ACT scores that are necessary for determining a high school graduates admission into college.
Criterion-referenced testing refers to testing that determines whether or not a student has achieved a specific objective or skill (Billings & Halstead, 2015). Unline norm-referenced testing, criterion-referenced testing is designed to measure a student's performance against a fixed set of predetermined criteria or learning standards (Criterion-Refernced Test, 2014). That is, a student will receive an independent grade or score that is not compared to grades of other students in the class. Criterion-referenced testing is commonly used in nursing education due to the fact that the goal of nursing education is for students to attain a mastery of the content. Billings & Halstead (2015) equate the use of criterion-referenced testing measure a student;s ability to ensure safety in practice. Regardless of how other students score, a student would need to achieve a set score, such as 95% or 100%, in order to show competence in drug dosage calculations (Billings & Halstead, 2015).
2. Two links to test writing strategies
Strategies for Writing Effective Questions for Objective Exams
The link above was helpful in test writing resources because it broke down specific common mistakes in test writing. It helped understand the best way to write multiple choice questions to promote success, but to avoid the students needing to guess. Guessing could be due to too many distractors within the question, or the implied possibility of more than one correct answer. Test questions should measure outcomes, only ask one specific question, and focus on content rather than logic. The author also shows a test writer how to bring attention to a part of the question appropriately as well as instructs how not to give away answers.
Criterion-referenced testing refers to testing that determines whether or not a student has achieved a specific objective or skill (Billings & Halstead, 2015). Unline norm-referenced testing, criterion-referenced testing is designed to measure a student's performance against a fixed set of predetermined criteria or learning standards (Criterion-Refernced Test, 2014). That is, a student will receive an independent grade or score that is not compared to grades of other students in the class. Criterion-referenced testing is commonly used in nursing education due to the fact that the goal of nursing education is for students to attain a mastery of the content. Billings & Halstead (2015) equate the use of criterion-referenced testing measure a student;s ability to ensure safety in practice. Regardless of how other students score, a student would need to achieve a set score, such as 95% or 100%, in order to show competence in drug dosage calculations (Billings & Halstead, 2015).
2. Two links to test writing strategies
Strategies for Writing Effective Questions for Objective Exams
The link above was helpful in test writing resources because it broke down specific common mistakes in test writing. It helped understand the best way to write multiple choice questions to promote success, but to avoid the students needing to guess. Guessing could be due to too many distractors within the question, or the implied possibility of more than one correct answer. Test questions should measure outcomes, only ask one specific question, and focus on content rather than logic. The author also shows a test writer how to bring attention to a part of the question appropriately as well as instructs how not to give away answers.
Designing Test Questions
This link looks at a variety of test writing strategies, what each style is good for, along with the advantages and disadvantages of each style. It allows the educator to determine which type of test question is beneficial in the particular situation, according to the objective that is being taught. The different type of test questions discussed are true/false, multiple choice, matching, short answers, essay, oral, portfolios, and performance. There is also a link at the bottom of the page that helps the educator in designing test questions. When you are testing a student, it is beneficial to understand which type of testing would be most effective and why that works best for certain situations.
3. NCLEX QUESTIONS
NCLEX questions are written with the idea that the practice of nursing requires the application of knowledge, skills, and abilities, and thus is written with the use of Bloom's taxonomy. The majority of the test items are written with higher levels of cognitive ability as it requires the test-taker to utilize a more complex thought process. The structure of the NCLEX is based on client needs as this is a universal structure for defining nursing actions and competencies and focuses on different clinical settings (OBGYN, Pediatrics, Orthopedics, etc) within the nursing profession. The four major client needs categories and sub-categories for test items are (National Council of State Boards of Nursing, 2013):
Safe and Effective Care Environment
Psychosocial Integrity
Physiological Integrity
NCLEX test questions formats include multiple choice, multiple response, chart/exhibit format, fill-in-the-blank calculations, ordered response and/or hot spots. The test items might also include multimedia such as charts, graphics sound or video (National Council of State Boards of Nursing, 2015). Scoring on the NCLEX is based on three different scenarios and partial credit is not considered. The 95% Confidence Interval Rule will allow the computer to stop administering test items when it is 95% certain that the test-takers ability is at or above the standard. The maximum-length exam disregards the 95% CI rule and considers the final ability estimate which estimates the student's ability to perform above the standard (pass) or below the standard (fail). The Run-Out-Of-Time Rule (ROOT) takes place when the student fails to answer the minimum number of required items in the designated time frame - automatic fail. When the minimum number of required items were answered, but time runs out, if the last 60 ability estimates were above passing standards, the student passes. If even one item drops below the passing standard throughout the last 60 times, the student fails (National Council of State Boards of Nursing, 2013).
5. Reference
Yeom, Y-J. (2013) An investigation of predictors of NCLEX-RN outcomes among nursing content standardized tests. Nursing Education Today. 33. 1523-1528. DOI: 10.1016/j.nedt.2013.04.004
Summary
This author set out to determine if standardized tests during nursing school could predict passing the NCLEX-RN on the first attempt. The standardized tests included Adult Medical-Surgical, Nursing Fundamentals, Pharmacology, Mother-Baby, Psychology, Community Health, and Leadership and Management. Of the 151 nursing students used in the study, 118 passed the NCLEX-RN on the first attempt and 33 failed on the first attempt. The author found significance between the two groups related to individual standardized test scores and that testing for adult med-surg, pharmacology and community health had the largest impact on NCLEX results.
To meet the increasing shortage of registered nurses, it is more important than ever to have students pass the NCLEX-RN on the first attempt. This affects not only the student, but the stakeholders, or hiring institutions. The student could feel such things as embarrassment, anxiety, a lower self-esteem and social stigma with not being successful on the first attempt. This can also negatively affect their further attempts at passing. Due to the need for passing students, a larger focus is placed on academic institutions to use standardized testing strategies in the determined topics of adult med-surg, pharmacology and community health, as well as the student’s overall GPA. The author mentions multiple different testing groups that can provide these standardized tests. These include Mosby Access Test, National League for Nursing Adult Health Comprehensive Test, National League for Nursing Maternal-Newborn Comprehensive Test, and the National League for Nursing Pediatric Nursing Comprehensive Test. Success was found in comparing individual test results to NCLEX-RN pass rate, but not in comparing student’s individual testing results to another student’s testing results. When students had an increase of mean scores, it was found that there was more likely to be a passing grade on the NCLEX-RN. The author ended the article with mentioning such limitations as the significant size difference in the passing and failing group and the fact that all information was gained from the same bachelor’s degree program at a University in the Midwest.
Reference
Carr, S. M. (2011). NCLEX-RN pass rate peril: one school's journey through curriculum revision, standardized testing, and attitudinal change. Nursing Education Perspectives, 32(6), 384-388.
Summary
In this article, the main subject was testing, and was it adequate for determining whether the nursing students were able to comprehend what was being taught, and pass the NCLEX examination. The pass rates for the NCLEX were poor and continued to decline, jeopardizing the program and the ability to maintain their accreditation. In order to keep the accreditation and program it was necessary to revamp their program to be more successful. There initial plan, to bring in a faculty member who had their doctorate to teach the students, was not successful and the NCLEX rates continued to decrease. At this point the success for first time pass rate was down to 63%. It was at this time the the faculty went through a transition, which included education, helping them understand how to write test questions and gain confidence in teaching test taking strategies. The faculty was able to gain much needed knowledge, and once the faculty understood they were able to pass the information along to the students. They looked at the test scores required nationally, from standard tests given to the students, and raised the bar and expectations for them. When the minimum standards were not met the student was required to remediate with staff. It took a lot of time and many revisits to the program, to come up with a plan that works. Over the course of two years they revamped the requirements, focusing on weak areas and fixing those, allowing for understanding by students, leading to a NCLEX pass rate of 92-93%. Once the students were able to see the effort and enthusiasm by the staff it encouraged them to be successful. The outlook the students had toward NCLEX exam had changed, and they wanted to be successful. Faculty engagement and encouragement as promoted optimal learning and success. This has had a huge impact on students, they now expect to be successful the first time they take the exam.
Criterion-Referenced Test. (2014, April 4). Retrieved on March 2, 2017 from The Glossary of Educatino Reform website http://edglossary.org/criterion-referenced-test/
National Council of State Boards of Nursing. (2013). NCLEX-RN Examination. Retrieved on March 2, 2017, from https://www.ncsbn.org/2013_NCLEX_RN_Test_Plan.pdf
National Council of State Boards of Nursing. (2015). Alternate Item Formats FAQs. Retrieved on March 2, 2017, from, https://www.ncsbn.org/2334.htm
Norm-Rererenced Test. (2015, July 22). Retrieved on March 2, 2017, from The Glossary of Education Reform website http://edglossary.org/norm-referenced-test/
Stanny, C.J. (n.d.) Strategies for Writing Effective Questions for Objective Exams. Center for University Teaching, Learning, and Assessment. Retrieved from: http://uwf.edu/media/university-of-west-florida/offices/cutla/documents/Strategies-for-Writing-Effective-Questions-for-Objective-Exams.pdf
NCLEX questions are written with the idea that the practice of nursing requires the application of knowledge, skills, and abilities, and thus is written with the use of Bloom's taxonomy. The majority of the test items are written with higher levels of cognitive ability as it requires the test-taker to utilize a more complex thought process. The structure of the NCLEX is based on client needs as this is a universal structure for defining nursing actions and competencies and focuses on different clinical settings (OBGYN, Pediatrics, Orthopedics, etc) within the nursing profession. The four major client needs categories and sub-categories for test items are (National Council of State Boards of Nursing, 2013):
Safe and Effective Care Environment
- Management of Care
- Safety & Infection Control
Psychosocial Integrity
Physiological Integrity
- Basic Care and Comfort
- Pharmacological and Parenteral Therapies
- Reduction of Risk Potential
- Physical Adaptation
NCLEX test questions formats include multiple choice, multiple response, chart/exhibit format, fill-in-the-blank calculations, ordered response and/or hot spots. The test items might also include multimedia such as charts, graphics sound or video (National Council of State Boards of Nursing, 2015). Scoring on the NCLEX is based on three different scenarios and partial credit is not considered. The 95% Confidence Interval Rule will allow the computer to stop administering test items when it is 95% certain that the test-takers ability is at or above the standard. The maximum-length exam disregards the 95% CI rule and considers the final ability estimate which estimates the student's ability to perform above the standard (pass) or below the standard (fail). The Run-Out-Of-Time Rule (ROOT) takes place when the student fails to answer the minimum number of required items in the designated time frame - automatic fail. When the minimum number of required items were answered, but time runs out, if the last 60 ability estimates were above passing standards, the student passes. If even one item drops below the passing standard throughout the last 60 times, the student fails (National Council of State Boards of Nursing, 2013).
5. Reference
Yeom, Y-J. (2013) An investigation of predictors of NCLEX-RN outcomes among nursing content standardized tests. Nursing Education Today. 33. 1523-1528. DOI: 10.1016/j.nedt.2013.04.004
Summary
This author set out to determine if standardized tests during nursing school could predict passing the NCLEX-RN on the first attempt. The standardized tests included Adult Medical-Surgical, Nursing Fundamentals, Pharmacology, Mother-Baby, Psychology, Community Health, and Leadership and Management. Of the 151 nursing students used in the study, 118 passed the NCLEX-RN on the first attempt and 33 failed on the first attempt. The author found significance between the two groups related to individual standardized test scores and that testing for adult med-surg, pharmacology and community health had the largest impact on NCLEX results.
To meet the increasing shortage of registered nurses, it is more important than ever to have students pass the NCLEX-RN on the first attempt. This affects not only the student, but the stakeholders, or hiring institutions. The student could feel such things as embarrassment, anxiety, a lower self-esteem and social stigma with not being successful on the first attempt. This can also negatively affect their further attempts at passing. Due to the need for passing students, a larger focus is placed on academic institutions to use standardized testing strategies in the determined topics of adult med-surg, pharmacology and community health, as well as the student’s overall GPA. The author mentions multiple different testing groups that can provide these standardized tests. These include Mosby Access Test, National League for Nursing Adult Health Comprehensive Test, National League for Nursing Maternal-Newborn Comprehensive Test, and the National League for Nursing Pediatric Nursing Comprehensive Test. Success was found in comparing individual test results to NCLEX-RN pass rate, but not in comparing student’s individual testing results to another student’s testing results. When students had an increase of mean scores, it was found that there was more likely to be a passing grade on the NCLEX-RN. The author ended the article with mentioning such limitations as the significant size difference in the passing and failing group and the fact that all information was gained from the same bachelor’s degree program at a University in the Midwest.
Reference
Carr, S. M. (2011). NCLEX-RN pass rate peril: one school's journey through curriculum revision, standardized testing, and attitudinal change. Nursing Education Perspectives, 32(6), 384-388.
Summary
In this article, the main subject was testing, and was it adequate for determining whether the nursing students were able to comprehend what was being taught, and pass the NCLEX examination. The pass rates for the NCLEX were poor and continued to decline, jeopardizing the program and the ability to maintain their accreditation. In order to keep the accreditation and program it was necessary to revamp their program to be more successful. There initial plan, to bring in a faculty member who had their doctorate to teach the students, was not successful and the NCLEX rates continued to decrease. At this point the success for first time pass rate was down to 63%. It was at this time the the faculty went through a transition, which included education, helping them understand how to write test questions and gain confidence in teaching test taking strategies. The faculty was able to gain much needed knowledge, and once the faculty understood they were able to pass the information along to the students. They looked at the test scores required nationally, from standard tests given to the students, and raised the bar and expectations for them. When the minimum standards were not met the student was required to remediate with staff. It took a lot of time and many revisits to the program, to come up with a plan that works. Over the course of two years they revamped the requirements, focusing on weak areas and fixing those, allowing for understanding by students, leading to a NCLEX pass rate of 92-93%. Once the students were able to see the effort and enthusiasm by the staff it encouraged them to be successful. The outlook the students had toward NCLEX exam had changed, and they wanted to be successful. Faculty engagement and encouragement as promoted optimal learning and success. This has had a huge impact on students, they now expect to be successful the first time they take the exam.
Criterion-Referenced Test. (2014, April 4). Retrieved on March 2, 2017 from The Glossary of Educatino Reform website http://edglossary.org/criterion-referenced-test/
National Council of State Boards of Nursing. (2013). NCLEX-RN Examination. Retrieved on March 2, 2017, from https://www.ncsbn.org/2013_NCLEX_RN_Test_Plan.pdf
National Council of State Boards of Nursing. (2015). Alternate Item Formats FAQs. Retrieved on March 2, 2017, from, https://www.ncsbn.org/2334.htm
Norm-Rererenced Test. (2015, July 22). Retrieved on March 2, 2017, from The Glossary of Education Reform website http://edglossary.org/norm-referenced-test/
Stanny, C.J. (n.d.) Strategies for Writing Effective Questions for Objective Exams. Center for University Teaching, Learning, and Assessment. Retrieved from: http://uwf.edu/media/university-of-west-florida/offices/cutla/documents/Strategies-for-Writing-Effective-Questions-for-Objective-Exams.pdf