The Angoff method is a method used to set passing scores for standardized tests. It is a standard-setting method that involves a panel of subject matter experts who are asked to estimate the difficulty of each test item and the minimum level of performance required to correctly answer that item. The experts then use their estimates to determine the overall passing score for the test. This method is often used for high-stakes tests, such as those used for licensure or certification. It is considered a more rigorous and defensible method than others, such as the “bright line” method, which simply sets a passing score based on a pre-determined cutoff point.
The Angoff method is considered a more subjective method of setting passing scores because it relies on the judgments of the subject matter experts. The process typically involves assembling a panel of experts who are familiar with the test content and the population that will be taking the test. The experts are then asked to review each test item and estimate the probability that a minimally qualified examinee would be able to answer the item correctly.
Once the experts have provided their estimates, the scores are compiled and the passing score is set based on the average of the experts’ judgments. This passing score represents the difficulty level of the test and is intended to ensure that only examinees who have the minimum level of knowledge and skill required for the profession or occupation being tested are able to pass.
The Angoff method is widely used and accepted in the field of educational testing and is considered a more robust and defensible method than other methods such as the “bright line” method, which simply sets a passing score based on a pre-determined cutoff point.
It is important to note that the Angoff method is not without criticism, some experts argue that the method is subject to bias, as the panel of experts may not be representative of the population taking the test and their estimates may be influenced by their own personal biases and experiences.
Here’s an example of how the Angoff method might be used to set a passing score for a licensure examination for teachers:
- A panel of subject matter experts, which might include experienced teachers, teacher educators, and curriculum specialists, is assembled.
- The experts are given a copy of the examination and asked to review each test item.
- For each item, the experts are asked to estimate the probability that a minimally qualified examinee would be able to answer the item correctly. This is typically done on a scale of 0-100, with 0 representing an item that is impossible for a minimally qualified examinee to answer correctly, and 100 representing an item that a minimally qualified examinee would be able to answer correctly with ease.
- Once the experts have provided their estimates, the scores are compiled and the passing score is set based on the average of the experts’ judgments.
- In this example, let’s say the average of the experts’ judgments for the test is 70. This means that the experts believe that a minimally qualified examinee would be able to answer 70% of the test items correctly.
- Therefore, the passing score for this examination is set at 70%. This means that any examinee who scores 70% or higher on the examination is considered to have the minimum level of knowledge and skill required to be a licensed teacher.
It is important to note that this is a simplified example, and in real-world situations, the process of setting a passing score using the Angoff method is more complex, and typically involves multiple rounds of review and consultation with experts.
Limitations
The Angoff method is a widely accepted and widely used method for setting passing scores for standardized tests, but it is not without limitations. Some of the limitations of the Angoff method include:
1. Subjectivity
The Angoff method is a subjective method of setting passing scores, as it relies on the judgments of a panel of subject matter experts. These experts may have personal biases and their judgments may be influenced by their own experiences and perceptions.
2. Panel composition
The composition of the panel of experts can also be a limitation of the Angoff method. The panel should be representative of the population taking the test but in practice, it may not be, leading to bias and lack of generalizability of the passing scores.
3. Lack of consistency
The Angoff method is based on the judgments of individuals, and these judgments may vary from one expert to another, or from one test administration to another. This can lead to inconsistencies in passing scores over time and between different tests.
4. Time-consuming
The Angoff method can be time-consuming and resource-intensive, as it requires assembling a panel of experts, providing training and support, and conducting multiple rounds of review and consultation.
5. Cost
The Angoff method can be costly, as it requires paying experts for their time and expertise.
6. Limited generalizability
The Angoff method is typically used for high-stakes tests, such as those used for licensure or certification, and may not be appropriate for other types of tests.
Despite these limitations, the Angoff method is still widely used and considered a more robust and defensible method than other methods such as the “bright line” method. It is important to note that any standard-setting method has limitations and the Angoff method should be used in conjunction with other methods and sources of information to establish passing scores.
