关键词: 两阶段作弊甄别方法, 答案抄袭检测法, 个人拟合检测法, 选择题, 主观题

Abstract: Answer copying detection method is one of the most widely used statistical methods in detecting cheaters, however, its power is unsatisfactory for cheaters with high scores or low amount of cheating. To overcome these weaknesses, a two-stage cheating detection method was proposed in this research by making use of information from both multiple-choice items that are easy to copy and subjective items that are difficult to copy. In the first stage of detection, personal fit detection method was used to detect discrepancy of ability information between two types of items on every examinee. Then, examinees performing excessive discrepancy could be regarded as cheating suspects who were going to be detected further in the second stage by the answer copying detection method. In this way, we could find cheaters more accurately. Present research simulated 600 responses of both normal examinees and cheaters based on the item parameters estimated from the real data of the Beijing Art Combination Test of College Entrance Examination. To be precise, item parameters derived from 35 multiple choice items by Nominal Response Model and 13 subjective items by Partial Credit Model. 600 examinees were distributed in 20 examination rooms with 30 examinees in each one. The ability of copiers were at least inferior to 50% of the other examinees, and they copied the answers from the source in the same examination room, with 3 levels of the ability(exceeding 100%, 80% and 60% of the other examinees) and 3 levels of copy proportion(100%, 80% and 60%). The two-stage cheating detection method with l-index in the first stage and ω-index in the second stage was used for evaluating personal fit according to the degree of discrepancy that examinees performed between multiple choice items and subjective items, then detecting the agreement in the responses of multiple-choice items of examinees with aberrant response in the first stage with other examinees. Results showed that the proportion of copying had a significant impact on the efficiency of the two stage cheating detection method, the higher the proportion of copying, the higher the correct detection rate, the lower the false rate. Compared to the detection result of only ω-index, the two-stage cheating detection method had a much better performance in the detection rate and false rate. As for cheaters with high scores or low proportion of cheating number, compared to a detection rate of only 20.32% or 39.55% along with very high false rate in answer copy detection method only, this new method with two stage could achieve an impressive detection rate of 62.1% or 49.4% along with very low false rate near 0. Overall, by integrating the information of both multiple choice and subjective items, making full use of the difference in cheating difficulty in two type of items, combining the advantage of answer copy detection method and personal fit methods, the two-stage cheating detection method increases detection rate and decreases the false rate, especially for cheaters with high scores or low proportion of cheating number. The thought of excluding normal examinees in cheating detection widen the research approach and scope of application. To summarize, it is an important innovation to develop the detection method, no matter in the practice of real cheating detection examination or theory research.

Key words: Two-stage cheating detection, Answer copying detection method, Personal fit detection method, multiple choice items, subjective items