Title: Reliability and validity of the direct observation clinical encounter examination (DOCEE)
Abstract: Medical EducationVolume 37, Issue 3 p. 205-212 Reliability and validity of the direct observation clinical encounter examination (DOCEE) Hossam Hamdy, Hossam Hamdy College of Medicine and Medical Sciences, Arabian Gulf University, BahrainSearch for more papers by this authorKameshwar Prasad, Kameshwar Prasad College of Medicine and Medical Sciences, Arabian Gulf University, BahrainSearch for more papers by this authorReed Williams, Reed Williams Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois, USASearch for more papers by this authorFathi A Salih, Fathi A Salih Department of Educational Measurements and Statistics, College of Postgraduate Studies, Arabian Gulf University, BahrainSearch for more papers by this author Hossam Hamdy, Hossam Hamdy College of Medicine and Medical Sciences, Arabian Gulf University, BahrainSearch for more papers by this authorKameshwar Prasad, Kameshwar Prasad College of Medicine and Medical Sciences, Arabian Gulf University, BahrainSearch for more papers by this authorReed Williams, Reed Williams Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois, USASearch for more papers by this authorFathi A Salih, Fathi A Salih Department of Educational Measurements and Statistics, College of Postgraduate Studies, Arabian Gulf University, BahrainSearch for more papers by this author First published: 24 February 2003 https://doi.org/10.1046/j.1365-2923.2003.01438.xCitations: 33 Professor Hossam Hamdy, MBChB, MCh, FRCS, FACS, Dean, College of Medicine and Medical Sciences, Arabian Gulf University, PO Box 22979, Bahrain. Tel.: 00 973 239702; Fax: 00 973 230730; E-mail: [email protected] Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Context The College of Medicine and Medical Sciences at the Arabian Gulf University, Bahrain, replaced the traditional long case/short case clinical examination on the final MD examination with a direct observation clinical encounter examination (DOCEE). Each student encountered four real patients. Two pairs of examiners from different disciplines observed the students taking history and conducting physical examinations and jointly assessed their clinical competence. Objectives To determine the reliability and validity of the DOCEE by investigating whether examiners agree when scoring, ranking and classifying students; to determine the number of cases and examiners necessary to produce a reliable examination, and to establish whether the examination has content and concurrent validity. Subjects Fifty-six final year medical students and 22 examiners (in pairs) participated in the DOCEE in 2001. Methods Generalisability theory, intraclass correlation, Pearson correlation and kappa were used to study reliability and agreement between the examiners. Case content and Pearson correlation between DOCEE and other examination components were used to study validity. Results Cronbach's alpha for DOCEE was 0·85. The intraclass and Pearson correlation of scores given by specialists and non-specialists ranged from 0·82 to 0·93. Kappa scores ranged from 0·56 to 1·00. The overall intraclass correlation of students' scores was 0·86. The generalisability coefficient with four cases and two raters was 0·84. Decision studies showed that increasing the cases from one to four improved reliability to above 0·8. However, increasing the number of raters had little impact on reliability. The use of a pre-examination blueprint for selecting the cases improved the content validity. The disattenuated Pearson correlations between DOCEE and other performance measures as a measure of concurrent validity ranged from 0·67 to 0·79. Conclusions The DOCEE was shown to have good reliability and interrater agreement between two independent specialist and non-specialist examiners on the scoring, ranking and pass/fail classification of student performance. It has adequate content and concurrent validity and provides unique information about students' clinical competence. Citing Literature Volume37, Issue3March 2003Pages 205-212 RelatedInformation
Publication Year: 2003
Publication Date: 2003-03-01
Language: en
Type: article
Indexed In: ['crossref', 'pubmed']
Access and Citation
Cited By Count: 51
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot