Evaluation Evolution: Designing Optimal Evaluations to Enhance Learning in Nephrology Fellowship

      Evaluations serve as the backbone of any educational program and can be broadly divided into formative and summative evaluations. Formative evaluations are “just in time” evaluations focused on informing the learning process, whereas summative evaluations compare fellows to a preset standard to determine their readiness for unsupervised practice. In the nephrology fellowship programs, evaluations assess competence in the framework of ACGME Milestones 2.0. A variety of learning venues, evaluators, and tools should be incorporated into the measurement process. It is important to determine which milestones can be best assessed in each education venue to decrease the burden of assessment fatigue. Additionally, programs can diversify the evaluators to include nurses, medical students, peers, and program coordinators in addition to faculty to provide a well-rounded assessment of the fellows and share the assessment burden. Lastly, the evaluation data should be presented to fellows in a format where it can inform goal setting. The evaluation system needs to evolve along with the changes being made in curriculum design. This will help to make fellowship learning effective and efficient.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Advances in Chronic Kidney Disease
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Abraham R.R.
        • Komattil R.
        Heutagogic approach to developing capable learners.
        Med Teach. 2017; 39: 295-299
        • Bansal A.
        • Jain S.
        • Sharma L.
        • Sharma N.
        • Jain C.
        • Madaan M.
        Students’ perception regarding pedagogy, andragogy, and heutagogy as teaching-learning methods in undergraduate medical education.
        J Educ Health Promot. 2020; 9: 301
        • Nasca T.J.
        • Philibert I.
        • Brigham T.
        • Flynn T.
        The next GME accreditation system--rationale and benefits.
        N Engl J Med. 2012; 366: 1051-1056
        • Wass V.
        • Van der Vleuten C.
        • Shatzer J.
        • Jones R.
        Assessment of clinical competence.
        Lancet. 2001; 357: 945-949
        • Tekian A.
        • Park Y.
        • Tilton S.
        • et al.
        Competencies and feedback on Internal medicine residents’ end-of-rotation assessments over time: Qualitative and Quantitative Analyses.
        Acad Med. 2019; 94: 1961-1969
        • Gomez-Garibello C.
        • Young M.
        Emotions and assessment: considerations for rater-based judgements of entrustment.
        Med Educ. 2018; 52: 254-262
        • Baker K.
        Feedback vs. evaluation: Geting past the reluctance to deliver negative feedback.
        Date accessed: December 12, 2021
        • Schuttpelz-Brauns K.
        • Karay Y.
        • Arias J.
        • Gehlhar K.
        • Zupanic M.
        Comparison of the evaluation of formative assessment at two medical faculties with different conditions of undergraduate training, assessment and feedback.
        GMS J Med Educ. 2020; 37: Doc41
        • De Haes J.C.
        • Oort F.J.
        • Hulsman R.L.
        Summative assessment of medical students' communication skills and professional attitudes through observation in clinical practice.
        Med Teach. 2005; 27: 583-589
        • Sharma S.
        • Sharma V.
        • Sharma M.
        • Awasthi B.
        • Chaudhary S.
        Formative assessment in postgraduate medical education - Perceptions of students and teachers.
        Int J Appl Basic Med Res. 2015; 5: S66-S70
        • Barrett M.
        • Georgoff P.
        • Matusko N.
        • et al.
        The Effects of feedback fatigue and Sex Disparities in medical student feedback assessed using a Minute feedback system.
        J Surg Educ. 2018; 75: 1245-1249
        • Vashe A.
        • Devi V.
        • Rao R.
        • Abraham R.
        Curriculum mapping of dental physiology curriculum: the path towards outcome-based education.
        Eur J Dent Educ. 2020; 24: 518-525
        • Stone S.L.
        • Qualters D.M.
        Course-based assessment: implementing outcome assessment in medical education.
        Acad Med. 1998; 73: 397-401
      1. Education, T.A.C.f.G.M. Nephrology milestones.
        • McLean S.
        • Edgar L.
        • Harsy B.
        The milestones 2.0 development process.
        J Grad Med Educ. 2021; 13: 4-7
      2. Education, T.A.C.f.G.M. Supplemental Guide: nephrology.
        • Holt K.D.
        • Miller R.S.
        • Nasca T.J.
        Residency programs' evaluations of the competencies: data provided to the ACGME about types of assessments used by programs.
        J Grad Med Educ. 2010; 2: 649-655
        • Oller K.L.
        • Mai C.
        • Ledford R.
        • O’Brien K.
        Faculty development for the evaluation system: a dual agenda.
        Adv Med Educ Pract. 2017; 8: 205-210
        • Park Y.S.
        • Riddle J.
        • Tekian A.
        Validity evidence of resident competency ratings and the identification of problem residents.
        Med Educ. 2014; 48: 614-622
        • Blankush J.M.
        • Shah B.
        • Barnett S.
        • et al.
        What are the associations between the quantity of faculty evaluations and residents’ perception of quality feedback?.
        Ann Med Surg (Lond). 2017; 16: 40-43
        • Norcini J.
        • Burch V.
        Workplace-based assessment as an educational tool: AMEE Guide No. 31.
        Med Teach. 2007; 29: 855-871
        • Anderson T.N.
        • Lau J.N.
        • Shi R.
        • et al.
        The utility of peers and trained Raters in Technical skill-based assessments a Generalizability theory study.
        J Surg Educ. 2022; 79: 206-215
        • Nathoo N.A.
        • Sidhu R.
        • Gingerich A.
        Educational Impact Drives Feasibility of implementing daily assessment in the workplace.
        Teach Learn Med. 2020; 32: 389-398
        • Kovach R.A.
        • Griffen D.L.
        • Francis M.L.
        Evaluating emergency medicine faculty at end-of-shift.
        West J Emerg Med. 2010; 11: 486-490
        • Phillips A.W.
        • Reddy S.
        • Durning S.J.
        Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102.
        Med Teach. 2016; 38: 217-228
        • Gerbase M.W.
        • Germond M.
        • Cerutti B.
        • Vu N.V.
        • Baroffio A.
        How many responses do We need? Using Generalizability Analysis to Estimate Minimum Necessary response rates for online student evaluations.
        Teach Learn Med. 2015; 27: 395-403
        • Chiu Y.L.
        • Tsai Y.J.
        • Lin C.H.
        • Hou Y.R.
        • Sung W.H.
        Evaluation of a smartphone-based assessment system in subjects with chronic ankle instability.
        Comput Methods Programs Biomed. 2017; 139: 191-195
        • Raymond B.L.
        • Fowler L.C.
        • Robertson A.C.
        The Impact of Converting from an 'Educator-Driven' to a 'Learner-Initiated' feedback model.
        J Educ Perioper Med. 2019; 21: E627
        • Phillips A.W.
        • Friedman B.
        • Utrankar A.
        • Ta A.Q.
        • Reddy S.
        • Durning S.
        Surveys of health Professions trainees: Prevalence, response rates, and Predictive factors to Guide Researchers.
        Acad Med. 2017; 92: 222-228
        • Jepson C.
        • Asch D.A.
        • Hershey J.C.
        • Ubel P.A.
        In a mailed physician survey, questionnaire length had a threshold effect on response rate.
        J Clin Epidemiol. 2005; 58: 103-105
        • Brown J.
        • Dearnaley J.
        Learning and teaching clinical communication in the clinical workplace.
        Clin Teach. 2016; 13: 283-286
        • Malone M.G.
        • Carney M.M.
        • House J.B.
        • Cranford J.A.
        Tit-for-tat strategy for increasing medical student evaluation response rates.
        West J Emerg Med. 2018; 19: 75-79
        • Williams R.G.
        • Klamen D.A.
        • McGaghie W.C.
        Cognitive, social and environmental sources of bias in clinical performance ratings.
        Teach Learn Med. 2003; 15: 270-292
        • Iramaneerat C.
        • Yudkowsky R.
        Rater errors in a clinical skills assessment of medical students.
        Eval Health Prof. 2007; 30: 266-283
        • Dayal A.
        • O’Connor D.
        • Qadri U.
        • Arora V.
        Comparison of Male vs Female resident milestone evaluations by faculty during emergency medicine residency training.
        JAMA Intern Med. 2017; 177: 651-657
        • Tatem G.B.
        • Gardner-Gray J.
        • Standifer B.
        • Alexander K.
        While You Don’t See color, I See bias: Identifying Barriers in access to Graduate medical education training.
        ATS Sch. 2021; 2: 544-555
        • Sabin J.
        • Guenther G.
        • Ornelas I.
        • et al.
        Brief online implicit bias education increases bias awareness among clinical teaching faculty.
        Med Educ Online. 2022; 27: 2025307
        • Salerno S.M.
        • Jackson J.L.
        • O'Malley P.G.
        Interactive faculty development seminars improve the quality of written feedback in ambulatory teaching.
        J Gen Intern Med. 2003; 18: 831-834
        • Stehman C.R.
        • Hochman S.
        • Fernandez-Frackelton M.
        • et al.
        Professionalism milestones assessments used by emergency medicine residency programs: a Cross-sectional survey.
        West J Emerg Med. 2019; 21: 152-159
        • Chandler N.
        • Henderson G.
        • Park B.
        • Byerley J.
        • Brown W.
        • Steiner M.
        Use of a 360-degree evaluation in the outpatient setting: the usefulness of nurse, faculty, patient/family, and resident self-evaluation.
        J Grad Med Educ. 2010; 2: 430-434
        • Wood L.
        • Hassell A.
        • Whitehouse A.
        • Bullock A.
        • Wall D.
        A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design.
        Med Teach. 2006; 28: e185-e191
        • Tariq M.
        • Govaerts M.
        • Afzal A.
        • Ali S.A.
        • Zehra T.
        Ratings of performance in multisource feedback: comparing performance theories of residents and nurses.
        BMC Med Educ. 2020; 20: 355
        • Miloslavsky E.M.
        • Boyer D.
        • Winn A.S.
        • Stafford D.E.J.
        • McSparron J.I.
        Fellows as teachers: Raising the educational bar.
        Ann Am Thorac Soc. 2016; 13: 465-468
        • Abudu B.
        Where is the patient Voice in clinical Clerkship evaluations?.
        Acad Med. 2019; 94: 610-611
        • Van der Heijden B.I.J.M.
        • Nijhof A.H.J.
        The value of subjectivity: Problems and prospects for 360-degree appraisal systems.
        Int J Hum Resource Management. 2004; 15: 493-511
        • Jug R.
        • Jiang X.S.
        • Bean S.M.
        Giving and Receiving effective feedback: a review Article and how-to Guide.
        Arch Pathol Lab Med. 2019; 143: 244-250
        • Telio S.
        • Ajjawi R.
        • Regehr G.
        The "educational alliance" as a framework for reconceptualizing feedback in medical education.
        Acad Med. 2015; 90: 609-614