Evaluator: You have been selected as a Program Evaluator because you have subject matter expertise in the discipline you will be evaluating. Every Program Evaluator on your team brings an expertise in a program area and is assigned a program to evaluate based on this expertise. Accepting a position on a Visit Team is a commitment to fulfill all the obligations required. The role of the PEV is not to be the police, to look for "gotchas," but rather to be a partner with the institution in the improvement of the program.

Team Member: As a PEV, you also play the important role of team member. Your interaction with your colleagues, particularly during team meetings, is crucial to an efficient and successful visit. Being team-oriented is an important competency for all PEVs. 

• Complete the pre-visit work, including inspecting the materials provided by the institution.
• Participate in a 2- to 4-day on-site evaluation.
• Evaluate information obtained according to ABET Criteria.
• Prepare materials in a timely manner, including recommended accreditation action.
• Send all required copies of visit documents to your ABET Member Society as required.
• Complete performance appraisal forms. (This will be covered in Module 6.)
• Submit expense reports to ABET within 10 days of your campus visit.

There are a number of activities and deliverables you will need to complete prior to arriving on campus:

• Review the Self-Study Report. This is the primary document the institution prepares to demonstrate compliance with ABET Criteria. Each program uses the Self-Study Questionnaire template appropriate to the commission. These are found on the ABET website. While the majority of the program's Self-Study Report will be in paragraph format, a program may use tables, graphs, charts, and figures to illustrate compliance. The Self-Study Questionnaire includes tables that a program is required to complete along with prescribed formats for certain types of information like course descriptions and faculty curriculum vita. Here are samples of what those may look like:
   
  • Course Description Sample 1
  • Course Description Sample 2
  • Course Description Sample 3
  • Faculty CV Sample 1
  • Faculty CV Sample 2

  You must thoroughly review the Self-Study Report prior to arriving on campus.

• Transcript Analysis. As part of your review of the Self-Study Report, you will need to analyze transcripts. ABET recognizes transcripts as the official record of student coursework. While summary sheets or transcript checklists provided by the institution may be helpful for transcript analysis, the transcript itself should be the key document analyzed. Procedures for doing this include:
  • Be sure that the transcripts identify the name (title) of the degree received in a way that clearly identifies the program as an ABET-accredited program according to the institution catalog and other documents and in a way that distinguishes it from any non-accredited programs with which it could be confused by a potential employer. Identify any problems in this regard to your Team Chair.
  • Make sure that the courses that are counted toward the degree are consistent with the published requirements of the program. In cases where the transcript is for a graduate of an earlier curriculum, the institution must provide a copy of the appropriate curriculum. The institution also should provide justification for any variances, such as transfer credits or substitutions that are not clearly documented on the transcripts.
  • Check to be sure that prerequisites are taken before each course that requires them and that the course sequence on the transcript does not vary unreasonably from the recommended sequence. If courses are taken out of sequence, check to see if there is an indication of difficulty for the students in terms of the course grades. (If there are difficulties for students, then there could be a problem with the mechanisms for advising and the enforcement of prerequisites. If there are no problems, it could indicate prerequisite requirements that are not needed.)
  • Ensure that the number of transfer credits and the number of course substitutions are reasonable.
  • Request clarification for any apparent problems in the transcripts. Do your transcript analysis and request clarifications soon enough to allow reasonable time for the institution to respond.
    
• Draft Plan for Visit. Your Draft Plan includes the questions and areas that require additional information based on your review of the Self-Study Report. The Draft Plan helps you manage your time efficiently when you are on campus. Be sure to
  • Convey your preferences and arrangements for interviews and meetings with faculty, students, and others to your Team Chair prior to contacting program head.
  • Tell your Team Chair about visits needed to supporting areas. (Team Chairs usually coordinate visits to supporting areas.)

• Pre-Visit Forms
  You will be provided with a variety of forms to help you walk through the Self-Study Report and conduct a preliminary evaluation of a program's compliance with the ABET Criteria. Forms to help with the review of the Self-Study Report include the following:
  • The PEV Visit Report, including Transcript Analysis, Curriculum Analysis, and Faculty Analysis
  • Program Evaluator Worksheet

    Applied Science Evaluation Forms

    • Program Evaluator Visit Report (contains Program Evaluator Worksheet)

    Computing Evaluation Forms

    • Program Evaluator Visit Report
    • Program Evaluator Worksheet

    Engineering Evaluation Forms

    • Program Evaluator Visit Report
    • Program Evaluator Worksheet

    Technology Evaluation Forms

    • Program Evaluator Visit Report (contains Program Evaluator Worksheet)

  You will be required to complete these forms prior to your arrival on campus.
  

• Applying the Criteria
  • Review the current Criteria, the Self-Study Report, and any other materials sent by the institution.
  • Assess areas of apparent strengths and shortcomings.
  • Make preliminary assessment of what ABET Criteria appear to be met or may not be met.
• Follow up with the Team Chair regarding additional information you will need when you arrive on campus. Your Team Chair may direct you to communicate this information to the program representative.
• Always respond promptly to requests from your Team Chair.

• Make a qualitative assessment of factors that cannot be documented in the Self-Study Report.
• Conduct a detailed examination of the materials compiled by the institution (this includes course materials and assessment materials).
• Provide the program with a preliminary assessment of its strong points and shortcomings.
• Review facilities.

• Examine assessment materials (usually located with the course display materials). The purpose is to corroborate the assessment plan for Program Educational Objectives and Outcomes that was provided in the Self-Study Report, including ensuring that appropriate documentation exists for the various phases of the assessment process.
  
• Examine course materials. The course materials display should include for each course in the program a copy of the syllabus, including learning outcomes for the course; a copy of the textbook(s) for the course; a copy of each assignment (or lab), project, test, and examination; and copies of graded student work for each test, examination, and, where feasible, assignments and projects as well. Graded student work should include examples of excellent, average, and poor work. (APPM section II.E.3.c.(10))

  The Team Chair may coordinate the course materials display or may delegate this coordination to the PEV. It is a good idea to ensure that the materials for your program will be available as far in advance of your visit as possible. The specification of exactly which courses are to be included, such as whether materials for supporting courses in areas like math and science must be included, can vary among Commissions and Team Chairs. The course materials display should also include assessment documentation. The display materials should be readily available throughout the visit.

  In examining the course materials, you should verify that:

  • The course is up-to-date and appropriate for the objectives of the program.
  • The course prerequisites are both adequate and important.
  • The learning activities are appropriate for the course outcomes.

  • The graded work adequately assesses the course learning outcomes. This includes such things as appropriate test construction (no excessive reliance on objective-type questions, etc.), and appropriate grading standards.

  • The grade distribution for the course is in line with those of other courses and indicates no inadequacies in prerequisite courses.

  Note that it is unlikely that a problem with a single course would be enough evidence for a weakness or a deficiency. Rather, the course materials should be used to corroborate potential problems indicated elsewhere, such as suspected failure by some faculty members to remain current, or should suggest potential problems that need to be corroborated by additional evidence, such as an apparent lack of rigor sufficient to achieve the course learning outcomes. As in other evaluation activities, the emphasis is on the achievement of Program Educational Objectives and Outcomes, not on the quality of individual courses or any other individual component of the educational process.
  

• Interview the program head. This is an opportunity to get a better feel for the program's strengths and weaknesses and the vision of its leadership for the future. Any issues that arose during your pre-visit preparation should be discussed to ensure that you are aware of all appropriate facts and have planned appropriate activities for further investigation. (NOTE: If a single program is being evaluated, the Team Chair may decide to interview the program head.)

  It is preferable not to make requests that require substantial effort to satisfy during this meeting (or at anytime during the visit). Such requests should be made well in advance if your visit preparation is adequate. However, do not hesitate to make requests for things that are really needed to ensure a fair and complete evaluation. It is advisable to discuss such requests with your Team Chair if they are likely to require a substantial amount of work from the program's staff.

  You should also interview others with administrative duties (class scheduling, student advising, transfer course evaluation, etc.) relative to the program. This does not include administrators at the college level, whose visits are coordinated by the Team Chair. If the program head is not the department chair or other administrative head over the program, then you also should interview the administrative head. If more than one program under the administrative head is being reviewed, you may wish to coordinate your interview with the PEV(s) for the other program(s), perhaps even doing the interview simultaneously with the other PEV(s).
  

• Interview faculty members. During the faculty interviews, you should try to get a feel for the philosophy of each faculty member towards teaching; activities undertaken to maintain currency; level of understanding of, and participation in, the assessment process and preparation for the visit; and the view of the strengths and weaknesses of the program and its future direction. Faculty members also can be a source of initial identification or corroboration for problems with such aspects as the quality and maintenance of facilities (labs, classrooms, library, etc.). Faculty attitudes regarding the work environment and their commitment to remaining at the institution are also important in assessing the adequacy of institutional support to maintain program stability during the potential period of accreditation.

  There are many personal preferences regarding the best way to interview faculty members. Ideally, it is best to interview faculty members in their offices (if private) on a one-on-one basis. However, if the faculty is large or offices are not private, an alternative approach may be needed. Individual appointments should last for at least 30 minutes. You should identify some of the faculty you are especially interested in interviewing if you will not have time to interview all of them. This all should be arranged before the visit and in consultation with your Team Chair.
  

• Interview students. Interviewing the students is often one of the most rewarding and enjoyable parts of the visit. The level of satisfaction with, and enthusiasm for, the program can be a good indicator of the program quality. Also, students can provide important corroboration for such aspects as the lack of adequate quality in certain courses, problems with advising, and the adequacy of facilities (labs, classrooms, library, etc.). As we know, however, the student perspective on the importance of certain courses and the characteristics of the program is often quite different from the perspectives of the faculty and administration. Although these perspectives can be valuable in getting a feel about the overall quality of a program, it is important to keep in mind that students are not always in a good position to judge the real value of an educational experience. Therefore, student complaints should not result in a conclusion that there is a problem unless the problem can be corroborated by more concrete evidence.

  Again, there are many individual preferences for selecting the students to be interviewed as a group. Ideally, the group will include a representative number of upper-level students and some other students as well. If necessary, the student interviews can be done during a scheduled class (The instructor should not be present.), but it is difficult to get a good representative sample this way. On the other hand, if the student interview is held during an open meeting with voluntary attendance, it can be difficult to get good representative attendance as well (or even good attendance at all). In any case, the group to be interviewed should not be selected by the program staff unless you ensure that those selected are a representative sample of the student body, especially the upper-level students. You should allow at least 30 minutes for meeting with the students.
  

• Tour the facilities. In touring the facilities, you want to ensure that they are sufficiently up-to-date and maintained to support the needs of the program. You also want to be aware of safety and access issues. The adequacy of the support staff is important, as is a mechanism for ensuring the continuing adequacy through the period of accreditation.

  Additionally, you may be asked to do one or more of the following during the visit:
  

  • Visit the library. When you visit the library, you are primarily verifying the evidence in the Self-Study Report: adequacy of the collection, availability of staff support, electronic access to information and materials, etc. The adequacy of the resources and the quality of the space are important to observe, and this is difficult to determine from the Self-Study Report.
    
  • Visit one or more supporting departments. Supporting departments (mathematics, physics, English, etc.) play an important role in the foundational preparation of students. In interviewing the department chair or other departmental representative, you should try to get a feel for how well the students of the programs that the team is reviewing do in the supporting courses and whether there are any problems with the nature of the courses from the perspective of the supporting department. If course materials have been provided in a course materials display area, you should review these materials before going to visit the supporting department. If the materials have not been arranged in advance, then you cannot expect to see much more than course textbooks and syllabi for courses that are currently being offered.
    
  • Visit the career support center, placement office, cooperative education office, etc.
    The intent of visits to offices such as these is to assess the level of support for students. You should inquire about the support that is available and general procedures for students to obtain the support. Sometimes you can assist the program, students, or faculty by identifying available support that is not being used effectively in the program. Understaffed offices, inadequate facilities, or outdated programs can indicate or corroborate problems that appear in assessment results, student or faculty interviews, etc.

• Initial Team Meeting
  • Review Visit Plan.
  • Provide initial Program Evaluator Visit Report and Program Evaluator Worksheet to the Team Chair.
  • Discuss pre-visit assessment.
  • Review Criteria, Policy, and Procedure changes.
• Visit program to evaluate materials.
  • Course materials (syllabi, texts, graded student work, etc.)
  • Assessment materials (results and analysis from surveys, etc.)
• Conduct other visits (labs, etc.), as needed.

Sunday Evening

• Team meeting
  • Identify all potential deficiencies, weaknesses, and concerns.
  • Work as a team to come to consensus to ensure consistency on findings program to program.
  • Stay focused!

Monday Morning

• Team meets with president and/or dean and designated guests.
• Meet with program head.
• Meet with program teams, faculty, students, and support staff.
• Attend optional luncheon for team with institutional officials and guests.

Monday Afternoon

• Continue meetings and interviews.
• Visit facilities
• Visit supporting departments.

Monday Evening

• Attend team meeting.
  • Report on findings relative to previously identified potential deficiencies, weaknesses, and concerns.
  • Discuss possible accreditation action.
• Prepare draft of Exit Statement.

Tuesday Morning

• Provide draft of program Exit Statement to Team Chair.
• Complete interviews and facilities visits not completed on Monday.
• Brief program head on findings (informal debrief).
• Team meeting (working lunch)

Tuesday Afternoon

• Provide Team Chair with PEV Visit Report and Program Evaluator Worksheet.
  • Update to reflect visit findings.
  • List of persons interviewed
  • Recommended accreditation action
  • Program Exit Statement
• Participate in exit meeting.
• Provide Team Chair with completed Program Evaluator Visit Report.
• Exit Statement
  • Address each Criteria for which there are findings.
  • Document facts, compliance, deficiencies, weaknesses, concerns, and suggestions for improvement.
  • Your statement must correlate with PEV Visit Report and Program Evaluator Worksheet.
• Exit meeting
  • Read your written statement, and do not ramble!

• Identify issues by Criterion.
• Determine appropriate finding.
• Select the key term that applies overall for each Criterion.
• Explain each concern, weakness, and deficiency.
• Discuss/recommend an accreditation action to the team only.

1. Introduction
2. For the institution
   1. Strengths
   2. Deficiencies
   3. Weaknesses
   4. Concerns
3. For each program
   1. Strengths
   2. Deficiencies
   3. Weaknesses
   4. Concerns

• Explain all shortcomings.
• Be consistent with Criteria.
• Include findings concerning evaluation and assessment processes.
• Include findings of results to improve effectiveness of program.

• Provide all completed forms to Team Chair prior to exit meeting.

• Complete an ABET Expense Report Form, sign, attach receipts, and send to your Team Chair, who will approve and forward to ABET headquarters for payment.
• Complete online appraisal form for Team Chair and PEVs on your Visiting Team.

ABET TRAVEL AND EXPENSE POLICY: "All travelers should be aware that only persons who are authorized prior to visiting a campus will be reimbursed for their expenses. ABET will reimburse reasonable expenses incurred by those individuals who are authorized to attend specific visits. The ABET Travel Policy and Procedure Manual, approved by the ABET Board of Directors, will be provided to Program Evaluators when they are assigned to visits. This manual explains in detail how to make reservations using the ABET travel agent and how to complete the ABET Expense Report Form. Original receipts must be submitted with your expense report. All expense reports should be completed within 10 working days after returning from your visit. Failure to comply with instructions as set forth in the manual will delay the reimbursement of your expenses."