ABET

Module 4: What You Need to Know On the Visit

A. Completing the On-Site Visit

At this point in the on-site visit, the team has assessed the program against the accreditation criteria and is transitioning from discovery to reporting. As team chair, your main goal shifts to completion of the exit statement(s) and program audit forms (PAFs) and preparation for the exit meeting.

Key Team Chair activities during the report phase will be described in detail in this module.

B. Writing the Exit Statement for a General Review

The exit statement for a General Review consists of the following elements:

  • Introduction
  • Program Strengths
  • Program Shortcomings (Deficiencies, Weaknesses, Concerns)
  • Observations

The introduction section includes such information as:

  • Type or special characteristics of the program.
  • Emphasis area(s).
  • Number of enrolled students.
  • Size of most recent graduating class.
  • Number of faculty members.
  • Other information that could be helpful to the next team.

Introduction Example

The industrial engineering program prepares students for careers in system design and optimization and ergonomics. All courses in the major are available by distance education, but students must come to campus or have facilities available through their employers for laboratory experiences in two courses. The current student enrollment is 150 students. Twenty-nine students graduated during the past academic year. There are 14 full-time faculty members supporting the program.

Program Strengths

Program strengths are exceptionally strong and effective practices or conditions that stand above the norm and have a positive effect on the program. For each program strength identified, state:

  • What was observed
  • What makes it stand above the norm
  • The positive effect it has on the program

In the following example, italicized text identifies the criterion/policy citation, underlined text the evidence, and bold text the effect on the program.

Strength Example

The Office of Student Services and Career Development is highly successful in placing the graduates of the college. Important services provided by this office include career counseling, workshops on interviewing readiness, and training in study skills. The office uses JOBTAK to assist with referrals and job placement. These activities help the individual programs achieve their objectives in the successful job placement of their graduates.

The following are NOT program strengths:

  • The faculty members are dedicated and very hard working.
  • The departmental chair is an excellent administrator and is well respected by the faculty.

Please Note: ETAC considers Strengths to be Observations and includes them at the end of the exit statement.

Program Shortcomings

Program shortcomings are Deficiencies, Weaknesses, or Concerns. For each program shortcoming identified, state:

  • Criterion/policy citation
  • What was observed
  • Effect on the program

The criterion/policy citation clearly states the criterion or policy that applies; excerpts may be quoted as needed. The “what was observed” section describes the evidence that was observed through the Self-Study, display materials, facilities tour, and/or interviews with faculty, staff, and students. The effect describes how the program is negatively affected by the shortcoming (if a deficiency or weakness) or may be potentially affected by the shortcoming (if a concern) with respect to the criterion or policy.

In the following example, italicized text identifies the criterion/policy citation, underlined text the evidence, and bold text the effect on the program.

Deficiency Example

Criterion 1 requires that students who graduate from the program meet all program requirements. While most students who graduated successfully completed all requirements, there were a few students who did not pass all of the required courses designated by the program. Students can graduate from the program without passing all the required courses. Thus, the program is not in compliance with the Student Criterion.

Weakness Example

The Continuous Improvement Criterion requires that each program have a documented process for assessing and evaluating the extent to which program educational objectives are attained. The program has relied on an alumni survey conducted every three years for assessing program educational objectives; the University manages this survey. The employer survey data is not differentiated by discipline, and the program did not provide evidence of any program-specific data. Without such program-specific data, the program is unable to evaluate and demonstrate the degree to which the objectives are attained. Thus, strength of compliance with the criterion is lacking.

Concern Example

The Institutional Support Criterion requires that resources be adequate to attract, retain, and provide for the continued professional development of the faculty. Resources must also be sufficient to acquire, maintain, and operate facilities and equipment appropriate for the programAt present, it appears that resources are adequate to support the program. However, there have recently been large reductions in the operating budget of the department. If these budgetary reductions continue, future compliance with this criterion may be jeopardized.

An Observation is “a comment or suggestion which does not relate directly to the accreditation action but is offered to assist the institution in its continuing efforts to improve its programs.” (APPM Section II.G.9.a.(3))

Observations

Observation Example

Students expressed a desire for opportunities to meet with alumni role models and to learn about possible career paths. A program of regularly inviting speakers from industry to the campus to interact with students is offered as a suggestion for addressing this desire.

Additional Guidelines for Writing Statements

  • Statements must be based on conclusions from evidence found by the team to be credible and significant.
  • Names of individuals or titles that identify individuals must not be used.
  • Avoid attributing opinions, conclusions, or recommendations to other individuals or groups of individuals besides the commission.
  • Poor: The team discovered
  • Better: Evidence indicated
  • Provide enough detail that the program and institution will know precisely what is inadequate.
  • Provide enough detail to the commission to justify the type of finding.
  • Provide enough detail that the team making an interim review will be able to determine the amount of progress that has been made since the original finding.
  • If wording does not add to or clarify the finding, then do not include it.
  • Avoid the use of acronyms or other abbreviations.
  • Avoid combining findings on two criteria into one statement.

Commission Style Guides

Sample guidelines and statements may be found in the Commission style guides located in the TC Workbooks.

Program Audit Form

The Program Audit Form (PAF) is left with the institution at the end of the visit. It MUST be consistent with the exit statement in level of compliance and wording used to describe what was observed and the effect on the program. These are located in the TC Workbooks.

 

C. Draft Team Chair Exit Statement

Based on the visit team discussion and PEV draft exit statements, draft the team chair exit statement. The team chair exit statement includes an introduction of the institution and only those strengths and/or shortcomings that are common across all programs evaluated. Use the same format for strengths and program shortcomings as used in the program exit statement. The shortcoming should appear on the Program Audit Form for each individual program. It should also appear in the draft statement for each individual program; it is included in the team chair exit statement as a means to simplify the exit meeting.

D. Conduct Informal Debrief With the Dean(s)

Late morning on Day 2, the team chair informally shares with the dean(s) findings related to the strengths and shortcomings of the programs being evaluated.

While, as team chair, you should be keeping the dean(s) informed of potential shortcomings throughout the visit, this is an opportunity to share the team consensus and to make sure the dean(s) understands what evidence was found to support each shortcoming.

The team chair should have copies of the draft exit statements prepared by each PEV and that are being informally debriefed with the program head(s) at the same time. It is also an opportunity to review the next steps after the visit: seven-day response and due process. It is appropriate to suggest ideas for improvement based on what was seen during the accreditation review as long as it is presented in the spirit of a suggestion for improvement, not as a “silver bullet” remedy or resolution for a shortcoming.

No statements of what would be an acceptable response (i.e., what would clear up a shortcoming) or anything that could be interpreted as contradictory to the exit or draft statements should be stated. DO NOT state the recommended accreditation action.

E. Conduct Day 2 Team Meeting

Suggested Outline for the Day 2 Team Meeting

Note: Team chair and team members in executive session — allow 2 hours maximum. Experienced chairs and PEVs should be able to conduct this meeting more expeditiously. The meeting is held over lunch, after the program head and dean informal debriefs and before the exit meeting.

1. Reiterate documentation requirements.

2. Review additional information/evidence obtained during the morning and discuss findings and recommendations.

Allow the team members to ask questions and to discuss evidence and criterion citation.
Seek consensus of recommendations.

3. Read exit statements.

  • Ask each PEV to read his or her exit statement to the entire team.
  • Ask for suggestions for improvement and consistency.
  • Discuss elements that are common across multiple programs for consistency and inclusion in team chair exit statement.
  • Pair up PEVs to review each other’s final exit statements.

4. Review procedure and plan for exit meeting.

5. Address any additional questions by the team.

F. Complete Team Chair Exit Statement and A2 (Short Form) Accreditation Action Form

Once the team has agreed to the final program exit statements and recommendations, finalize the team chair exit statement. Also, complete the Accreditation Action Form, A2, or commonly known as the Short Form. Complete the A2a worksheet header with the name of the institution, NGR cycle, and visit dates. Identify the commission, team chair, and editors in the spaces provided. For each program evaluated, provide the previous action, the program name as it appears on the Request for Evaluation (RFE), specify whether the name has been changed since the last review, the degree (i.e., AS, AAS, BA, BS, BE, MS), the name of the PEV, the society represented by the program evaluator and the PEV and team chair recommended action (NGR, IR, RE, IV, VE, NA, SCR, SCV, SE, T) in the appropriate columns. Identify the names of any State Board Observers and society Observers on the team. Also complete the second worksheet in the file, A2b, by providing institutional contact information.

The A2 consists of two worksheets. Don’t forget to complete them both.

G. Lead Exit Meeting

Suggested Outline for Exit Meeting

Note: Institutional personnel present are to be as determined by the chief administrative officer of the institution. Preference is for the President/Chancellor, provost, dean(s), and program heads. Entire visiting team is present. Time will be dependent on the number of programs evaluated. Strive to keep within 30 minutes to an hour.

1. Try to schedule departure from the institution so that the exit meeting need not be rushed, but keep it short enough to preclude its becoming a debate session.

2. Thanks on behalf of team for courtesy, cooperation, hospitality, etc.

3. Other compliments as appropriate, such as to dean(s), program head(s), faculty members, staff on excellent preparation/display materials, arrangements for visit, etc., to president and others for making themselves available, etc.

4. Review purpose of meeting:

  • Team will report major findings. (Team chair determines who will report and order.)
  • Request identification of errors of fact or omission or need for clarification. Note that debate or rebuttal is not appropriate at this time.
  • Emphasize the role of the team as information gatherers — including qualitative factors — for the Commission of ABET.
  • Review the sequence of presentations as well as definitions of Deficiencies, Weaknesses, and Concerns.
  • Reiterate that team’s recommendations are not final and are subject to review and change through the extensive editorial and due process procedures.
  • Remind the audience that there is to be no stenographic transcript or tape recording and that a copy of the Program Audit Forms will be left with the dean or his/her designee. If the institution insists, reiterate that team’s comments are not the official ABET report, and use great care in the discussion.
  • Remind the institution of their opportunity to prepare and submit a seven-day response to this exit meeting statement. Discuss content of an appropriate response — e.g., short and to-the-point; addressing only the issues introduced in the exit statement. Also remind the institution of the additional opportunity for a 30-day response to the Draft Statement, when it arrives in several weeks.
  • Reiterate what constitutes an appropriate response (as in the seven-day response). Ask that the institution ABET liaison notify you via email when the responses have been sent.

5. Review ABET policy on public release of information, proper terminology, and confidentiality of reports. (Refer to APPM section II.A.)

6. Review procedure for Draft Statement and response. (Refer to APPM sections II.G.10 and II.G.11.)

7. Presentations (it is preferred that the exit statement(s) be read to provide a precise and concise report):

  • Team chair reads the team chair exit statement, including institutional factors that affect the unit and/or all programs: Strengths, Deficiencies, Weaknesses, and Concerns.
  • Some or all team members read exit statement for individual programs: Strengths; Deficiencies; Weaknesses; Concerns. Each program is to be addressed, in alphabetical order. If there is more than one PEV for a program, the team chair must assure that there is only one consensus report delivered.

8. Return unneeded copies of the Self-Study questionnaire to the institution. Leave a copy of the Program Audit Form(s) with the dean.

9. At the conclusion of the exit meeting, depart from the institution promptly. Do not stay around for further conversation. As the team disperses, remember to thank each member and remind them that you will be in touch.

H. Levels of Compliance and Accreditation Actions

When writing the exit statement for the program you will be reviewing, you will need to use a variety of terms and statements to define levels of compliance (Accreditation Policy and Procedures Manual II. G. 9. a. and Glossary.)

Findings of Concern: A Concern indicates that a program currently satisfies a criterion, policy, or procedure. However, the potential exists for the situation to change such that the criterion, policy, or procedure may not be satisfied.

Findings of Weakness: A Weakness indicates that a program lacks the strength of compliance with a criterion, policy, or procedure to ensure that the quality of the program will not be compromised. Therefore, remedial action is required to strengthen compliance with the criterion, policy, or procedure prior to the next review.

Findings of Deficiency: A Deficiency indicates a criterion, policy, or procedure is not satisfied. Therefore, the program is not in compliance with the criterion, policy, or procedure.

Statements of Observation: An Observation is a comment or suggestion that does not relate directly to the accreditation action but is offered to assist the institution in its continuing efforts to improve its programs.

Statements of Strength: A Strength is an exceptionally strong, effective practice or condition that stands above the norm, and has a positive effect on the program.

Accreditation Actions

Based on program review, the team will select one of the following potential actions as described in the Accreditation Policy and Procedure Manual, Section II.G.12.

NGR – Next General Review
IR – Interim Report
IV – Interim Visit
RE – Report Extended
VE – Visit Extended
SCR – Show Cause Report
SCV – Show Cause Visit
SE – Show Cause Extended
NA – Not to Accredit
T – Terminate

Take time to read the Accreditation Policy and Procedure Manual, Section II.G.12 now.

I. General Review Terminology vs. Action

If in the team’s judgment a program has a Weakness within a given criterion, the team must recommend either an Interim Report or an Interim Visit action. An Interim Visit is usually recommended over an Interim Report when the nature of the shortcoming(s) requires observation to determine the appropriateness of the action taken to correct the situation.

If in the team’s judgment a program has a Deficiency within a given criterion, the team must recommend a Show Cause action if this is a General or Interim Review or a Not-to-Accredit action if this is an initial review. A Show Cause Visit is usually recommended over a Show Cause Report when the nature of the Deficiency(ies) requires observation to determine the appropriateness of the action taken to correct the situation.

Weakness? No Yes Yes
Deficiency? No No No Yes Yes
Actions NGR IR IV SC NA
Definitions Next General Review Interim Report Interim Visit Show Cause (only for re-accreditation) Not to Accredit (only for new programs)
Duration (years) 6 2 2 2

J. Team Management

The Team Chair must manage the visit team throughout the entire visit process. We will explore this through several scenarios. While these are rare occurrences, they have happened on real visits!

Scenario 1: One of your PEVs is too prescriptive, insisting on his/her own solution as to what the program needs to do and how to do it.

  • Remind the PEV that the mission of ABET accreditation (APPM Section I.C) is to promote quality and innovation in education, not to prescribe improvement.
  • Remind the PEV that it is the institution’s responsibility to demonstrate how the program meets the relevant criteria.
  • Focus the PEV on the wording of the criteria.
  • Remind the team that the recommendation is a TEAM recommendation, and they need to help ensure the criteria are being applied correctly.
  • Make note of the situation and include in the team chair appraisal of the PEV.
  • To minimize the occurrence of this scenario, discuss the mission of ABET and the role of the PEV with the visit team during a pre-visit conference call and again during the first Day 0 meeting.

Scenario 2: Your flight is delayed, and you will not arrive until late Sunday night or early Monday morning. What does the team do?

  • Ask one of the experienced PEVs on the visit to lead the Day 0 team meetings.
  • Forward all appropriate documentation regarding the Day 0 team meetings and other arrangements to the experienced PEV electronically.
  • Notify the dean(s) as soon as possible and provide the name of the experienced PEV that will be leading the team meetings on Day 0 so they can coordinate logistics. Make sure you have a home phone or cell number of the dean(s) so you can contact him/her in such situations.
  • To avoid this situation, make your travel arrangements to arrive on site the day before Day 0 (typically Saturday).
  • Have the cell phone number of every visit team member with you to be able to contact one or more while in transit and during the visit.

Scenario 3: In discussions with a program head, you discover that a PEV is being overly aggressive and unprofessional with faculty and staff discuss the feedback with the PEV and remind him/her of the PEV interpersonally skilled competency.

  • Discuss the feedback with the PEV and remind him/her of the PEV interpersonally skilled competency.
  • Review the questions the PEV has planned for the remaining interviews with faculty, staff, and students, and provide suggestions on how to deliver without coming across as aggressive and unprofessional.
  • If your schedule allows, attend one or more interviews with the PEV to observe their behavior first-hand.
  • To minimize such scenarios, review the PEV Competency Model with the visit team during a pre-visit conference call and the first Day 0 meeting.

Scenario 4: One of your PEVs is dressed casually (jeans) on the first day of the visit and insults the dean and several others.

  • When first seeing the PEV, ask him/her if he/she has more appropriate business attire. If so, ask him/her to return to his/her room and change. Arrange for separate transportation to the dean’s meeting if needed. If the PEV does not have more appropriate business attire, notify the dean and ask for recommendations on where to quickly find a store for the PEV to purchase more appropriate attire (Wal-Mart is open 24 hours!) without impacting the visit schedule.
  • To minimize such situations, review appropriate attire during a pre-visit conference call. Also, revisit with the team during the first Day 0 meeting and make arrangements for any offending PEVs to visit a local store to purchase more appropriate attire without impacting the visit schedule.

Scenario 5: Two evaluators for different programs in the same department have very different opinions about the compliance of a criterion that is addressed in a common way across both programs.

  • Confront the matter directly as soon as it arises. During the pre-visit review, if identified at that time.
  • Challenge both PEVs to clearly explain the evidence found (or not found) regarding the criterion.
    • If the rationale is grounded in real differences between the programs, fine. Make sure the exit statements and PAFs clearly distinguish the differences.
    • If not, work with the PEVs to resolve their differences of opinion BEFORE the informal debriefs with the program head and the dean.
  • To avoid such scenarios, request that each PEV provide you with a copy of their pre-visit review at least one week prior to the visit. Identify such potential situations and engage both PEVs prior to the visit.

Scenario 6: A PEV has drafted an exit statement and PAF in which language is not consistent with the level of compliance recommended by the team.

  • Do NOT accept the exit statement and PAF.
  • Work with the PEV to modify the language to be consistent with the level of compliance recommended by the team.
  • To minimize such situations, review the definitions of the levels of compliance with the team on a pre-visit conference call and during the Day 0 meeting. Also review the format for writing the exit statement. Stress the importance of consistency.

Scenario 7: One of your PEVs has a family emergency on Sunday afternoon of the visit and has to return home immediately.

  • Gather all notes and preparatory materials from the PEV and work with the remaining team members to determine how to review the program.
    • If the program is being visited by more than one PEV, the second may simply be able to take the leaving PEVs notes and complete the review from both perspectives.
    • If the visit team includes a society Observer (new PEV) from the same society as the leaving PEV or a complementary program area, put the Observer into service. Note: In this case, you will need to mentor and support the Observer.
    • Split responsibility for interviews and display material review across team members. Note: This will require significant coordination between team members.
    • If the visit team is small, see if you can re-arrange your schedule to conduct part of the program review. Note: It is important that you review each program Self-Study in sufficient detail so that you can step into the PEV role in such situations.
    • If the visit team includes a co-team chair, one or both of you conduct the program review.
  • To minimize this situation, ask visit team members to notify you of any potential family emergencies that may disrupt their full participation in the visit as soon as possible. If prior to visit, work with the ABET Accreditation Director and the member society to either find a replacement or to have a back-up plan.

Scenario 8: After you arrive on site, you discover that one of your PEVs is unprepared for the visit, having done little or no advanced preparation.

  • Determine how little the PEV has done. Determine whether to send him/her home based on his/her experience level and the number of PEVs visiting the program.
    • If you send the unprepared PEV home, notify the editor and the ABET Accreditation Director IMMEDIATELY; and follow actions described for Scenario 7 to evaluate the affected program.
    • If you decide not to send the PEV home, the team should carefully scrutinize the PEV’s findings and recommendations.
  • Make note of the situation and include in the team chair appraisal of the PEV.
  • To avoid such scenarios, request that each PEV provide you with a copy of their pre-visit review at least one week prior to the visit.

Important Note: Always have the phone number(s) of the two editors and the Accreditation Director with you during the visit should you need to contact them for advice as a result of a team, program, or institutional issue.

K. Summary of Module 4

In summary, the team chair is very busy during the on-site visit managing the visit team, managing the visit team schedule, and ensuring the quality of the output (exit statement(s) and PAF(s)) of the visit team. All six team chair competencies are needed: technically current, effective communicator, good team manager, professional, organized and responsive, leadership.

Go to Proficiency Assessment #4>