Module 4: Shortcomings

There are three levels of shortcomings:

A. Deficiency:

a criterion, policy, or procedure is not satisfied.

If you write a finding that contains a phrase such as “not satisfied”, “no indication of”, “inadequate”, “non-existent”, or “no evidence of”, then you are describing a Deficiency.

B. Weakness:

a program lacks the strength of compliance with a criterion, policy, or procedure to ensure that the quality of the program will not be compromised.

  • If you write a finding that contains a phrase such as “lacks strength”, “poor”, “marginal”, “minimal”, or “not yet fully” then you are probably describing a Weakness.
  • If “no” is found, you will get edited.

C. Concern:

a program currently satisfies a criterion, policy, or procedure, however, the potential exists for the situation to change such that the criterion, policy, or procedure may not be satisfied.

    • If you write a finding that contains a phrase such as “potential exists”, “suspect”, or “instability” then you are probably describing a Concern.
    • Should make it clear in your finding that the criterion is currently satisfied.
    • A Concern is not a “Weakness-lite.”

General Comments:

  • Editor 1s and Editor 2s see a lot of DS and FS reports and have a lot of experience.
  • Avoid “shadow criteria” in your evaluation. A “shadow criterion” is a statement written by an ABET Expert indicating a shortcoming that has the appearance of being from the Criteria but is actually not documented in the Criteria. It comes about when a PEV or TC has an interpretation of the written criteria that doesn’t reflect what is actually written – possibly driven by their view of how a program should meet a criterion. In some cases “shadow criteria” are applications of an older version of the criteria. As an example, a criterion might read that faculty should be qualified to teach a particular curricular element. It would be up to the program to defend the determination. If a PEV were to say “there is no tenure status”, “the faculty member is an adjunct”, and/or “the faculty member does not have a Ph.D.”, then these would be “shadow criteria”.
  • It is a given that an Editor will push back on you if:
    • You present a conclusion that does not align with the wording of the criterion cited.
    • You reach a conclusion that does not appear to be consistent with conclusions from other programs.
    • You present a conclusion with insufficient evidence.
    • You present a conclusion with conflicting evidence (e.g. indicate a weakness but use language describing a deficiency).
    • You present one or more “shadow criteria” as a basis for reaching a conclusion.
  • Consistency is very important, both from program to program within an institution, and from program to program across all institutions.
  • If pushback occurs, the TC should be able to justify the conclusions reached on shortcomings with little or no input from PEV – make sure you understand and agree with the PEV’s findings before the end of the site visit.
  • Make sure to provide sufficient evidence to justify your findings. Your editors can always take things out, but cannot put things in for you.
  • Remember that your editors were not on the visit and did not read the Self-Study; they are relying entirely on the submitted DS for their information.
  • An inconsistency in language can mean the language is wrong or the level of shortcoming is wrong – TC should know which it is and be able to support the finding.