Charles Handler recently blogged on a topic that I have written about several times as well—the challenges involved with gathering the data needed to explore the degree to which assessment programs brings value to an organization.  These include problems associated with measuring job performance (subjective ratings tend to be quite unreliable so doing this properly requires an investment of time and resources), small samples (which create difficulties for statistical analysis), and translating this information into terms the rest of the organization understands (that is, dollars and cents).

Charles makes a number of suggestions that I think are worth considering. The first is to target “low-hanging” fruit. Here, the approach is to find those positions for which there is already decent job performance data and start your efforts here. If successful, you will have built up some good will that may convince the organization to invest the resources to tackle those job positions which will require more effort to obtain good outcome measures. Another is to consider using supervisory ratings. While sometimes suspect, they can be used as effectively if supervisors are trained in their use and supported in their efforts. A third suggestion is for organizations to look to vendors in the pre-employment selection space who have substantial experience in implementing local validation studies.

While there are no silver bullets, it is important for organizations that use assessments to seriously consider making the effort to collect the type of job performance data that will enable to make a reasonable decision about the value they obtain from their assessment programs. Otherwise, decisions are often made upon beliefs that may or may not be consistent with the actual situation.