Educational Decision Making Should Be Empirically Driven,
Not Driven by Intuition or Tarot Card Reading
By Bill Matthew, Special Ed Director / School Psychologist

Print this page

Dear Pete & Pam,

I've taken a thorough look at your web page. I think it's outstanding, especially the court cases. I'm a special ed director who is constantly trying to improve our special education services via good, sound empirically-driven practices. That's the KEY.

Special Educaton Should Use Empirically-Driven Practices

Educational decision-making should be empirically driven - not driven by intuition or tarot card readin - the latter is part and parcel of much psychology, especially in assessment.

Along that line, Pete's Letter to Bobbie about Discipline is excellent, especially the reference to animal training.

It galls me that special education training programs don't mandate coursework in applied behavior analysis which has a sound empirical base. Such training would greatly improve the education received by nearly all special ed populations.

We can train Shamu but we can't train a kid to be compliant or read?? Give me a break!!

Suggestions for Tests & Measurements Article

I'd like to make some suggestions about your Tests and Measurements article concerning some concepts you've presented that really don't have empirical support.

I know you want parents to have the best information available, so I really think you should make the following changes so your page will be highly credible, or at least read some of the more empirically based literature that I've suggested in previous emails. (Note from Wrightslaw: The changes recommended by Bill were made)

In your section on Composite Scores (the example with John), the percentiles were arithmetically manipulated (found a mean)--this can't be done mathematically. Percentiles are ordinal scales which can't be averaged.

Using Age and Grade Equivalent Scores - May be Inaccurate

Discourage the use of grade & age equivalents -- they're also ordinal measures, terribly inaccurate, they promote inaccurate typological thinking and they're grossly inaccurate for measuring growth. I know parents like 'em, but they can lead to erroneous conceptions.

Using Subtest Scatter

The use of subtest scatter analysis - this is a common practice, and may be okay if one only looks at the variation in performance as a hypothesis-generator for more in-depth investigation.

Even then, the psychometric validity/reliability of the specific subtests must be considered on their own merit before deciding to put any real value on the subtest score. To do so in the absence of sound psychometric evidence is at the very least unethical.

The use of subtest scatter (e.g., WISC-III) in the attempt to "differentially" diagnose a disability/problem is not supported by the empirical literature, even though it has some intuitive appeal and "testimonials" from clinicians abound.

Using Projective Tests

Your reference to projective tests as a valuable source of data really needs to be re-examined. This stuff really has no place in school psychology. If I were a parent and attended an IEP on my son where such data was presented, I'd be outraged. There's better stuff available.

I refer you to Howard Knoff's chapter on Personality Assessment in NASP's Best Practices-III for a thorough treatment of the personality assessment issue.

Using Tests That are Psychometrically Sound

In your section on private/independent assessments, you infer that school districts don't have the assessment capabilities that private practitioners have.

Keep in mind that "good" school districts will follow the law and only use only tests that have been "validated for the purpose for which they're used", tests that are psychometrically sound.

For example, my psychologists are not allowed to use many tests - projective included - simply because they don't have the necessary reliability (.85 test-retest or higher), validity, or standardization characteristics. This constraint - a good one - is a quality control issue. Private practitioners are not bound by this. They have access to an array of tests that may have a lot of "face validity", but no real empirical support.

Warn Parents About Evaluations That Do Not Use Sound Psychometrics

Parents must be cautioned against seeking a private evaluation that only serves to support their position in the absence of good sound psychometrics.

Exceptions to this is in the area of conducting systematic behavioral observations in the classroom or other relevant environments. Most school psychs/diagnosticians are not trained to do this, and a private examiner may be if they've had good applied behavior analysis training. This data is infinitely more valuable (reliable and valid) in determining behavioral discrepancies in SED cases and in devising appropriate treatment plans( IEP's) and in measuring progress.

Advocating for Kids with Emotional Disorders

I would love to be an advocate for parents in SED cases where "appropriate" treatment is being determined. Here in CA that means a one hour session where a verbally based form of insight therapy is used, typically with an elementary aged, low SES, conduct disorder/oppositional behavior type.

Absolutely an unethical, ineffective treatment model for this population of kids.

If you want to do some good advocacy work, come out here and help the CA Dept. of Mental Health set up such parameters for the mental health treatment of special ed kids. I'd love it. Kids have a right to effective treatment, and traditional therapy is not where its at with these conduct disorder kids.

Really enjoy your web page -

Bill Matthew, Ph.D
Director, Special Education
Delano Union School District
Delano , CA
Email: bmatthew@duesd.org