Many — including writers for this paper — have sung the praises of this fall’s Undergraduate Student Experience and Satisfaction report, and its promise for improving the life of UCSD students. Pity that so much fuss has been made over a document so methodologically flawed.
For those still unacquainted with the study, the document was written by a committee tasked with finding out essentially why UCSD seems to suck more than other UC campuses. The hope was that these defects would be fixed after they were identified.
From the start, the committee faced key design decisions: Optimally, given an unlimited budget and deadline, it would have drawn a random sample of students — statistically similar to the rest of campus — and gone from there. Alas, this was not the case. Instead of systematic, valid measurements, the committee opted to create what is essentially a collection of anecdotes (which the committee itself admits on page eight). In its own words: “The committee felt that personal interviews offer insights with a depth and richness that no check-box questionnaire could easily uncover. Each student and each student group we talked with had stories to tell, and each one of those stories reflected a personal experience and a personal view.”
As amusing as personal experiences and views may be to read, they unfortunately do little to identify institutional and systematic flaws.
The design problems also greatly limit the usability of the report’s conclusions. Consider, for example, the portion identifying “communities for whom campus life has great potential for improvement:” ethnic groups; commuters; transfers; athletes and Greeks.
Now look at the special interests that dominated the pool of students interviewed in the process of preparing the report (listed in Appendix C): Cross-Cultural Center, Transfer Student Advisory Board, Triton Athletic Council, Panhellenic Association and the Inter-Fraternity Council.
Similarities? Indeed, if the campus’ lesbian vegetarians organized into a registered student org, and if the org had met with the report’s authors, I’d wager good money that they would also be listed as such a “community.”
Which highlights the problem: Did the committee recommend greater funding for athletics because this is indeed an important issue, or simply because the Triton Athletic Council had an opportunity to catch its ear? Other similar questions abound.
Admittedly, the report does a marvelous job documenting the dissatisfaction of UCSD students. Where it fails is in providing a similarly rigorous analysis of what to do about it.
In key areas, the report seems to be particularly vague — to say the least. At one point, the report recounts a scene in which a professor, in the process of handing out the Course and Professor Evaluations, tells students that their “comments were irrelevant because I bring in too much to the campus in research grants.” The report concludes, rather philosophically: “Whether this is an accurate story or an urban myth, this represents the common perception among students.”
Well, shouldn’t the committee have actually investigated whether the story is accurate, and whether C.A.P.E. scores do matter? Indeed, the report labels the perception that faculty care more about their research than teaching as likely another such “urban myth.”
However, had the committee taken the time to review the university’s Academic Personnel Manual — in particular section 667 — it would have found that professors often receive “additional” compensation for bringing in outside research grants, which may explain why they would value research over instruction. Nowhere, though, does the report address the university’s compensation system, or consider the perverse incentives it creates, though it does devote half a page to students complaining about their “Nazi-like” residential security officers.
At the end, the study is a political document, doing more to chronicle the political demands of the self-interested student groups that participated in its creation than the needs of the average student. It does little to go beyond the campus college council (the other major contributors) to represent the views of the more than 80 percent of students who don’t bother voting in campus elections.
Do students who don’t give a rat’s behind about sports, who don’t attend A.S. Council meetings and who aren’t members of Pi Kappa Alpha really believe that what ails the university is the absence of a “college village” on campus (the creation of which is the committee’s main recommendation)?
The report never tells us.
