(I'd swear I posted this years ago, but I cannot find it. Since the topic keeps coming up, I'll post it now.)
Many people misunderstand "objective" to be somehow more valid than "subjective". This is not true at all.
Objective merely means in accordance with an external standard. For example, a person can use the objective standard of a ruler or yardstick to measure the width of a room. However, if that objective standard is itself wrong--say the yardstick used is not 36" long--then the result is certainly invalid.
A subjective measurement is based on internal rather than external standards, but can be just as (or more) valid. If someone who is very good at judging distances tells you how wide the room is, he may be more accurate than someone who measures with a yardstick, especially if that yardstick is faulty.
Many matters cannot be measured objectively, because we have no way to do so. Can "customer satisfaction" be measured objectively? No, it is necessarily subjective. Yet it is nonetheless of vital importance to, say, computer support personnel. If you try to measure how "good" computer support is by counting things, you may come to faulty conclusions. Simple example: is it good that over time there are more and more calls to a computer Help Desk in a corporation? Well, you could conclude that more people realize that the Help Desk really does help (some don't, you know). Or you could conclude that the Help Desk isn't doing very well, so people have to call back. Or you could conclude that the training for workers, so that they can manage to do things themselves, is faulty, so they have to call the Help Desk. And so forth. An increase in number of calls can be argued both ways. The only way to be sure is to survey the people who use (or could use) the Help Desk, a subjective measure that is much more valid than the objective measure of number of phone calls.
Most sports, for example, rely on subjective evaluations (referees, judges in figure skating or boxing, etc.). Sometimes people are unhappy, but the prizes are still awarded. The judges/referees/umpires use "objective" standards, but they necessarily apply them in a subjective way--"judgment calls".
How about the famous dog shows, judged by a single referee? We could just have a time trial and measure the running speed of the dogs, and that would be an objective measurement, but it wouldn't tell us which was "best", only which was fastest in a time trial.
There appears to be no *meaningful* objective measure available to determine whether a game is "great" or "flawed" or "awful" or somewhere else in between. We can try to use a combination of "kind of meaningful" (do lots of people play it?) and "kind of measurable" (the effect of the game on people, which we have no means to measure objectively) and "kind of objective" (is there player elimination, etc.)
I see people applying the (wrong) objective standard to so many things, because they cannot easily measure what is really important. I'm a college teacher. In teaching, accreditation bodies measure whether people have a degree rather than whether they know the subject (that's too hard to determine). They don't even imagine they should try to measure whether the person is a good teacher--it's too hard, so they measure something that is truly unimportant. My neighbor tells me that a respected teacher at her high school, who has taught special education for 32 years, got a letter from the state telling him he was unqualified--because he didn't have the "proper degree". In assessment of students, we see "end of class" multiple choice tests as the only determiner of student/school quality--MC tests are a poor way to determine whether someone knows something--and as a result we have students coming into colleges who have been trained, not educated, and cannot think for themselves. Even the good students don't understand what they're doing.
At one high school system in a major city, they have a cooperative agreements with the colleges, and instead of running "AP" (Advance Placement) classes they run classes via the colleges--except for American history The entire school system's worth is measured by how students do on their American History End-of-class test because that's all there is to "objectively" measure. Balderdash and poppycock!
Examples can be multiplied in other fields, I'm just using ones (computer support, education) that I'm familiar with.
Friday, August 26, 2011
Subscribe to:
Post Comments (Atom)
"Always do right--this will gratify some and astonish the rest."Mark Twain
"A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away." Antoine de Saint-Exup'ery
"Not everything that can be counted counts, and not everything that counts can be counted." Albert Einstein
"Make everything as simple as possible, but not simpler." Albert Einstein
1 comment:
You are brutal, which is fine, but your thoughts on the next generation of game designers and game developers is too much. I get that there are dreamers but turning this dream and passion of mine into a reality is what keeps me going every day. I have no experience in the industry but that does not mean that I think everything will be easy. I did not start down this path because of delusions about what I think it will be like to work in the video game industry. I don't usually rant or even comment for that matter but I felt that this was necessary. I do enjoy your blog and will continue to read it. Thank you.
Post a Comment