Where to Find Information About Creativity Tests
Last week a reader asked “Where could one find information about reliable and updated creativity tests?” What a great question! Not an easy one to answer, but an important one.
The question brings up the basic issue of validity and reliability in testing, a topic about which many books have been written. Both are important issues in creativity testing, as well as any other kind of testing. In creativity testing, they are especially slippery.
First, the definitions of creativity seem to be as numerous as there are writers on the subject. If researchers cannot agree on what creativity is, then when using a test of creativity, test users will have a problem being sure that what they find or don’t find is actually creativity, or creative ability. Maybe the test maker’s definition of creativity is different from the user’s definition.
This is the issue of validity. Does the test measure what it purports to measure? It may measure some elements of the characteristic or behavior, or ability, but does it “work” (by identifying creativity) for all types of people (those of different ages, races, or genders)?
The validity of any measure is never perfectly resolved. It is a goal toward which test developers work. When they start to test a psychometric measure, they can make claims of validity only to the degree that the test has been used and found satisfactory with a particular group of users. As the testing continues, they may expand the claim of validity as the test makers use different groups, different situations, and still find that the measure is identifying the desired quality.
Another problem with tests of creative ability is that they do not measure other correlates of successful creative performance. These include motivation, perseverance, attention to detail, and risk-taking. It is sometimes possible to “psych out” or figure out what the test is looking for and fake the test by selecting “far out” answers, or by simply generating a large number of answers.
If you share my point of view that “the proof is in the pudding,” you’ll look not for creative ability (because every human being has the ability to be creative), but look instead at the products that the person has created, and assess these.
Reliability, how certain you are that if a person takes the test today, his score will be similar to his score in the future; or that if a group of raters look at a particular creative response they will have substantial agreement about the degree of creativity shown. Reliability is more easily tested for and assured. Usually reliability is stated statistically, and assumed to be reliable within certain range of tolerance.
There are some resources that you can consult to find reviews of educational tests, including those for creativity. Buros Mental Measurements Yearbook compiles published reviews of many tests, and their online database offers these reviews for a fee. The ETS website offers some actual tests, not reviews. Even when using Buros, one must remember that the reviewers may have their own biases. You’ll still have to decide for yourself.
5 Pitfalls to Avoid When Judging Creativity: Going with an outdated or untested measure of creativity
5. Going with an outdated or untested measure of creativity.
There are many old, “tried and true” creativity tests out there. Some of them are used even today to help select job applicants for the next step in their selection process. This can be a problem when you realize that these tests may have become familiar to prospective employees, giving them a “test-wise” bias. Also, realize that the work skills that were sought by these tests may have changed over the decades since they were first introduced.
In an effort to avoid these “war horses,” some organizations have made up their own creativity tests. These “quick and dirty” tests may be useful in identifying prospects, but their results may not prove valuable. They may fail to identify the qualities being sought, or they may be unreliable for general use in the company. Look instead for a measure that has stood the test of time, yet has been updated with improvements.
5 Pitfalls to Avoid When Judging Creativity: Looking for potential, rather than performance
4. Looking for potential, rather than performance.
If there is anything that the “genius studies” of Lewis Terman clearly demonstrated, it is that potential for greatness is often mistaken as evidence th
at the future will bring achievement and success. Terman spent decades at Stanford working the Stanford-Binet IQ test and on testing that thought would identify “geniuses” and track their success over the years. Terman hoped to be able to follow, test, and cultivate a generation of geniuses to show how to nurture special talent in children. His conception of giftedness was closely linked to academic talent, and he expected that the youngsters he identified would be successful in both school and life. His research was colored by his biased attitudes about the supposedly limited abilities of girls, many immigrants, and persons of any race but his own.
Over the decades of his studies, many of the students he followed did make impressive achievements. But, it must be admitted that many others, identified as having great potential, often did not manifest this talent through their achievements. What’s more, other children – not included in his studies — who didn’t appear to have that much going for them were able to achieve greatness, including for example, winning Nobel Prizes (which none of the “Termites” had achieved.) This gives hope to all of the rest of us “non-geniuses.”
If you want to find a creative person, look at what she has made, done, or built. If one has been creatively productive in the past, he or she will be likely to do it again. Don’t make the mistake of thinking that creative potential will imply creative productivity in the future.
5 Pitfalls to Avoid When Judging Creativity: Timing the Test
3. Timing the test.
Creativity requires the kind of relaxed attention that allows one to be “in the flow.” Sometimes that comes quickly, but it often takes time to tease out new ideas from the many mundane ideas that pass through our awareness.
Consider the new TV reality show: Work of Art: The Next Great Artist, where aspiring artists are pitted against one another in timed competition to create works of art of a particular type in 12 to 14 hours. This brings the shocking juxtaposition of our on-demand lifestyle to the world of culture.
While some professional art fields (industrial and graphic design, for example) require production on a schedule, the constraints of the TV show’s production schedule make the artistic products more like art school test projects than actual works of art. It must be frustrating for the artists on the show to have to set their artistic goals to what can be achieved in the time available.
On creativity tests, as well, the fact that you’re being timed can be an inhibition to creative production. Certainly, one cannot wait forever for a creative idea, but putting pressure on test takers is not conducive to the most creative solutions. If you have to limit the time that is allowed to solve a problem, make the amount of time provided ample so that test-takers are not rushed.
5 Pitfalls to Avoid When Judging Creativity: Assessing creativity with a tool that looks only for divergent thinking
2. Assessing creativity with a tool that looks only for divergent thinking.
Creative thinking makes ample use of divergent thinking, so it makes good sense to consider it when trying to assess creativity. When creativity tests were first devised, test designers were looking for ways to measure the ability to think “far out” as a way to counter the effects of the convergent thinking training we get in school. Fluent divergent thinkers can come up with lots of new ideas. They may not all be good ideas, but they can think up dozens.
When a creativity test asks for as many uses for a paper clip as you can think of, it is calling for divergent thinking. But, for an idea to be truly creative, it must be novel and useful, while attractive enough to encourage adoption. Few pencil and paper tests of creativity actually consider all three dimensions of creative production. A test that merely counts the number of new ideas (as some do) is not a good way of judging real life creativity.