Billy Wilder on audiences:
Individually, they’re idiots. Collectively, they’re a genius.
I’m not sure when Hollywood began running test screenings, but I know it goes way back. In 1942, two preview screenings of The Magnificent Ambersons (written and directed by Orson Welles) went over like wet blankets and RKO, the studio, lopped 40 minutes off and re-shot the ending. Likewise, the recent World War Z had a third act that tested poorly and was completely re-written and re-shot. One might conclude not much has changed.
Criticisms aside, there is a scientific method to modern test screenings, which are more often aimed at figuring out how to market the film than how to re-edit it. The two main companies that make a business of running audience-recruitment screenings — and make no mistake, sample quality is more important than sample size — are NRG (a division of Nielsen, expensive) and MPG (geared toward indies).
What is the standard format for a test screening?
Rent a theater, show an audience a cut, have them fill out a questionnaire.
Depending on what you want to test, you gather an appropriate audience. If you want to see how the movie plays among suburban soccer moms, you need to get out to the soccer fields and offer free orange wedges.
You can get good and useful opinions from friends and family, but it is hard to know how much bias they bring. You definitely know they bring bias. Sometimes, they over-compensate by hating on your movie much more than the general public. This is a very real phenomenon. The screening companies even try to screen out anyone who works in the film industry. (Although the recruiters they employ don’t try too hard, since they are paid based on who shows up to the screenings.)
Any way, I hope you don’t need a lot of convincing to imagine that the best test screening audience is one that A) Doesn’t know the filmmakers; B) is not a wanna-be filmmaker; C) approximates more or less the type of audience that the actual marketing for the film will pull in.
That last reason is why you’ll see screenings advertised as “GENRE starring ACTOR A and ACTOR B about BASIC PLOT DESCRIPTION” — e.g. A thriller starring Arnold Schwartzenegger and Zach Galifinakis about a father and son trapped in an avalanche.
While the screening companies are tight-lipped about their methodology, I’ve come across a sample questionnaire that looks very similar to ones I’ve seen at screenings.
You will have to customize it, obviously, to your film and what you’d like to know. John August’s favorite test screening question is, “Given a pair of magical scissors, is there anything you’d snip out?”
*Note: To make your own version of the Google Doc, go to ‘File > Make a copy…’
What are the top two boxes?
The top two boxes are the two “Yes” answers to “Would you recommend this movie to your friends?” I have heard that the rule of thumb is that movie is ready to release when it scores 80 or more on the top two boxes, meaning 80%+ of the audience would recommend it. Word of mouth is still the holy grail for movie marketing.
If your film doesn’t manage to score what you wanted with your target audience, then maybe it’s time to rethink the film — or the target audience.
What are the limits of audience testing?
"What is wrong with audience research? It doesn't work. If it worked, there would be no flops." - David Mamet
Audience testing is a guide, but the ultimate artistic decisions to hear or ignore it will determine whether it mauls a masterpiece (as in the supposed case of Magnificent Ambersons) or whether it rescues a turkey (as in the supposed case of World War Z).
‘Movie theater’ image by Bonita Sarita