Rotten Tomatoes is a review aggregator website. It was founded in 1998. There were previous sites that did — and still do — the same basic thing, like Metacritic. For lazy journalists, the Rotten Tomatoes score has become a shorthand for quality of a film. Headlines proclaim Wonder Woman 97% Fresh and The Mummy 23% Rotten.

In fact, the method by which these scores are generated is unscientific, arbitrary and ultimately harmful to movie-making. Let’s break it down…

1. Reviews don’t neatly divide between “rotten” and “fresh”

Film critics do not write at the top of their review “Mostly Positive” or “Mostly Negative”. I’ll allow that some reviews take a strong stance that’s easy to categorize. But what about when a review’s tone is mixed or ambiguous?

Rotten Tomatoes will go so far as to contact some critics and ask them how it should be categorized. When they don’t respond, or when the critic says “you decide,” someone on the Rotten Tomatoes staff has to make the call. Call all the 50/50 reviews one way and you could have a film with a high Fresh rating. If a different staff member with a different sensibility calls it the other way, you get a completely different score. Same movie, same reviews.

Even if a critic has put a star rating on a film, what does that star rating mean? Some people rate films based on their expectations. Some people based on where it fits among all films in the genre, or all films ever made. I might say a film is an excellent movie that most people would enjoy, but I didn’t personally like it. The criteria will vary for each critic.

Forget films for a second. Lets create an aggregate score to show which planets in our solar system are “fresh” or “rotten”.

  • Critic #1 rates each planet based on the amount oxygen in the atmosphere. She gives Earth 10/10 and Mars 1/10.
  • Critic #2 rates each planet based on how red it is. He gives Mars 10/10 and Earth 1/10.
  • Critic #3 rates planets on distance from the sun. Earth is 3/8, Mars is 4/8.

Critic #1 obviously thinks Earth is “fresh”. Critic #2 obviously thinks Earth is “rotten”. Critic #3 it’s a tough call, but it looks like Earth is on the more rotten side of his scale while Mars is exactly in the center. So, depending on how you break a tie, Mars is overall ‘67% Fresh’ or ‘33% Rotten.’ Earth is ‘33% Rotten.’

The point is, whether the planet is “fresh” is arbitrary. Would you rely on this score to predict anything specific about those planets? Nope.

And that’s before taking into account that each Rotten Tomatoes score is assembled from a different group of critics writing at a different cultural moment. Imagine Earth and Mars were reviewed three years apart, and in the intervening time everyone decided the most important factor for evaluating a planet was carbon dioxide, not oxygen. Is Mars really more “fresh”?

A ripe red tomato and a squishy green splat make for great visuals. But they don’t, in fact, represent categories of film opinions.

2. All reviews are not created equal

But let’s say you are interested in a general sense of the critical reaction, rather than whether audiences will be enjoying a film. Surely Rotten Tomatoes, by virtue of surveying so many critics, must give you a broad picture…

Nope. We can safely assume that not all critics will be making up their own minds independently from each other. Other than critics who make a habit of taking controversial positions, the usual aim of human beings is to fit in. There will probably always be ‘clumping’ when it comes to movie opinions.

It may not be intentional. All the critics who grew up admiring Roger Ebert’s reviews will probably unconsciously evaluate films in the same way Roger Ebert did.

A critic who writes for The New Yorker will probably review films from a point of view that is closer to the average reader of that magazine.

Movie critics in general are still overwhelmingly white and overwhelmingly male.

Critics influence other critics. They attend the same screenings and chat with each other before and after.

(To be fair, Rotten Tomatoes does break down their scores between “Top Critics” and “All Critics” if you click in the right places. The division appears to be arbitrary. I can find no criteria on the Rotten Tomatoes website about what separates a Top Critic from the rest. Is it the circulation of the publication they write for? The amount of time they have been a critic?)

Because critics don’t make up their minds simultaneously, you must also take into account when a review is published. Leading up to opening weekend, a film studio may choose not to screen a film for major critics, or only for critics they believe will be sympathetic. Then the sympathetic critics write the first wave of reviews. This is a way to ‘goose’ your initial RT score, potentially influencing the opening weekend headlines (and the next round of critics).

You think the studios don’t take advantage of that? Which leads to the next point…

3. Rotten Tomatoes scores can allegedly be “bought”

I want to be very careful with this. I don’t actually go in for these conspiracy theories where Disney is paying critics to give Marvel movies good reviews and DC movies bad ones. That is premised on the studios caring what critics say more than what audiences spend. (They don’t.)

However, Hollywood studios do have marketing and PR departments and it’s the job of those employees to go out and hype the product. Sometimes that means putting on elaborated “junkets” for film critics and journalists. The critics might get flown to a cool location, put up in a nice hotel, fed free food and allowed some one-on-one access with the movie’s stars. Critics might (quite rightly) suspect that if they write savage reviews after such cozy treatment, they won’t make it onto the list for the next junket.

Even setting junkets aside, PR agents have the ability to control access to the filmmaker interviews and screenings that are the raw material of movie journalism. You can see how the pressures on even fair-minded critics could squeeze them to give the benefit of the doubt here or there. As Upton Sinclair once said, “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

Studios looking to influence the RT score don’t have to beguile the smartest, most-ethical critics. Just the opposite. They can woo the most desperate, venal ones — so called “quote whores”.

The ‘All Critics’ Tomatometer counts a reputable review and a questionable one just the same. Thus it’s in the interest of the studios to promote the careers of movie critics who like pretty much every movie they see. Is it a coincidence that Cars 3 is currently “Rotten” among Top Critics but “Fresh” when you count all critics?

A studio PR person once explained to me how the game works. If they want to boost a Rotten Tomatoes score, they can. They have the tools to do it. It’s nothing diabolical. It’s their job to influence public opinion. But the byproduct is that the idiosyncratic critical voices get lost among squawking of the parrots.

I enjoy seeing movies that take risks and try new things. How will studios be encouraged to make them, and how will we hear about them, if the critical discussion around a film is simplified and compromised?

In Conclusion

The people at Rotten Tomatoes must be aware of the flaws in their methodology. They have taken a few steps to address them and could take many more. But why should they? That is not the business they are in.

They are not some non-profit think tank. They are not some collection of economists who are generating stats that help compare movies across time and space. They are, in fact, part of the movie-promotion machine.

The score they put out is used to hype or dis recent movies, sometimes by the studio PR people when it is to their advantage. Critics are also manipulating it to generate pageviews. When you are one of the few critics to pan a movie, you get a lot of hate-clicks.

My advice: Don’t fall for this imaginary number. In the same way you know an advertisement for a film is manipulated to make the film look good, you know the RT score is also distorted. Ignore the group-think. Ignore the hive. Think of movies in the millions of dimensions that live beyond “rotten” and “fresh”! Make up your own mind!