|
|
We recently conducted a phone survey and found a 100% correlation between people we called and people who have access to phones. Research also indicates that (a) the number of four-masted sailing ships carrying pirates has declined in every year since daylight savings time was instituted, (b) 4 out of 5 doctors surveyed recommend Crest, and (c) 74% of all statistics quoted in articles like this are made up on the spot.
Yes, there’s a great deal of fun to be had with questionable numbers—aka “lies, damn lies, and statistics.” And yet, when you look at what appears to be statistical jibber jabber, you can sometimes find a nugget of actionable truth that turns out to be rather important. Such is the case with a recent study from HP Labs that claims to prove that Twitter chatter can provide a reliable indicator of how well a movie will do at the box office.
Unlike the fifty-kazillion people who tweeted about the report when it came out, we actually read it. And one issue pops up pretty quickly: the predictive value is pretty much limited to Twitter traffic during the week prior to, and two weeks after, a movie’s release. This means that, no, this is not the magic 8-ball studio marketers are desperately searching for. By the week of release, to borrow a phrase from Ross Perot, the barn doors are open and horses are out of the stable. (In other words, you’ve already spent 99% of your $500 million on Avatar.) The results also work only for wide releases, so the model doesn’t apply to indies and slower-building limited releases, which are precisely the sorts of films that would benefit most from better modeling.
So, is Twitter a useful box office prediction tool? Within the above limits, yes. First off, the findings seem legit, coming from HP Labs eggheads without a specific conflict of interest. Second, the data pretty clearly shows that (a) a simple model built from the rate at which tweets are created a can outperform market-based and news-based predictors, and (b) tracking sentiment within Twitter can further improve the model’s forecasting power, but only in the post-release period.
We spoke to one longtime industry insider and great skeptic, who said “This may be true for certain genre product—you know, like Kick Ass—but for family films like The Blind Side or The Spy Next Door, not so much.” Well, turns out this demo-based prejudice (“only geeks use Twitter”) doesn’t hold water: the model holds up just as well for both The Blind Side and The Spy Next Door, which were analyzed in the study. What’s more, the sentiment filtering tracked beautifully with real life. After a decent opening weekend ($34M), The Blind Side saw an enormous increase in positive sentiment on Twitter, which was matched by its box office jump the next week (to $40M) and subsequent long run. Twilight: New Moon saw the opposite effect. Released the same weekend as Blind Side, it posted astounding numbers, but Twitter sentiment plummeted, followed by box-office revenue ($142M to $42M). The statistical correlations are impressive.
Here’s another juicy tidbit: The amount of pre-release publicity materials in the system doesn’t offer much predictive punch. Turns out, the majority of tweets in the pre-release week have to do with links to, and retweets of, publicity materials (photos, videos, blogs, news), while those posted after the release focus on positive and negative sentiments. While there is some correlation between those asset-related tweets and performance, the study found it to be “quite low… surprising since we would expect promotional material to contribute significantly to a movie’s box-office income.”
Key takeaways? It’s still hard to prove the value of pre-release hype. The ratio of positive-to-negative tweets offers a better indicator, at least following opening weekend. But ultimately, no measure proves as significant as the rate of tweets themselves. Which is to say, having a film that moves enough people to spend 20 seconds tweeting about it after they’ve seen it? Priceless.
|
