If there’s one defining feature for the annual NCAA college basketball tournament, it’s unpredictability. Upsets occur on the regular during March Madness. This year, Microsoft is partnering with the NCAA to spot those upsets before the players hit the court.
The partnership will put Bing Predicts—a prediction engine that uses machine-learned models to analyze and detect trends from the Web, social activity, and external data—in the roll of bracketologist, analyzing match ups and offering its take on the eventual winner.
Walter Sun, principal applied science manager at Bing, told the Daily Dot that Bing Predicts engine is getting some tweaks from last year to improve its analysis. According to Sun, Bing Predicts will “synthesize over a decade of exclusive tournament data including team and player stats, tournament history, schedule information, trends and more.”
Along with the historical data from teams and the tournament, Bing Predicts will also process information gathered by the search engine arm of Bing. Sun explained, “Data from social signals allows Bing’s advanced predictions technology to calculate a confidence percentage and make a case for what needs to happen for either team to win or lose.”
That might make it sound like Bing is a bit of bandwagon fan, jumping on board with whatever team people are talking about, but the system seemed to work out OK last year. “Our predictions for [the NCAA tournament] were in the upper 30 percent,” Sun said.
On the Bing Predicts site, it claims to have been 73 percent accurate during last year’s tournament, but that’s a pretty generous assessment of the actual results. Once the field was narrowed after the First Four games, Bing did accurately pick the winner of 47 out of 64 games—enough for the claimed 73 percent success rate.
But Bing Predicts wasn’t playing by the same rules as everyone else to get to that score. It analyzed each game individually to arrive at that figure, revising the winner based on who actually advanced. In the bracket that it filled out like everyone else, picking every game—including the First Four—before the tournament begins, Bing Predicts scored 41 out of 67 games correctly. That’s good for a 61 percent success rate.
It wasn’t a score near high enough to put Bing in the upper echelon of human pickers, but that’s a bit of an unfair comparison. Humans are just trying to get this year right and might have a completely different approach next time; Bing is trying to fine tune its engine so it can consistently generate odds of success for every game, every year.
Still, last year was a good year for Bing to make its entrance into the tournament pool, and it failed to make a splash. 2015 was a top-heavy year for the brackets. Kentucky entered March Madness undefeated, and according to the Simple Rating System (SRS), the tournament had some of the strongest top-seeded teams since the event expanded to 64 teams in 1985.
When the brackets have dominant teams at the top, it becomes more likely for a “chalk” outcome—the team with the better seeding tends to win. At one point during the tournament, the better seed won 23 consecutive games.
That’s not to say there weren’t upsets. When 14 seeds University of Alabama at Birmingham and Georgia State beat 3 seeds Iowa State and Baylor in the first round, more than 99 percent of people had their shot at a perfect bracket ruined. But it was, to some degree, an “easy” year.
This year, the choices for Bing Predicts and everyone else filling out a bracket won’t be quite as clear. The NCAA has been full of parity this year. There is no dominant team or clear top tier, meaning teams sliding into the mid and lower seeds have a better chance to provide some surprises.
Sun said that poses no problem for the Bing Predicts engine. Instead, he sees it as an opportunity to prove the system’s worth. “Greater parity allows us to provide insights on more close match-ups which might not be as obvious,” he explained.
At the professional ranks, the NBA and WNBA use a series system that has teams playing a best-of-seven game contest to determine who advances. This almost always leads to the better team winning. According to SportingCharts, the worse seeded team has only a 16.7 percent chance of winning a series in seven games.
By contrast, every game in the NCAA Tournament is winner-take-all. To add to the variation, most of the match-ups are against teams that are from different regions and conferences, meaning there’s next to no familiarity with their players and strategies. If each round was a best-of-seven series, trends would begin to emerge, teams would adjust, and it would be much more likely the better seeded team would advance.
https://www.youtube.com/watch?v=Pes91XdfJsM
Odds are good that no model is ever going to accurately predict the entirety of the NCAA Tournament—which is the point of the format in the first place. The best team doesn’t always win.
There’s a case to be made that the NCAA Tournament model produces a worse product in the end. Take the 2014 Championship game between the seven seed Connecticut and eight seed Kentucky. The three best players in the country and top three draft picks in that year’s NBA draft—Jabari Parker, Andrew Wiggins, and Joel Emiid—were bounced from the tournament early when their teams suffered upsets in the early rounds.
As much as everyone likes an upset in the moment, when it comes to the finals, most people want the best teams possible facing off. That didn’t happen, and viewership of the 2014 title game dropped by over 2 million from the prior year’s match-up.
Bing’s prediction engine will likely get better this year, even if it doesn’t get as many picks right. The more data it has, the better it can parse the statistics that indicate advantages for each team and calculate the likely winner. But it’s always going to miss some games.
There are over 9.2 quintillion possible combinations for a 64-team bracket. With some basic knowledge of the tournament—knowing a 16 seed has never beaten a one seed, for example—DePaul mathematician Jeffrey Bergen estimates the odds of picking a perfect bracket to be low as one in 128 billion. Not even the finest-tuned algorithm is likely to beat those odds.
H/T GeekWire | Photo via Chad Cooper/Flickr (CC BY 2.0)