How will Utah Jazz’s Quin Snyder and others use the NBA’s new coaches challenge rule?

There’s no red flag. Instead, there’s a green light.

But inspired by football’s coach’s challenge rule, the NBA has instituted a similar process for the 2019-20 season. During all 48 minutes, a coach can challenge a called foul on the court. Except for in the last two minutes of a game (when those plays are reviewed anyway), a coach can also challenge an out of bounds call or a goaltending. As in the NFL, the visual evidence has to be clear and conclusive for a call to be overturned.

To signal for a challenge, the coach has to call a timeout, then tell the referees he’d like a challenge by swirling his finger in the air. This illuminates a green light on the scorers table.

The biggest difference between the NBA and NFL rules? In the NBA, regardless of whether or not the coach wins or loses the challenge, coaches have just one challenge all game long. And win or lose, the timeout is gone.

With just one shot at it, choosing when exactly to challenge is not just a matter of a coach using it when he thinks a call is wrong. If a coach uses it early, he won’t have it available late in the game. If he waits, and the game isn’t close — or if he’s out of timeouts — he may not get to use it at all. If he uses it on a two-point foul, a bad three-point call might be around the corner. If he uses it to contest a foul that sent a poor free-throw shooter to the line, it may not make sense. A coach may want to keep his timeouts rather than risk a challenge.

It is, in other words, the kind of “What If” game that has potential to keep coaches up at night. Jazz coach Quin Snyder says it already has — though that’s not unusual, he noted.

“There’s gonna be opportunities in a very visible way to analyze those situations,” Snyder said. “We’ll analyze the heck out of them.”

Snyder credited the work of Cory Jez, Jazz coordinator of basketball analytics, in helping him and his staff understand when the best scenarios are to make that challenge. That work isn’t publicly available: obviously, the Jazz want to use challenges more effectively than the opposition, given that a well-timed challenge might be the difference between a win and a loss.

But some more public work done might help point toward what Jez and his team have likely found. For one, data from the G-League, where this rule has been in place for three seasons now, shows that coaches are worse at successfully challenging plays than you might expect. In both 2016-17 and 2017-18, overturn rates hovered at only 32%, according to 2 Ways and 10 Days, a site that covers the G-League. That being said, part of the G-League’s problem was lack of conclusive camera angles — many G-League games only have one or two cameras — whereas NBA games have much more to work with here. The 2 Ways and 10 Days site estimated that up to 30-40% of calls were rejected due to inconclusive evidence.

A more recent study, conducted in September by two university students at the NBA’s annual Hackathon gives us even more information. C.B. Garrett and Peter Zanca, from Syracuse and Notre Dame, studied NBA play-by-play data, as well as the G-League success rates of each type of reversible call, to figure out which calls are the best bets for coaches to challenge.

The best play to challenge? Basket interference. Incorrect basket interferences were overturned about 47% of the time. Since those are always worth two points, using a challenge on a close one of those is a great idea. Goaltending was overturned a bit less, 38% of the time, but has an additional drawback: if a goaltending is overturned, a jump ball takes place to figure out who gets the next possession. So it’s about half as valuable as a basket interference overturn.

Out-of-bounds calls were overturned about 36% of the time in the G-League, a number that’s easy to imagine jumping in the NBA, where more camera angles are available. But how valuable an overturn is depends on the shot clock: if there are six seconds or less left on the shot clock, the remaining possession isn’t worth much anyway, only about 0.6 points on average. But if there’s a lot of time left, an incorrect out-of-bounds call might be a good one to challenge.

Fouls are the exciting ones to challenge, but G-League coaches hardly ever got those right. On 2-point shooting fouls, referees only overturned 14% of them, and 3-point fouls only saw 11% overturned. If they’re overturned, the foul turns into a jump-ball, too. And naturally, it’s smarter to challenge fouls if a good free-throw shooter is going to the line.

Notice that the calls with high overturn rates — interference, goaltending and out-of-bounds — are already being reviewed in the last two minutes anyway. So it makes sense that an analyst apparently told L.A. Clippers coach Doc Rivers to be ready to use it at any point during the game.

“Analytically, they said we should use it anytime,” Rivers said. “I don’t agree with that. We’ll see how it goes.”

Even if that’s the right approach analytically, for coaches, it never feels good to be holding an empty bag at the end of the game. Or worse, twirling a useless finger.