The challenge rule sucks. In football, and anywhere else it might be applied.
There should be an official (or maybe a team of officials, for objectivity's sake) watching the game on TV like most of us. When there is an obviously blown call (which TV has an uncanny knack for showing us, repeatedly...) they should call the lead official and say "hey, y'all blew that one...here's what we saw".
No red flags, no booth calls in the last 2 minutes, just common sense "here's one you missed" review of what happens on the court/field of play.