You probably went to bed on the 26th hearing that New York City was going to get buried in snow.
You probably woke up on the 27th and learned that it was Boston and New England east of New York that got buried.
One problem is that forecasters went with what they were comfortable with rather than what was most probable.
There was plenty of evidence the #blizzardof2015 wouldn’t be record-setting in New York. http://t.co/ovoCRvn6aa pic.twitter.com/eLNllFx4HH
— FiveThirtyEight (@FiveThirtyEight) January 28, 2015
Additionally, lots of computing power goes into weather forecasting, and even 24- and 12-hours in advance of events, there are still very meaningful error bars on predictions. Uncertainty has been a problem for media forecasters who have historically been too confident in predicting precipitation events. A study of TV meteorologists in Kansas City found that when they predicted with 100 percent certainty that it would rain, it didn’t one-third of the time. Forecasters typically communicate margin of error by giving a range of outcomes (10 to 12 inches of snow, for example). In this instance, I don’t think the range adequately showed the disagreement among the models. Perhaps a probabilistic forecast is better.We have our own local problem with forecasting. And here too forecasters are riding what's comfortable rather than what's probable.
 |
| Have we exaggerated likely traffic in 2031? |
Planners, policy makers, and electeds are all making decisions on a single traffic forecast pushed very far out into the future. We should know more about how reliable it is likely to be.
 |
| Trend-line mania: 61 out of 61 projections were too high. Also, the same slope on the trend line always! |
Planners may think the perspective here is equally wrong in the other direction. Fine. But they must admit that their modeling inputs now assume a high level of growth in driving.
And it is no longer reasonable to assume only this; and conversely it is no longer unreasonable to perform similar modeling with assumptions for low and even declining levels of driving.
Washington State is in fact making their primary forecast with a declining level, and the Feds have revised theirs and moved to a forecast with low growth instead of higher growth.
We are out of step with our neighbors to the north, and we are out of step with the Federal Highway Administration. Shouldn't we get with the program?
 |
| via Washington State Ferries Transportation Revenue Forecast, October 2014 (Sightline link broken) |
Even if you think the low projection is unlikely, any statistician would tell you now that it has a substantial non-zero chance of being closer to reality than the high projection.
Don't be like bad weather forecasters, ok?
5 comments:
This is a huge issue.
As someone who once worked around the clock for weeks at a time to turn uncertain fuzzy data sets into understanding and prediction, I can tell you that traffic projections are nothing but politics and have absolutely zero to do with trying to make reasonable estimates about the future.
anyone who has done modeling for any length of time has the ability to go back and determine for themselves their own bias, and integrate that into future projections. But if scouts and oh.where to do that, they would have to admit that in their business, the only thing that matters for them is being able to draw a straight line from a previous assumption to a foregone conclusion him.
Traffic projections is like political redistricting – the urge to gerrymander is overwhelming to the political actors who do the job. The only solution is to take the job away from them, and crowd source The task to anyone who cares to take it up, and allow contestants to submit their models and results to an objective, independent organization that has no financial stake in the outcome.
As the saying goes, war is too important to be left to the generals, and traffic projections are way too important to be left to Department of Transportation.
I see that Salem has traffic counters on some roads. I've only seen them around the Grant neighborhood but it might be a bigger study. I hope they use the data to find ways for people to drive less and not increase road capacity.
Here's a link to the City's traffic counts.
It's interesting that if you go (via the Sightline link) to the last page of the Washington State report (which is primarily a revenue forecast report that includes many factors other than VMT forecasts to predict revenue), their equation uses an assumption of continually increasing gas prices ($3.50/gallon in 2015, $4.50 a gallon in 2028, and on). It goes to show that even short-term forecasts can be dramatically wrong.
Also, I wonder if the VMT forecasts are only for vehicles that pay for gas, i.e. are they assuming a increasing share of personal vehicles will be electric and therefore not contributing to revenue? It would take additional research to answer that question.
(Sightline VMT graphic broken, so updated with link to WashDOT source document.)
Post a Comment