Costs and (or lack of) benefits of transport projects
Auckland Transport Blog have put up a post today discussing the costs and benefits of the planned transport projects in Auckland that the government is backing. I’ll discuss that in a second, but first there is something I want to get off my back regarding the assessment of transport projects.
BCR ratios (aka CBA by another name)
One thing that has always intrigued me when I hear about transport projects is that rather than talking about the “net benefits” from a “cost benefit analysis” (CBA), they talk about “benefit cost ratios” (BCR). Now the BCR is just the calculated benefits divided the costs (B/C). The surprising thing to an economist is that the BCRs are often < 1.
This means one of two things:
- The projects are money hole and shouldn’t be built (we get less benefits than we put in)
- There are significant non-quantifiable (or difficult to measure) benefits, and bureaucrats have made the judgement call that the non-quantifiable benefits are enough to tip the balance to give a true BCR>1. So there is some objective analysis, and some “gut feel”, but this can’t be avoided.
Hopefully not many fall into the 1 category, but given the scope for politicians and bureaucrats to drive these decisions, I wouldn’t be surprised. Even for the category 2 projects, it’s quite possible we are going to get bad decisions made, as the decision is ultimately going to be subjective. Many projects depend on who wins an election, which I think illustrates the problems here quite nicely.
There is also another, frightening alternative which I will now discuss in the context of Auckland
Auckland’s transport package
The crux of the ATB post is some quotes by Brownlee in question time about the costs of the various projects and a table of the benefits (I’m assuming both are “gross” and not “net”) They correctly note that the benefit figures are in net present value (NPV) terms and the costs are not, so they aren’t directly comparable. However, if we assume the construction time isn’t longer than a couple of years and the NPV is discounted back to around the same period of time, it isn’t two outrageous to compare the two numbers (Using un-discounted costs does of course understate the BCR, depending on whether the benefits need to be discounted further so the start dates are consistent.). Three things jump out
- The projects seem to either have a BCR substantially less than 1 or only moderately greater than 1
- The second harbor crossing costs $4.7b yet only has benefits of $568m
- The East-West link costs $1b and the project has the backing of the government even though the benefits are “yet to be assessed”
As mentioned above, (1) isn’t particularly worrying if you believe that the benefits of transport projects are hard to quantify. On (2), that is a hell of a lot of non-quantifiable benefits to make a 2nd crossing worthwhile. And (3) provides us with the 3rd category that could have been in the first half of this post “projects for which the benefits have not been assessed, yet someone has decided it’s a good idea anyways”. I’ll call this the “entirely gut feel” category. I don’t know about you, but $1b is a lot of tax payer money to spend on something without actually thinking about what the benefits are relative to the costs. (Sidenote: it will make my day if someone can point out that I have misunderstood and this isn’t actually the case!)
Update: as this post went to press everyone has been talking about the Puhoi-to-Wellsford “holiday highway” (Herald and ATB). Looks like no attempt has been made to quantify the benefits, beyond estimates of saved travel time for people going on holiday up north (which could be used to estimate a benefit if you have an estimate of the opportunity cost of people’s time, which if the stereotype of rich Aucklanders heading up north to holiday homes is true , it might be quite high!). This particular Road of National significance (RoNS) suffers from the same problem as above, i.e. having politicians decide to build a road first, then think about whether it is a good idea afterwards. At least this road is going to be tolled. (Sidenote: is the same true for all of the RoNS? i.e. no CBA?)
Update 2: If you are interested in transport funding in NZ, this paper mentioned by detmackey in the comments looks to be great read. I will certainly be giving it a thorough to read to lessen my ignorance on the institutional framework for road funding in NZ
The numbers for Wellington’s roads of national significance are as bad. Seems to be a consistent pattern with all the roads of national significance except Victoria Park tunnel and possibly Waterview.
You can find the BCRs for the RoNS in on the fifth page of this quite good paper: http://igps.victoria.ac.nz/publications/files/30aaf9a4518.pdf
Note that a number of discrete projects with BCRs less than one are often bundled with other projects greater than one so that the overall package is greater than one. This is true of the RoNS as a group, and individually (ie, there are a number of discrete projects in the Wellington Corridor RoNS).
That paper looks great (I’m biased because it is by an economist!), will take a peak.
Sounds like the government is applying bundling theory to road investment!
NZTA lay out their assessment process here: http://www.pikb.co.nz/assessment-framework/
Cheers, that will make some “light” weekend reading for me sometime:)
Good post Mat. Seems to be a pattern under this govt to spend money on roads irrespective of the analysis
Matt’s posts have the kiwi avatar next to them, I’m one of the other authors that lurks in the background:)
Sorry about that Will, I need to pay more attention
haha, no worries, the blog is pretty Matt dominated so I’m not offended! We only recently changed it so that avatars show up next to posts since a few of us are writing more these days:)
Based on what you did today – am looking forward to the next one
Good post Will. Yes you’re broadly right with your intuitions. This is good reading in tandem with the Michael Pickford article, which ‘detmackey’ linked below.
(One thing to bear in mind is that the capex for those big projects is delayed by some number of years, and then construction can take years, so it’s not ok to compare undiscounted costs to discounted benefits. That said, though, it is still grim, as the BCRs are usually under 1.)
I’ve always found it odd that bureaucrats thumb their nose at the BCR, and yet nobody has formally established what exactly is omitted from BCRs that is actually captured adequately by ‘strategic fit’ and ‘effectiveness’.
I’ve often heard senior officials argue that BCRs miss heaps of stuff. Sadly, I’ve noticed that usually the stronger their conviction, the less they know about cost-benefit theory.
“I’ve always found it odd that bureaucrats thumb their nose at the BCR, and yet nobody has formally established what exactly is omitted from BCRs that is actually captured adequately by ‘strategic fit’ and ‘effectiveness’. ”
Spot on.
I cried a little on the inside when I read “strategic fit” and “effectiveness” in Dr Mike’s article…that sounds like a carte blanche to “build roads that will get us elected again”.
I certainly don’t think that CBA/BCR is perfect, but what I would prefer is transparency around what the non-quantifiable benefits for each project are (social/environmental/whatever) and why in their subjective judgement they think it tilts the balance to a BCR>1. As opposed to “BCR is less than one, but it has a good strategic fit”. This may already happen to a large extent, I haven’t read any of the assessments of these projects, but it sounds like the legal multi-criteria test in place doesn’t force this type of explicitness.
And I have a finance background so would never usually compare discounted and non-discounted numbers:) Since I am time constrained/lazy, it appeared to be a vaguely useful back of the envelope calc.
NZTA’s lay out their assessment process here: http://www.pikb.co.nz/assessment-framework/
Not sure that you are on the right track – blaming public servants. Under the last government these types of projects were subject to cost benefit analysis. They are still undertaken, but seem to be ignored. The Bureaucrats haven’t changed much – so its probably something else. (And I suspect to start with that it was a desire to build anything in order to keep the economy ticking over after the GFC, but not sure what sustains the practice of ignoring this type of input to decisions)
How accurate do CBA forecasts projections tend to be, in practice?
Minor point: CBA is the method of collecting the data and organising the information; BCR is a metric produced by a CBA (net benefit is another metric). And yes, you should be worried by BCR<1, especially when other potential uses of public funds have BCRs of 3 or more.
Great post Will. I would note that the private sector can also suffer from vested interests – I can think of 5 multi-billion dollar projects in Australia that went bust recently because the project proponents were able to pull the wool over the equity investors’ eyes.
As to why, here’s a funny-but-true article (applies to both public and private spending). It’s called “Survival of the unfittest – why the worst infrastructure gets built” http://oxrep.oxfordjournals.org/content/25/3/344.short.
This is why I greatly prefer strong monetary policy to fiscal policy if there’s a demand shortage in the macroeconomy. If you weaken your institutional controls on major spending projects, expect to reap what you sow.
Interesting, reading that abstract made me want to laugh and cry – often the sign the of good article!
It did make me wonder about how this interfaces with PPPs and whether they solve any of the issues. I guess you still have the issue that the government is picking the projects, so you don’t necessarily get the “right” projects. Guess I will have to read the article to find out what the potential solutions are since it is only teased in the abstract!
I think PPPs can be good in lots of ways. Aussie state govts have a rep for running a competitive process, lots of tendering so bidders have to sharpen their pencils. Another positive is that a portfolio of infrastructure equity and debt can be quite good thing for pension funds and individual investors, if the equity’s not too levered.
I wonder how they do the BCR analysis for the CRL, as it seems one of those threshold items without which you can never become a top tier city (I can’t think of a great city without metro rail – the closest candidate being LA, but I think its lack of a decent rail system stops it from being in the same tier as New York). I think you would need an Ed Glaeser type analysis to estimate the true increase in productivity. On the other hand, PT is a waste of money unless you simultaneously allow the denser living arrangements that make it viable. I’m not sure Auckland’s completely there yet, although it’s moving in the right direction.
I think your second para is the crux of the debate currently happening in Auckland, and why I have recently become very interested in the debates around PT and density in Auckland. Sounds like that is what Auckland is trying to do, though, and this isn’t an accurate source, my twitter feed suggests that a few important measures for density have been rolled back at the final stage in the last week. I have randomly been out to Botany and Howick a couple of times recently…if that doesn’t convince you density is a good thing then nothing will!
I mostly agree with your point about a proper metro/PT, the cities I have visitied and really enjoyed have generally been those with great PT/proper metro, but that could be a biased perception because PT reliant cities are easier for tourists. Though I think there is a fair amount of cross over with what’s pleasant for tourists and residents.
Based on my understanding of the East West link, you are right, they have not yet assessed the benefits. One reason for this is the East-West link cant even really be called a project yet. It is just a set of options to improve traffic in a general (fairly large) area of Auckland. Most of the options have similar overall strategic goals, but there are certainly significant differences in functionality between them.
So the government has committed to a project that has not even been defined yet. I found this strange at the time.
I worry about relying to heavily on BCRs mainly because they inevitably fail to capture all the benefits – my background is financial modelling and with the best will in the world it is very difficult to capture everything that may influence the future – the aim is to capture the most relevant and leave it at that.
However, as roads are long lived assets, the aim should really be to widen the net as far as possible. I live in Auckland and am a regular traveller into Northland. The Puhoi – Wellsford upgrade has been demonised as the ‘holiday highway’ but to the people of Northland it can’t come soon enough. Why?
The current road is a narrow two lane pig with limited passing opportunities. It has speed reductions, a number of very nasty curves and has a very significant bottleneck in Warkworth. The traffic on the road is significant given its current design and structure.
So one of the benefits is quicker travelling times – that’s important when you have a significant bottleneck through Warkworth both north and south. It will remove all the difficult curves and provide a safe four lane expressway.
So what are the wider benefits? Firstly, it will make it easier for Warkworth to expand to the west – currently SH1 cuts the town in two. Secondly, I’m sure you are all aware of Auckland’s house prices – by lowering travelling times and developing a safer route it makes Warkworth a more viable satellite suburb for Auckland and in particular the North Shore – so over time the benefits include increasing population around Warkworth and therefore increasing patronage on the road.
It will also improve the quality of life for residents in Warkworth who will no longer have major queues, noise and the other delights of traffic rumbling clear through the middle of their town. And many of the people living in Warkworth and environs are elderly… so better transport links means better and faster access to hospital and medical care in Auckland (North Shore).
Now these points I am making are at the ‘blue sky’ end of things, but they are nevertheless real. They are hard to quantify onto a spreadsheet and probably represent a ‘fat tail’ in a normal distribution – value to the community but hard to put a number on… the usual response is to ignore them… but ignoring them for the tidiness of a BCR calc doesn’t mean they don’t exist…