In their guidelines for fighting bid-rigging in public procurement, the Organization for Economic Cooperation and Development (OECD) (2009, 2–3) mentions as their first two criteria items that relate to the removal of restricted tendering:
Small number of companies. Bid-rigging is more likely to occur when a small number of companies supply the good or service. The fewer the number of sellers, the easier it is for them to reach an agreement on how to rig bids.
Little or no entry. When few businesses have recently entered or are likely to enter a market because it is costly, hard or slow to enter, firms in that market are protected from the competitive pressure of potential new entrants. The protective barrier helps support bid-rigging efforts.
The OECD recommendation of the OECD council on fighting bid-rigging in public procurement clearly recognizes the wide range of benefits that arise through the role of competition fostered by open tendering. They state:
RECOGNISING that, in public procurement, competition promotes efficiency, helping to ensure that goods and services offered to public entities more closely match their preferences, producing benefits such as lower prices, improved quality, increased innovation, higher productivity and, more generally, “value for money” to the benefit of end consumers, users of public services and taxpayers;
RECOMMENDS that Members assess the various features of their public procurement laws and practices and their impact on the likelihood of collusion between bidders. Members should strive for public procurement tenders at all levels of government that are designed to promote more effective competition and to reduce the risk of bid rigging while ensuring overall value for money. (OECD 2012, 2)
They specifically recommend the promotion of “competition by maximising participation of potential bidders by . . . establishing participation requirements that are transparent, non-discriminatory, and that do not unreasonably limit competition.”
Bid-rigging, its prevalence when competition is restricted, and its negative effects have been documented in a wide range of academic studies including Robert Porter and J. Douglas Zona (1993) and Srabana Gupta (2001), who also review numerous other studies. Weishaar (2013) provides more current international evidence as well as reviews of numerous other studies.
Econometric Methodologies for Estimating Cause-and-Effect Relationships
As a first step in outlining potential methodologies for estimating the effect of restricted tendering on construction costs, it is useful to outline the methodologies that have been advanced in econometrics for estimating the causal relationship between an intervention or treatment (restrictive tendering in this case) and outcomes (cost of contracts in this case). The emphasis in these methodologies is to estimate the extent to which the relationship is causal (i.e., restrictive tendering causes an increase in costs) rather than simply correlational or associative. A variety of econometric methods have been used to estimate underlying causal relationships. Understanding these methodologies may trigger ideas that can be useful for designing methodologies for estimating the cost implications of restrictive tendering.
Establishing the underlying causal relationship is important for policy purposes so that policy interventions (e.g., removing the restrictions on bidding) can deal with the causes and not just the symptoms. If the relationship is simply correlational, then removing the restrictions may not lead to lower costs. Establishing the underlying causal relationship is also important for predicting the future. Higher construction costs may not continue in the future unless the underlying cause (restrictive tendering) continues. Note that this assumes a differentiation between economic outcomes (i.e. the costs of contracts) and other non-economic outcomes which, in their own right, might be legitimate rationale for policy change. As Ray Fisman and Tim Sullivan note, “democracy wasn’t designed to be as smooth, as fast, as profitable, or as efficient as possible.”6 Issues of democratic responsibility, respect for and protection of freedom of association are matters that are at least equally, if not more, important for government policy. The purpose of this paper is to examine one factor – economic outcomes. It is the responsibility of public official holders to appropriately understand these competing factors in pursuit of responsible public governance , and indeed, even in the highly unlikely case that empirical evidence on closed tendering pointed to lower costs, it might still be incumbent upon government to open tendering for democratic reasons.
Random Assignment
Randomly assigning units to a treatment group and others to a control group, as is common in medical trials, is generally considered the gold standard for estimating the effect of a treatment on outcomes. If the assignment is truly random, then the effect of the intervention is simply the mean difference in outcomes between the treatment group and the comparison group.
Randomly assigning restricted or open tenders is not feasible. Nevertheless, this procedure can serve as a benchmark for judging other procedures that may approximate random assignment.
Natural Experiments
Natural experiments involve what could be considered an “act of nature” that approximates randomly assigning some to a treatment group (e.g., restricted tendering) and others to a control group (open tendering). The act of nature is simply some event that is generally regarded as exogenous in the assignment to the treatment or intervention. If, for example, restricted tendering were the norm, but some open tendering occurred “by accident,” then this could be considered the equivalent of random assignment, and mean cost differences could be compared. It is not obvious, however, that such “accidents” have occurred whereby some construction tenders are restricted while others are open.
Comparisons across contiguous regions are sometimes used as natural experiments, if the regions are otherwise similar but differ mainly in the extent of the treatment or intervention. Other differences can be controlled for by the use of regression analysis. Where tenders are restricted in one region but open in a contiguous region will have to be explored in greater detail.
Before-and-After Comparisons
Before-and-after comparisons are often used to estimate the effect of a policy change. In this case, it would involve a change from restricted tendering to open tendering or from open tendering to restrictive tendering. If no other factors are changing to influence outcomes, then comparisons of mean outcomes in the “after” period minus the “before” period provides an estimate of the effect of the policy change. In effect, the “before” period serves as the control group. If other variables are changing that can affect the outcomes, then they can be included as explanatory control variables. The estimates are more reliable if longer periods around the policy change are available, although the longer the periods the more likely other factors are changing.
To the extent that Ontario has had periods of time when restrictive tendering was in place and other periods when it was not in place, then such before-and-after comparisons of construction costs could be made. The cost estimates, however, would have to be for uniform or standardized construction projects to control for the possibility that the product itself was changing over time. The estimates would also have to control for other factors that could affect construction costs over time such as technological change or changes in the cost of other inputs.
Difference-in-Difference Estimates
A better comparison can be made if, in the before-and-after period, a control group is also included that is not affected by the event or treatment at all. Any change in their behaviour around the event will be picking up and controlling for the effect of other factors that may also be changing and that would otherwise contaminate the before-and-after comparisons. Their change in behaviour or outcomes can then be subtracted from the treatment group to get the net or causal effect of the treatment, purged of the effect of contaminating factors that are affecting the control group. That is, the difference in the before-andafter behaviour of the control group is subtracted from the difference in the before-and-after behaviour of the treatment group—hence the phrase difference-in-difference. Regression analysis can also be used to control for other variables that may change differently across the treatment and control groups.
It is desirable if there are common trends in the outcome measure (e.g., construction costs) between the treatment and comparison group prior to treatment. It is also desirable if the treatment and control groups have similar values of the control variables (i.e., common support) prior to the treatment or policy intervention.
For estimating the effect of restrictive tendering on construction projects in Ontario, the difference-indifference procedure essentially requires information on standardized construction costs before-and-after the introduction of restrictive tendering compared to open tendering, or vice versa, as well as standardized construction costs in a jurisdiction or industry where open tendering prevailed throughout the same time period. It would also require information on other factors that can affect changes in construction cost so as to control for the effect of changes in such factors.
Regression Analysis
If some contracts in construction involved restricted tendering and others involved open tendering, then an approximation to the cost effect of restricted tendering could be estimated by simply comparing the average standardized construction costs in the two regimes. The product or services would have to be homogenous or converted to homogenous units. Regression analysis could be used with a dummy variable coded one for restricted contracts and zero for open contracts. Control variables could be added to control for the effect of other factors that may influence the costs, provided that data is available on those factors. The regression coefficient on the restricted tendering dummy would give the effect on costs of restricted as opposed to open tendering.
This procedure essentially requires information on whether the tendering was restrictive or open as well as the cost of a homogenous or standard construction project and information on other variables that may affect construction costs.
A problem with this procedure is that the treatment, in this case restrictive tendering, may not be exogenously determined (i.e., akin to being random). The decision to allow a contract to be subject to restrictive as opposed to open tendering may be based on other unobservable factors that cannot be controlled for in any regression analysis, but that nevertheless affects the cost outcome. For example, restricted contracting may be used in situations where the contract is risky in the sense that it is difficult for the contractor to make an appropriate bid—they may overestimate or underestimate by substantial amounts. In such circumstances, all bids may contain a risk premium. Any higher cost for restricted contracts could reflect this risk premium, not an excess payment because of the restrictive tendering.
Instrumental Variable Analysis and Selection Correction Procedures
Instrumental variable analysis is designed to deal with this problem. It essentially involves a two-stage procedure where the first stage involves estimating the probability of being in the treatment group (e.g., restrictive tendering) compared to the control group (e.g., open tendering). This would require finding a variable or variables that affect the probability of receiving the treatment (e.g., restrictive tendering) but that do not affect the outcome (e.g., construction costs). In the treatment effect literature, such variables (termed exclusion restrictions) are generally extremely difficult to find, and the analysis often flounders on this difficulty.
If such a first-stage estimate turns out to be feasible, then the predicted value from this estimate is included in a second-stage outcome equation and estimated by two-stage least squares. The resultant coefficient from this estimate gives the causal effect of the treatment (e.g., restrictive tendering) on the outcome (e.g., construction costs).
A variant of this procedure is the Heckman two-step procedure. It involves a similar estimate of the first-stage equation. The predicted value of this is then used to construct a sample selection correction term (inverse Mills ratio) that is included in the second-stage outcome equation. The coefficient on the restricted-versus-open-tendering dummy variable would give the cost effect of restricted as opposed to open tendering after controlling for the effect of other variables that can affect costs, including unobservable factors that may influence the choice of restricted versus open tendering.
Separate Treatment and Control Equations
The regression procedure described above with a dummy variable for restricted contracts versus open contracts assumes that the other control variables have the same effect on the outcome (e.g., costs) for restricted contracts versus open contracts.
As an alternative or complementary procedure, separate regressions can be run on the restricted-contracts sample and the open-contracts sample. The mean difference in the outcomes (e.g., costs) can then be decomposed into two component parts. The first is differences in the characteristics (explanatory variables) that affect the outcomes. The second component is differences in the outcomes, costs in this case (regression coefficients), that are associated with the different characteristics.
In the case of construction contracts, this would seem useful mainly if one wanted to drill deeper and explain the cost difference between winning contracts for standardized construction projects under restricted versus open tendering. If the purpose is just to compare costs between winning contracts for standardized construction projects under restricted-versus-open tendering, then the regression analysis with a simple dummy variable for restricted versus open tendering would be adequate.
Regression Discontinuity Procedures
Another methodology for approximating random assignment is to use regression discontinuity (RD) procedures. The RD design basically requires a cutoff score such that those just above the score get the treatment (e.g., are awarded the contract) while those just below the score do not get the treatment (e.g., are not awarded the contract) and are in the control group. Such individuals just below and just above the cutoff are so close to each other in terms of getting the contract that being awarded the contract can be considered the luck of the draw (i.e., randomly assigned to the treatment). Such a procedure requires a large number of observations around the cutoff point. Observations away from the cutoff can be included with lower weights attached to them, but this comes at the expense of starting to compare “apples and oranges.” Control variables can be added to control for other factors that may affect the outcomes.
Such a procedure would not seem viable for construction projects since there are typically only a few bidders, with only one winning bid. There is simply not a large number of bids with some that just won and others that just lost, and with a clustering of bids around the won-lost cutoff.
Propensity Score and Other Matching Procedures
Propensity score and other matching procedures have been used to control for unobservable factors that can affect outcomes between two regimes (e.g., restricted versus open tendering) and that cannot be controlled for by including as control variables in a regression context.
The procedure basically involves estimating the probability of being in a treatment regime (e.g., restricted tendering) versus a control or comparison group regime (e.g., open tendering). The predicted probability of being in the treatment regime (e.g., restricted tendering) is then estimated (i.e., the propensity score). Those observations in the treatment regime (e.g., restricted tendering) are then matched to observations in the control group (e.g., open tendering) that have the same or similar probabilities (propensity score). Mean differences in the outcomes (e.g., costs) are then compared. They are considered causal estimates since both the treatment and comparison groups have the same probability of being treated; the treated groups just happened to receive treatment (by random chance). The assumption underlying this analysis is that selection on the observables (the variables that affect being in the treatment versus the control group) controls for selection on the unobservables. Such an analysis requires data on contracts that were awarded under restrictive tendering and those awarded under open tendering, for the same homogenous or standardized construction projects.
Synthetic Control Method
The synthetic control method is a matching procedure that basically involves synthetically or artificially finding a comparison group from a “donor pool” of potential comparison groups that most closely resembles the treatment group (in terms of covariates that can affect the outcome) in the time period prior to the treatment. The pre-treatment variables can also include the outcome measure in the period prior to the treatment. The unit of observation is typically a country, region, state or firm, with the synthetic comparison group constructed from the donor pool of other countries, regions, states or firms that did not implement restrictive tendering. The difference between the outcomes between the treatment group and the synthetic comparison group before the treatment and after the treatment (i.e., a difference-in-difference type comparison) is then made to yield the treatment effect. Essentially this procedure refines the difference-in-difference procedure by restricting the comparison group to a set of observations that most closely resembles the treatment group in terms of factors that can affect the outcomes.
As applied to the issue of estimating the effect of restricted tendering on contract costs, jurisdictions (e.g., cities or municipalities) that implemented restrictive tendering would form the treatment group. The change in their contract costs after the implementation of restrictive tendering would then be compared to a synthetic comparison group of other jurisdictions that did not implement restrictive tendering. That synthetic comparison group would be selected from the donor pool of all jurisdictions on the basis of most resembling the treatment group in terms of factors that can affect the cost of contracts, including the outcome (cost of contracts) in the period prior to the implementation of restrictive contracts in the treatment group.
Other Methodologies for Estimating Cost of Restrictive Tendering
The econometric methodologies for estimating the causal effect of restricted versus open tendering tend to have data requirements that are not likely to prevail in the construction contracting area. The previous discussion of the literature often highlights points that have implications for methodologies for estimating the cost of restrictive tendering in public construction projects in Ontario. Potential methodologies are outlined in this section with the intent of providing information for narrowing down such methodologies and highlighting additional information that would be needed to arrive at cost estimates.
Simulations Based on the Union Wage Impact and Ratio of Labour Cost to Total Cost
An estimate of the cost effect of restrictive tendering could be made if it is reasonable to assume that restrictive tendering involves the union wage rate and open tendering involves the non-union wage rate. Adjustments could be made for the fact that if tendering were open then unionized contractors would obviously still bid, but they would likely have to bid closer to the non-union wage rate to compete with the non-union contractors. How close to the non-union wage they would have to bid is an open question.
A decision would also have to be made as to whether it is appropriate to use the gross union-nonunion wage differential in ICI construction, which is simply the difference between the average union-nonunion wage in ICI construction, or the net union-nonunion wage differential after using regression analysis to control for the effect of other factors that affect that differential (e.g., age, education, training etc). The net differential is generally regarded as the pure union wage effect. However, non-union contractors are likely not only to not pay the union wage but also to use a different mix of workers.Providing that the mix was appropriate for meeting the requirements of the contract, then the gross union-non union wage differential would be appropriate to use. Perhaps a reasonable strategy would be to use the net union-nonunion wage differential as a lower bound and the gross differential as an upper bound of the cost differential.
Fang and Verma (2002) estimate the net union–non-union wage differential in construction in Canada as 19 percent after using regression analysis to control for other factors. Their estimate is for the construction industry in general and not for ICI separate. They also do not report a gross wage differential. However, based on a special request we made to Fang, he estimated the gross differential of 22.1 percent, similar to the net differential of 19 percent, suggesting that the cost implications will not be sensitive to the use of a gross or net union wage premium in construction.
Using the union–non-union wage differential as an estimate of the wage cost difference between restricted tendering (union wages) and open tendering (non-union wages) would imply that the labour cost saving is between 19 and 22 percent depending on whether the net or gross union wage premium is used. O’Grady, Armstrong, and Chaykowski (2006, 37) estimate that labour costs are about 33 percent of total contract costs in construction in Canada. This would imply that the contract cost saving would be between 6.3 and 7.3 percent depending on whether the net or gross union wage premium were used. This difference between labour cost and contract cost savings based on the share of labour cost to total cost in construction is one reason why reports of cost saving can vary so much—they are sometimes expressed as a percent of labour cost and sometimes of contract cost.
If this procedure is followed, then a wide range of adjustments and judgment calls would have to be made. Updated estimates of the union–non-union wage differential in ICI construction in Ontario would be necessary, preferably on both gross and net bases, the latter requiring regression estimates to control for the effect of other variables that can affect wages. Estimates from the literature could also be used, but it is not clear how relevant they are for ICI in construction in Ontario. As indicated earlier, estimates from the United States place the pure union wage premium in construction more in the range of 40–50 percent (Bilginsoy 2013; Blanchflower and Bryson 2004; Bratsberg and Ragan 2002; and Linneman, Wachter, and Carter 1990), although more moderate estimates for the United States of 26 percent are given in Bloch (2002, 287) and 26 percent in Kessler and Katz (2001, 271). O’Grady, Armstrong, and Chaykowski (2006, 37) estimate the union pay premium in construction to be 34 percent in Canada.
In addition to estimates of the union–non-union pay differential in ICI construction in Ontario, it would be necessary to have estimates of the share of labour cost to total cost in government construction contracts in Ontario. As indicated previously, an estimate of 33 percent was provided in O’Grady, Armstrong, and Chaykowski (2006). Based on different US studies, Lyons (1998, 82) reports wage costs as a share of total construction costs as 25–40 percent in one study, 30–40 percent in another study, 33.3 percent in another, and 33.6 percent in a fourth study.
Since the results depend on the union–non-union pay differential and the share of labour cost to total cost in government construction projects in Ontario, and a range of these estimates are likely to prevail, a sensitivity analysis could be portrayed. A grid could be constructed with the union–non-union pay gap on one axes and the share of labour to total cost in the other axis. The cell entries would indicate the additional costs of paying the union rate as opposed to the non-union rate for each assumed ratio of labour cost to total cost. For example, the cell entry for a union wage premium of 30 percent and an assumed ratio of labour cost to total cost of 33 percent would imply an additional contract cost of 10 percent. The grid would give the likely range of cost estimates depending on the union wage premium and the ratio of labour cost to total costs. Preferred estimates could be highlighted on the grid.
A version of this procedure was used by Max Lyons (1998) when he estimated the cost of US federal Project Labour Agreements that required paying union wages on large construction projects. He estimated union wages to be 19–24 percent higher than the alternative wage that would have been paid, which was the Davis-Bacon wage. For labour cost as a share of total construction cost he used a figure of 33.6 based on National Income and Product Accounts estimate of labour’s factor share of the value of production for the construction industry. This would yield an estimate of the additional cost of the PLA over and above the Davis-Bacon wage of 6.4 to 8.1 percent based on the union premium of 19 or 24 percent respectively.
Simulations from Models That Relate Costs to the Number of Bids
As discussed previously, there is an extensive literature that highlights that the costs of contracts falls as the number of bids increase. To the extent that this relationship can be established and deemed relevant to the situation for Ontario, then it could be used to predict the expected cost increase of the reduction in the number of bids that result from the restrictive tendering in Ontario. This, of course, would require information on the expected reduction in the number of bids that results from the restrictive tendering in Ontario. This is not straightforward since information likely only exists on the number of bids received for each contract, not on the counterfactual or the number that would have prevailed if there was not restrictive tendering. That counterfactual or hypothetical number could perhaps be estimated from a comparison group of jurisdictions that did not have the restrictive tendering.
In addition to providing the relationship between contract costs and the number of bids, this procedure would require an estimate of the expected number of bids that would prevail if restrictive tendering became open to more competition.
An example may illustrate. Based on US data, Damnjanovic et al. (2009, 20) estimate a strong negative relationship between the number of bids and the final project price. The relationship is non-linear, with a reduction in the price of about 8 percent in going from two to three bids, 14 percent for four bids, 18 percent for five bids, 21 percent for six bids, 23 percent for seven bids and 25 percent for eight bids. If such a bid-price relationship prevailed for government construction contracts in Ontario and if it were established, for example, that open contracting would increase the number of bids from four to eight, the cost of the restrictive tendering would be about 11 percent (i.e., 25 percent minus 14 percent). Even if precise numbers of the expected increase in the number of bids could not be established, a range of estimates from those familiar with the bidding process could be used and a sensitivity analysis applied where the cost savings could be illustrated for different hypothetical increases in the number of bids.
Cost Comparisons in Proxies for Restricted and Open Tendering
Comparing the costs of standardized projects that are conducted under restricted and open bidding could provide an estimate of the cost implications of restricted tendering. In their analysis of the cost implications of government contracting practices in the GTHA, McGuinness and Bauld (2010, 25) compare the per-square-foot construction costs of government versus private commercial offices, based on standard industry pricing reference books such as the Canadian edition of Hanscomb’s Yardsticks for Costing. They show that government construction costs relative to those of the private sector have been trending upward, so that by 2008 they were 23 percent higher.
An obvious problem with such comparisons is that government and private sector buildings can be different in other dimensions, so it is not clear that “apples are being compared to apples.” McGuinness and Bauld (2010, 26) recognize this when they say:
Comparing absolute costs of Government and private-sector buildings can be difficult, because with a few exceptions (such as public administration and commercial office buildings) the types of construction being carried out tend to be very different.
If it were feasible to compare government and private-sector projects that are considered similar or homogenous—they suggest public administration and commercial office buildings in the above quote— then such cost differences could reflect higher government costs for the same output. This would shed light on the cost of restricted versus open tendering only if the government projects involved restricted tendering and the private ones involved open tendering.
Prism Economics (2001) provides a guide to construction cost sources, describing twelve such sources that can be used for different purposes.
Municipal Financial Information Return and Regression Procedures
Our research suggests that the following information is available from Ontario’s Financial Information Return program and other sources.
1. Municipal capital budgets—available data includes aggregate expenditures, as well as detailed expenditures that show approved expenditures on a project-by-project basis. Significant time horizon available, depending on municipality. Acquired through the province of Ontario’s Financial Information Return program (https://efis.fma.csc.gov.on.ca/fir/ViewFIR2015.htm)
2. Project-specific information, including:
a. Number and name of bidders
b. Whether bidding is open or restricted
c. Union affiliation of bidders
d. Labour costs/hour of most bidders
e. Dollar amount of bids
f. Type of project (e.g., public housing, school, water treatment plant)
g. Scope of project
h. It is currently unclear whether or not final cost of projects can be attained.
i. The number of firms pre-qualified to compete in various jurisdictions.
3. The number of firms qualified to compete in various jurisdictions over time. And potentially entrance of new firms over time.
4. Municipal data, including number of citizens, CPI, tax rates, etc.
5. Municipal infrastructure data, including
a. Useful life of capital assets such as roads and water treatment plants
b. Average age of capital stock (e.g., age of water treatment plants) as objective measure and as percentage of useful life
c. Development charge rates (Cardus project framework memo)
Such data could be amenable to being used by some of the methodologies previously outlined. They include the following:
A. Comparison of the number of bids between open and restricted contracts.
Open contracts should obviously lead to more bids (and the literature clearly indicates that the greater the number of bids the lower the price). But the question becomes: How many more bids tend to occur with open bidding? This could involve a simple tabulation of the average number of bids in projects with open tendering versus closed tendering. The unit of observation would be a project. In a simple regression of the number of bids and a dummy variable for open versus restricted contracts, the coefficient on the dummy variable would also give the mean difference, with the standard error also enabling determining if that difference were statistically significant.
A potential issue with this procedure is that projects may differ in other characteristics that also affect the number of bids. For example, large projects may attract fewer bidders because of the resources required to complete the contract. Two possible ways to deal with this are as follows: (1) include other characteristics such as bid size as control variables in a regression, or (2) restrict the comparisons to relatively homogenous types of projects. Once the effect of the restricted contracts on the number of bids is estimated, then the cost implications could be estimated from (1) external information on the bid-price relationship, or (2) possibly estimating the bid-price relationship from the Financial Information Return data.
B. Regression with a dummy variable for restricted versus open bidding
It is obviously not possible to compare the cost of restricted versus open bidding within a project since the projects themselves are designated as restricted versus open bidding. However, it is possible to compare the cost of restricted versus open bidding across projects. The dilemma is that project costs depend on numerous other factors besides open and closed bidding. Three possible ways to deal with this are as follows: (1) include other characteristics such as bid size as control variables, (2) restrict the comparisons to relatively homogenous types of projects, and perhaps adding further controls, and (3) use a standardized measure such as labour cost/hour. These different ways are not mutually exclusive but can be combined in various fashions.
C. Union–non-union as proxy for restricted versus open bidding
As indicated, it is obviously not possible to compare the cost of restricted versus open bidding within a project since the projects themselves are designated as restricted versus open bidding. However, within the set of projects with open bidding, there will likely be union and non-union bids. The average gap between union and non-union bids within projects may serve as a proxy for the cost of restricting bids to union contractors. This would be a conservative estimate, however, since union bids are likely to be restrained downward by the threat of non-union bidding in such contestable markets. There may also be non-credible bids that could distort the averages, and those may have to be omitted as outliers.
D. Regression discontinuity (RD)–type union–non-union comparison
A way around the fact that average union and non-union bids may not all be credible bids would be to compare winning bids with the next closest bid in situations of open contracting based on the subset of projects where the winning bid was a union bid and the next closest bid was nonunion, or vice versa. This could also be restricted to projects where those two bids were close. As in a RD design, the assumption would be that the bids are so close that winning could be considered the “luck of the draw”—that is, approximating random assignment, yielding causal estimates of the effect of restricted tendering. Again, this would be a conservative estimate since union bids are likely to be restrained downward by the threat of non-union bidding in such contestable open bidding markets.
Delphi Technique
If the quantitative methodologies discussed above do not prove feasible, a more qualitative method— the Delphi technique—may merit consideration. It could also be used to supplement any of the more quantitative methods.
Damnjanovic et al. (2009, 20) use a Delphi technique to provide information from experts on factors that could be used to reduce cost on government construction contracts. While their work involved cost reduction techniques, it could potentially be applied to getting expert opinion on the potential cost saving from open bidding on projects. This is especially the case if more quantitative methods are not feasible. Their description of the method merits quoting in full.
The objective of this method [the Delphi technique] is to provide a procedure that is able to provide more reliable results for complex problems that are difficult to analyze quantitatively, compared to subjective decision-making by individuals. The Delphi technique involves an iterative process in which expert opinions are processed and used as a feedback for further refinement of opinions generated in the earlier round. . . . The Delphi technique is not intended to replace or substitute for statistical and model-based techniques or human judgment, but it is intended for use where objective decisions are not possible in the absence of historic, economical, or technical data pertinent to the subject. . . . Delphi analysis allows synthesis of the collective opinion of experts when the issues are more of strategic nature and difficult to numerically quantify.
Conclusion and Next Steps
The overwhelming consensus taken from economic literature and from existing policy in government procurement processes suggests that open tendering will lead to lower – and perhaps significantly lower – costs for governments as they build the infrastructure projects they have committed to build. But when it comes to determining the exact nature of those cost implications, clearly there are a wide range of potential methodologies, each with their pros and cons, which could be used to estimate the cost implications of restrictive bidding. The Phase II part of this analysis will sift through these different methodologies, eliminating some as non-starters. Potential “starters” will then be identified and one or more methodologies selected in part also based on availability of the requisite data.