Joint cross-border procurement in the EU/EEA (plus UK) 2019-2021 -- update on Locatelli's (2019) TED analysis

A couple of years ago, Ivo Locatelli — a Senior Procurement Expert at the European Commission — published a first analysis of the emerging trends in cross-border procurement in the EU/EEA based on TED data and, in particular, based on the publication of contract award notices (CANs). His paper is available as: I Locatelli, ‘Do European public buyers purchase together? An assessment of joint cross-border procurement contracts published in TED in 2017 and beyond’ (2019) 1 Ius Publicum art 1.

Locatelli reported that, in 2017 only, 34 CANs were published concerning joint cross-border procedures involving buyers in different Member States. The paper provides detailed analysis and classification of those 34 instances of cross-border joint procurement. The paper acknowledged that, in the grand scheme of things, this was meagre (but important) cross-border experimentation, and Locatelli was hopeful for more intense cross-border joint procurement in the future, once the best practices of a ‘group of brave buyers’ were disseminated and some policy interventions by the European Commission took root.

I am now working on a paper on cross-border procurement with Kirsi-Maria Halonen, so I thought it would be a good idea to try to update Locatelli’s analysis, following as close a methodology as I could. This should allow for a longer view analysis of emerging trends over an almost five year period (2017, as per Locatelli's analysis, plus 2018-2021 to date). This blogpost reports the results and reflects on some issues preventing a proper understanding of the emergence of cross-border joint procurement ‘on the ground’ [for theoretical analysis, see A Sanchez-Graells, ‘The Emergence of Trans-EU Collaborative Procurement: A “Living Lab” for European Public Law’ (2020) 29(1) Public Procurement Law Review 16-41].

‘Mining’ TED for 2018-21

Given how counterintuitive I find the advanced searches in TED, I thought I would cast my net wide (if anyone has suggestions for a more effective approach, I would be most grateful to receive them). So, I searched TED for CANs with the free text “joint procurement” and then manually checked whether there was a cross-border element. I thought the search would, if anything, be overinclusive, as the mandatory CAN standard form requires in part I.2) to indicate whether there is any element of joint procurement and ‘In the case of joint procurement involving different countries, state applicable national procurement law’.

184 results were returned. This is the breakdown of what came up, organized by country of the buyer:

The results were a little surprising, if nothing else for being such few, as well as for the very clear bulges of ‘activity’ in the UK and Norway and, to a lesser extent, Denmark. In order make the analysis of the CANS for the 2018-2021 period comparable to Locatelli’s for 2017, I screened them manually and applied the same exclusion criteria detailed in the paper.

To carry out his analysis of emerging trends in ‘true’ cross-border procurement, Locatelli conducted an ‘assessment of joint cross-border procurement … limited to cases of a contract being awarded, or a framework agreement being concluded, either jointly or via a joint entity, by several contracting authorities located in various Member States or via two foreign CPBs. Therefore, coordinated procurement implying several parallel procedures managed by buyers in different Member States is not covered here.’ The paper also clarified that ‘CANs relating to European Union (EU) institutions’ procurement procedures that are open to agencies and institutions located in different Member States are not covered by this assessment’, and that ‘CANs concerning pre-commercial procurement are not included since public procurement Directives apply to public service contracts for research and development services only where specific conditions are met’.

Doing that resulted in only 11 eligible cross-border joint procurement projects for the period 2018-2021 — with only very few pre-commercial procurements and procurements of the EU Institutions excluded. Interestingly, the bulk of the CANs referred to single-country joint procurement and mostly to municipal or regional collaboration in Norway and the UK, to utilities collaboration in Denmark, as well as collaboration within the English national healthcare system.

What was in TED for 2018-21?

The CANs identified above show a limited picture of cross-border collaboration mainly in the Nordic countries (and remarkably in defence and security procurement), and mostly where there is either a physical or regulatory network requiring (or justifying) cross-border management, or where there is EU funding for a specific activity. The sample is way too small to try to extrapolate any clear trends, so it is worth listing the 11 projects here (from newer to older), in case anyone wants to dive deeper:

General procurement (with EU funding)

• Spanish-led collaboration with UK for the procurement of innovative healthcare services, funded by the EU (project RITMOCORE) [2021/S 040-100288 (and also 2020/S 255-642149)]

• Swedish-led collaboration with Finland for a digital service gathering travel information for visitors in the Stockholm and Turku archipelago, with EU funding (EU Central Baltic funding: CB767) [2020/S 107-259917]

• French-led collaboration (with Spain, Germany and Italy) for the acquisition of super-computers, funded by the EU (under PPI4HPC) [2020/S 092-219297]

  Network procurement (not necessarily utilities)

• Swedish-led Nordic (plus Dutch) collaboration for the maintenance and further development of software for financial reporting via the Northern Transaction Reporting System (NTRS), which was already jointly procured in 2016 [2021/S 094-247849]

• Austrian-led collaboration (with Slovakia, Croatia, Bulgaria and Romania) for the setting up of a transnational Waterway Monitoring System (“WAMOS”) within the Framework of the programme “FAIRway Danube”, co-financed by the Connecting Europe Facility (CEF) programme [2018/S 097-221820]

• German-led collaboration (for the entire Eurosystem, under the auspices of EPCO) to acquire rating agency services [2018/S 069-153490]

• Finnish-led collaboration (with Estonia, Latvia and Lithuania) regarding the identification of the most suitable option for a Baltic-Finnish energy market [2018/S 014-029097]

  Defence and security procurement

• Norwegian-led collaboration with Denmark for a joint procurement for inspections and maintenance of the respective countries’ C130J Hercules planes [2021/S 076-195801]

• Swedish-led collaboration with Norway to acquire UAS-Systems (Unmanned Air System) for their police forces [2019/S 182-442948]

• Danish-led collaboration with Norway for the purchase of type rated courses to maintain the C-130J Block 6.1 Hercules air crafts [2018/S 179-407131]

• Swedish-led collaboration with Finland to procure strategic sealift operations [2018/S 031-068254]

Is this really all the cross-border joint procurement there is?

I think there are problems with this information. Anecdotally, I am aware of a cross-border joint procurement between France and Italy in 2019 that did not show in the search results. There are also some examples in this recent iProcureNet report that also do not show in the search results. More broadly, I think that the poor form-filling that tends to affect TED notices may mask some of the joint cross-border procurement taking place, in particular because mistakes in a small sample can have more relevant effects than in a large sample (contra, Locatelli 2019: 7-8).

Be it as it may, even if the search was defective and the results were massively under-reported by (say) 10 to 1, the picture that emerges is one of extremely limited action in joint cross-border procurement. If the EU-funded projects are excluded and Nordic defence collaboration is set aside, all that is left is joint procurement linked to physical or regulatory network activities with an obvious EU dimension. Therefore, there seems to be very little ‘grassroots’ collaboration on the public buyer side of the EU’s public procurement internal market, except at its margins.

What then?

To my mind, this poses a few relevant questions. First, whether the existence of language and legal barriers that are generally brushed under the carpet in EU policy-making need a serious reconsideration (and I am not the only one to think this; see eg MA Simovart, ‘Choice of law applicable to joint cross-border public procurement by central purchasing bodies or under occasional collaboration agreements’ (2021) 1 Procurement Law Journal 1-18). Second, whether joint cross-border procurement can really be the channel for trans-EU collaboration that the European Commission hopes — eg in relation to the adoption of AI, where the Commission considers that ‘collaborative cross-border procurement has the potential to exploit synergies and achieve higher critical mass in bringing AI solutions to the public sector market across Europe‘ [as announced in the 2018 Coordinated Plan on AI, and detailed in a recent Innovation Procurement Newsletter]. Third, whether the low uptake of collaboration between public buyers and the structurally low level of (direct) cross-border tendering by foreign potential suppliers (as recently reconfirmed, in this new Study on the measurement of cross-border penetration in the EU public procurement market) warrant the current regulatory approach, not only by the Commission, but also by the European Court of Justice.

As you see, there is plenty to think about and discuss. Kirsi and I hope to publish a draft of our paper in a few months. So stay tuned if this is of interest. And, as always, all comments and suggestions most welcome: a.sanchez-graells@bristol.ac.uk.

Interesting paper on effects of open procurement data on outcomes: Duguay, Rauter & Samuels (2019)

A recently published working paper has assessed the impact of increased availability of procurement data on competition for public contracts and on procurement outcomes in the EU context: R Duguay, T Rauter & D Samuels, ‘The Impact of Open Data on Public Procurement’ (November 22, 2019).

Duguay, Rauter & Samuels concentrate on the increased availability of TED data in a (more) user-friendly format in July 2015 (when the data started being available for a bulk download on CSV format) to assess the effects that easier access to procurement data has on the functioning of procurement markets and on procurement outcomes. The paper is very interesting and their results are quite striking.

However, it is important to stress the important caveat that their analysis is still based on TED data and, thus, potentially affected by the quality shortcomings of that data. As mentioned in other occasions, the TED database has problems because it is constructed as a result of the self-declaration of data by the contracting authorities of the Member States, which makes its content very inhomogeneous and difficult to analyse, including significant problems of under-inclusiveness, definitional fuzziness and the lack of filtering of errors—as recognised, repeatedly, in the methodology underpinning the SMSPP itself (see here and here).

With that in mind, however, it is interesting to look closely at their findings.

A seemingly striking insight derived from the paper is that ‘the new European government contracting provisions have anti-competitive effects‘ (at 17). This is in the context of an analysis of the ‘likelihood that government agencies allocate public contracts through an open procedure‘ and should thus not be surprising, given the flexibilisation in the use of procedures involving negotiations. However, even with this regulatory effect, the authors find that more open data triggers more use of open procedures, in particular in EU countries with weaker institutional frameworks (at 18-19, and see below). This could be symptomatic of the fact that more complexity in procurement subjected to higher levels of transparency pushes for a risk-averse approach to procurement compliance. The same would be supported by their finding of higher levels of award of contracts on the basis of price-only award criteria (at 25, and see below).

This tension between procurement complexity and transparency is generally strongly evidenced in the paper.

On the one hand, and in line with claims of the pro-competitive nature of more openness in procurement data (note, not of more openness or transparency of contract opportunities), the authors find that

• the likelihood of competitive bidding increases sharply for TED contracts around July 2015 and that this increase persists through the end of our sample period [ie to the end of 2018] (at 18);

• open procurement data leads government officials to implement more competitive bidding processes [ie open procedures], and that this increase in competitive bidding is driven by countries that do not have the institutions to effectively monitor public officials (at 19);

• the number of bids increases sharply for TED contracts soon after the open data initiative, and this increase persists throughout our sample period (at 20);

• public officials are 8.7 percentage points more likely to award government contracts to new vendors after the open data initiative (at 21);

• contract values fall by approximately 8% ... after the open data initiative (at 23).

On the other hand, and also in line with theoretical expectations of a degradation of procurement decisions subjected to higher levels of transparency (and the fact that this transparency does not concern contract opportunities, but more general open procurement data), the authors also find that

• [the results] are inconsistent with the idea that easier access to procurement data fosters cross-border competition throughout the European Union … open procurement data fosters local competition among vendors by reducing barriers to entry but does not promote cross-border competition across the European single market (at 22);

• after the open data initiative, the likelihood of a contract modification increases by 2.9 percentage points for contracts above TED publication thresholds (at 24);

• after the open data initiative, public officials are 38% ... more likely to award contracts above TED publication thresholds exclusively based on price (at 25);

• the performance ... is significantly worse if price was the only award criterion in the allocation decision (at 26);

• the increase in modifications is driven by contracts awarded to new government suppliers, consistent with information asymmetries contributing to the observed deterioration in contract performance. Moreover, this evidence suggests that procurement relationships before the open data initiative were not necessarily corrupt or otherwise inefficient (at 26);

• the decline in contract performance is stronger for complex procurements, consistent with project complexity exacerbating the potential allocative distortions of open procurement data (at 27).

Their overall conclusion is that

Comparing government contracts above and below EU publication thresholds, we find that increasing the public accessibility of procurement data raises the likelihood of having competitive bidding processes, increases the number of bids per contract, and facilitates market entry by new vendors. After the open data initiative, procurement prices decrease and EU government agencies are more likely to award contracts to the lowest bidder. However, the increased competition comes at the expense of lower contract performance, particularly if suppliers are new, procurement projects are complex, and contracts are awarded solely based on price.

Overall, our results suggest that open data on procurement awards facilitates competition and lowers ex-ante procurement prices, but does not necessarily increase allocative efficiency in government contracting (at 27-28, emphases added).

I find these results striking and difficult to assess from the perspective of evidence-based policy-making. There are two issues of particular concern/interest to me.

One, the finding that more availability of data does not generate more cross-border procurement, and that the push for more competitive (ie open) procedures is mostly appreciable in countries with weaker institutional frameworks. This could support the position that institutional robustness is an alternative to data transparency, which would significantly alter the prioritisation of systemic procurement reforms and take the sides of systems that favour strong institutional oversight in a context of relative opacity.

Second, that transparency exacerbates problems at execution phase, in particular in complex projects and/or projects with new suppliers. This would take the wind out of the sails of reform and policy-making approaches concentrating on perceived or apparent competition for the contract at award stage, and rather force a refocus on an analysis of procurement outcomes at the end of the relevant project. This would also side with approaches that would advocate for more robust institutional approaches to contract design and performance management, rather than relying on transparency to correct contract execution problems.

The mixed results of the paper are also interesting in the context of the long-term effect of more open procurement data on competition, as well as on cartelisation and bid rigging risks, which are not assessed in the paper.

On the round, I think that the paper offers some interesting evidence to back up that there is a need to reconsider the level of transparency given to procurement data. I do not think this should stop the development of an improved procurement data architecture in the EU. To the contrary. I think this should reignite and prioritise discussions concerning the level of disclosure or public access to that information (ie its openness), which cannot be simply assumed to be positive in what, in my view, is currently an excessively simplistic approach in leading policy-making and think tank proposals. For more (but not new) discussion, see here and here.

Further thoughts on data and policy indicators a-propos two recent papers on procurement regulation & competition: comments re (Tas: 2019a&b)

The EUI Robert Schuman Centre for Advanced Studies’ working papers series has two interesting recent additions on the economic analysis of procurement regulation and its effects on competition, efficiency and value for money. Both papers are by BKO Tas.

The first paper: ‘Bunching Below Thresholds to Manipulate Public Procurement’ explores the effects of a contracting authority’s ‘bunching strategy’ to seek to exercise more discretion by artificially estimating the value of future contracts just below the thresholds that would trigger compliance with EU procurement rules. This paper is relevant to the broader discussion on the usefulness and adequacy of current EU (and WTO GPA) value thresholds (see eg the work of Telles, here and here), as well as on the regulatory decisions that EU Member States face on whether to extend the EU rules to ‘below-threshold’ contracts.

The second paper: ‘Effect of Public Procurement Regulation on Competition and Cost-Effectiveness’ uses the World Bank’s ‘Benchmarking Public Procurement’ quality scores to empirically test the positive effects of improved regulation quality on competition and value for money, measured as increases in the number of bidders and the probability that procurement price is lower than estimated cost. This paper is relevant in the context of recent discussions about the usefulness or not of procurement benchmarks, and regarding the increasing concern about reduced number of bids in EU-regulated public tenders.

In this blog post, I reflect on the methodology and insights of both papers, paying particular attention to the fact that both papers build on datasets and/or indexes (TED, the WB benchmark) that I find rather imperfect and unsuitable for this type of analysis (regarding TED, in the context of the Single Market Scoreboard for Public Procurement (SMPP) that builds upon it, see here; regarding the WB benchmark, see here). Therefore, not all criticisms below are to the papers themselves, but rather to the distortions that skewed, incomplete or misleading data and indicators can have on more refined analysis that builds upon them.

Bunching Below Thresholds to Manipulate Procurement (Tas: 2019a)

It is well-known that the EU procurement rules are based on a series of jurisdictional triggers and that one of them concerns value thresholds—currently regulated in Arts 4 & 5 of Directive 2014/24/EU. Contracts with an estimated value above those thresholds are subjected to the entire EU procurement regulation, whereas contracts of a lower value are solely subjected to principles-based requirements where they are of ‘cross-border interest’. Given the obvious temptation/interest in keeping procurement shielded from EU requirements, the EU Directives have included an anti-circumvention rule aimed at preventing Member States from artificially splitting contracts in order to keep their award below the relevant jurisdictional thresholds (Art 5(3) Dir 2014/24). This rule has been interpreted expansively by the Court of Justice of the European Union (see eg here).

‘Bunching Below Thresholds to Manipulate Public Procurement’ examines the effects of a practice that would likely infringe the anti-circumvention rule, as it assesses a strategy of ‘bunching estimated costs just below thresholds’ ‘to exercise more discretion in public procurement’. The paper develops a methodology to identify contracting authorities ‘that have higher probabilities of bunching estimated values below EU thresholds’ (ie manipulative authorities) and finds that ‘[m]anipulative authorities have significantly lower probabilities of employing competitive procurement procedure. The bunching manipulation scheme significantly diminishes cost-effectiveness of public procurement. On average, prices of below threshold contracts are 18-28% higher when the authority has an elevated probability of bunching.’ These are quite striking (but perhaps not surprising) results.

The paper employs a regression discontinuity approach to determine the likelihood of bunching. In order to do that, the paper relies on the TED database. The paper is certainly difficult to read and hardly intelligible for a lawyer, but there are some issues that raise important questions. One concerns the authors’ (mis)understanding of how the WTO GPA and the EU procurement rules operate, in particular when the paper states that ‘Contracts covered by the WTO GPA are subject to additional scrutiny by international organizations and authorities (sic). Accordingly, contracts covered by the WTO GPA are less likely to be manipulated by EU authorities’ (p. 12).  This is simply an acritical transplant of considerations made by the authors of a paper that examined procurement in the Czech Republic, where the relevant threshold between EU covered and non-EU covered procurement would make sense. Here, the distinction between WTO GPA and EU-covered procurement simply makes no sense, given that WTO GPA and EU thresholds are coordinated. This alone raises some issues concerning the tests designed by the author to check the robustness of the hypothesis that bunching leads to inefficiency in procurement expenditure.

Another issue concerns the way in which the author equates open procedures to a ‘first price auction mechanism’ (which they are not exactly) and dismisses other procedures (notably, the restricted procedure) as incapable of ensuring value for money or, more likely, as representative of a higher degree of discretion for the contracting authority—which is a highly questionable assumption.

More importantly, I am not sure that the author understood what is in the TED database and, crucially, what is not there (see section 2 of Tas (2019a) for methodology and data description). Albeit not very clearly, the author presents TED as a comprehensive database of procurement notices—ie, as if 100% of procurement expenditure by Member States was recorded there. However, in the specific context of bunching below thresholds, the TED database is very likely to be incomplete.

Contracting authorities tendering contracts below EU thresholds are under no obligation to publish a contract notice (Art 49 Dir 2014/24). They could publish voluntarily, in particular in the form of a voluntary ex ante transparency (VEAT) notice, but that would make no sense from the perspective of a contracting authority that seeks to avoid compliance with EU rules by bunching (ie manipulating) the estimated contract value, as that would expose it to potential litigation. Most authorities that are bunching their procurement needs (or, in simple terms) avoiding compliance with the EU rules will not be reflected in the TED database at all, or will not be identified by the methodology used by Tas (2019a), as they will not have filed any notices for contracts below thresholds.

How is it possible that TED includes notices regarding contracts below the EU thresholds, then? Well, this is anybody’s guess, but mine is that a large proportion of those notices will be linked to either countries with a tradition of full transparency (over-reporting), to contracts where there are any doubts about the potential cross-border interest (sometimes assessed over-cautiously), or will be notices with mistakes, where the estimated value of the contract is erroneously indicated as below thresholds.

Even if my guess was incorrect and all notices for contracts with a value below thresholds were accurate and justified by the existence of a potential cross-border interest, the database cannot be considered complete. One of the issues raised (imperfectly) by the Single Market Scoreboard (indicator [3] publication rate) is the relatively low level of procurement that is advertised in TED compared to the (putative/presumptive) total volume of procurement expenditure by the Member States. Without information on the conditions of the vast majority of contract awards (below thresholds, unreported, etc), any analysis of potential losses of competitiveness / efficiency in public expenditure (due to bunching or otherwise) is bound to be misleading.

Moreover, Tas (2019a) is premised on the hypothesis that procurement below EU thresholds allows for significantly more discretion than procurement above those thresholds. However, this hypothesis fails to recognise the variety of transposition strategies at Member State level. While some countries have opted for less stringent below EU threshold regimes, others have extended the EU rules to the entirety of their procurement (or, perhaps, to contracts up to and including much lower values than the EU thresholds, to the exception of some class of ‘micropurchases’). This would require the introduction of a control that could refine Tas’ analysis and distinguish those cases of bunching that do lead to more discretion and those that do not (at least formally)—which could perhaps distinguish between price effects derived from national-only transparency from those of more legally-dubious maneuvering.

In my view, regardless of the methodology and the math underpinning the paper (which I am in no position to assess in detail), once these data issues are taken into account, the story the paper tries to tell breaks down and there are important shortcomings in its empirical strategy that, in my view, raise significant issues around the strength of its findings—assessed not against the information in TED, but against the (largely unknown, unrecorded) reality of procurement in the EU.

I have no doubt that there is bunching in practice, and that the intuition that it raises procurement costs must be right, but I have serious doubts about the possibility to reliably identify bunching or estimate its effects on the basis of the information in TED, as most culprits will not be included and the effects of below threshold (national) competition only will mostly not be accounted for.

(Good) Regulation, Competition & Cost-Effectiveness (Tas: 2019b)

It is also a very intuitive hypothesis that better regulation should lead to better procurement outcomes and, consequently, that more open and robust procurement rules should lead to more efficiency in the expenditure of public funds. As mentioned above, Tas (2019b) explores this hypothesis and seeks to empirically test it using the TED database and the World Bank’s Benchmarking Public Procurement (in its 2017 iteration, see here). I will not repeat my misgivings about the use of the TED database as a reliable source of information. In this second part, I will solely comment on the use of the WB’s benchmark.

The paper relies on four of the WB’s benchmark indicators (one further constructed by Djankov et al (2017)): the ‘bid preparation score, bid and contract management score, payment of suppliers score and PP overall index’. The paper includes a useful table with these values (see Tas (2019b: Table 4)), which allows the author to rank the countries according to the quality of their procurement regulation. The findings of Tas (2019b) are thus entirely dependent on the quality of the WB’s benchmark and its ability to capture (and distinguish) good procurement regulation.

In order to test the extent to which the WB’s benchmark is a good input for this sort of analysis, I have compared it to the indicator that results from the European Commission’s Single Market Scoreboard for Public Procurement (SMSPP, in its 2018 iteration). The comparison is rather striking …

Source: own elaboration.

Clearly, both sets of indicators are based on different methodologies and measure relatively different things. However, they are both intended to express relevant regulators’ views on what constitutes ‘good procurement regulation’. In my view, both of them fail to do so for reasons already given (see here and here).

The implications for work such as Tas (2019b) is that the reliability of the findings—regardless of the math underpinning them—is as weak as the indicators they are based on. Likely, plugging the same methods to the SMSPP instead of the WB’s index would yield very different results—perhaps, that countries with very low quality of procurement regulation (as per the SMSPP index) achieve better economic results, which would not be a popular story with policy-makers…  and the results with either index would also be different if the algorithms were not fed by TED, but by a more comprehensive and reliable database.

So, the most that can be said is that attempts to empirically show effects of good (or poor) procurement regulation remain doomed to fail or , in perhaps less harsh terms, doomed to tell a story based on a very skewed, narrow and anecdotal understanding of procurement and an incomplete recording of procurement activity. Believe those stories at your own peril…