Creating (positive) friction in AI procurement

I had the opportunity to participate in the Inaugural AI Commercial Lifecycle and Procurement Summit 2024 hosted by Curshaw. This was a very interesting ‘unconference’ where participants offered to lead sessions on topics they wanted to talk about. I led a session on ‘Creating friction in AI procurement’.

This was clearly a counterintuitive way of thinking about AI and procurement, given that the ‘big promise’ of AI is that it will reduce friction (eg through automation, and/or delegation of ‘non-value-added’ tasks). Why would I want to create friction in this context?

The first clarification I was thus asked for was whether this was about ‘good friction’ (as opposed to old bad ‘red tape’ kind of friction), which of course it was (?!), and the second, what do I mean by friction.

My recent research on AI procurement (eg here and here for the book-long treatment) has led me to conclude that we need to slow down the process of public sector AI adoption and to create mechanisms that bring back to the table the ‘non-AI’ option and several ‘stop project’ or ‘deal breaker’ trumps to push back against the tidal wave of unavoidability that seems to dominate all discussions on public sector digitalisation. My preferred solution is to do so through a system of permissioning or licencing administered by an independent authority—but I am aware and willing to concede that there is no political will for it. I thus started thinking about second-best approaches to slowing public sector AI procurement. This is how I got to the idea of friction.

By creating friction, I mean the need for a structured decision-making process that allows for collective deliberation within and around the adopting institution, and which is supported by rigorous impact assessments that tease out second and third order implications from AI adoption, as well as thoroughly interrogating first order issues around data quality and governance, technological governance and organisational capability, in particular around risk management and mitigation. This is complementary—but hopefully goes beyond—emerging frameworks to determine organisational ‘risk appetite’ for AI procurement, such as that developed by the AI Procurement Lab and the Centre for Inclusive Change.

The conversations the focus on ‘good friction’ moved in different directions, but there are some takeaways and ideas that stuck with me (or I managed to jot down in my notes while chatting to others), such as (in no particular order of importance or potential):

  • the potential for ‘AI minimisation’ or ‘non-AI equivalence’ to test the need for (specific) AI solutions—if you can sufficiently approximate, or replicate, the same functional outcome without AI, or with a simpler type of AI, why not do it that way?;

  • the need for a structured catalogue of solutions (and components of solutions) that are already available (sometimes in open access, where there is lots of duplication) to inform such considerations;

  • the importance of asking whether procuring AI is driven by considerations such as availability of funding (is this funded if done with AI but not funded, or hard to fund at the same level, if done in other ways?), which can clearly skew decision-making—the importance of considering the effects of ‘digital industrial policy’ on decision-making;

  • the power (and relevance) of the deceptively simple question ‘is there an interdisciplinary team to be dedicated to this, and exclusively to this’?;

  • the importance of knowledge and understanding of the tech and its implications from the beginning, and of expertise in the translation of technical and governance requirements into procurement requirements, to avoid ‘games of chance’ whereby the use of ‘trendy terms’ (such as ‘agile’ or ‘responsible’) may or may not lead to the award of the contract to the best-placed and best-fitting (tech) provider;

  • the possibility to adapt civic monitoring or social witnessing mechanisms used in other contexts, such as large infrastructure projects, to be embedded in contract performance and auditing phases;

  • the importance of understanding displacement effects and whether deploying a solution (AI or automation, or similar) to deal with a bottleneck will simply displace the issue to another (new) bottleneck somewhere along the process;

  • the importance of understanding the broader organisational changes required to capture the hoped for (productivity) gains arising from the tech deployment;

  • the importance of carefully considering and resourcing the much needed engagement of the ‘intelligent person’ that needs to check the design and outputs of the AI, including frontline workers and those at the receiving end of the relevant decisions or processes and the affected communities—the importance of creating meaningful and effective deliberative engagement mechanisms;

  • relatedly, the need to ensure organisational engagement and alignment at every level and every step of the AI (pre)procurement process (on which I would recommend reading this recent piece by Kawakami and colleagues);

  • the need to assess the impacts of changes in scale, complexity, and error exposure;

  • the need to create adequate circuit-breakers throughout the process.

Certainly lots to reflect on and try to embed in future research and outreach efforts. Thanks to all those who participated in the conversation, and to those interested in joining it. A structured way to do so is through this LinkedIn group.

Meaning, AI, and procurement -- some thoughts

©Ausrine Kuze, Distorted Reality, 2021.

James McKinney and Volodymyr Tarnay of the Open Contracting Partnership have published ‘A gentle introduction to applying AI in procurement’. It is a very accessible and helpful primer on some of the most salient issues to be considered when exploring the possibility of using AI to extract insights from procurement big data.

The OCP introduction to AI in procurement provides helpful pointers in relation to task identification, method, input, and model selection. I would add that an initial exploration of the possibility to deploy AI also (and perhaps first and foremost) requires careful consideration of the level of precision and the type (and size) of errors that can be tolerated in the specific task, and ways to test and measure it.

One of the crucial and perhaps more difficult to understand issues covered by the introduction is how AI seeks to capture ‘meaning’ in order to extract insights from big data. This is also a controversial issue that keeps coming up in procurement data analysis contexts, and one that triggered some heated debate at the Public Procurement Data Superpowers Conference last week—where, in my view, companies selling procurement insight services were peddling hyped claims (see session on ‘Transparency in public procurement - Data readability’).

In this post, I venture some thoughts on meaning, AI, and public procurement big data. As always, I am very interested in feedback and opportunities for further discussion.

Meaning

Of course, the concept of meaning is complex and open to philosophical, linguistic, and other interpretations. Here I take a relatively pedestrian and pragmatic approach and, following the Cambridge dictionary, consider two ways in which ‘meaning’ is understood in plain English: ‘the meaning of something is what it expresses or represents’, and meaning as ‘importance or value’.

To put it simply, I will argue that AI cannot capture meaning proper. It can carry complex analysis of ‘content in context’, but we should not equate that with meaning. This will be important later on.

AI, meaning, embeddings, and ‘content in context’

The OCP introduction helpfully addresses this issue in relation to an example of ‘sentence similarity’, where the researchers are looking for phrases that are alike in tender notices and predefined green criteria, and therefore want to use AI to compare sentences and assign them a similarity score. Intuitively, ‘meaning’ would be important to the comparison.

The OCP introduction explains that:

Computers don’t understand human language. They need to operate on numbers. We can represent text and other information as numerical values with vector embeddings. A vector is a list of numbers that, in the context of AI, helps us express the meaning of information and its relationship to other information.

Text can be converted into vectors using a model. [A sentence transformer model] converts a sentence into a vector of 384 numbers. For example, the sentence “don’t panic and always carry a towel” becomes the numbers 0.425…, 0.385…, 0.072…, and so on.

These numbers represent the meaning of the sentence.

Let’s compare this sentence to another: “keep calm and never forget your towel” which has the vector (0.434…, 0.264…, 0.123…, …).

One way to determine their similarity score is to use cosine similarity to calculate the distance between the vectors of the two sentences. Put simply, the closer the vectors are, the more alike the sentences are. The result of this calculation will always be a number from -1 (the sentences have opposite meanings) to 1 (same meaning). You could also calculate this using other trigonometric measures such as Euclidean distance.

For our two sentences above, performing this mathematical operation returns a similarity score of 0.869.

Now let’s consider the sentence “do you like cheese?” which has the vector (-0.167…, -0.557…, 0.066…, …). It returns a similarity score of 0.199. Hooray! The computer is correct!

But, this method is not fool-proof. Let’s try another: “do panic and never bring a towel” (0.589…, 0.255…, 0.0884…, …). The similarity score is 0.857. The score is high, because the words are similar… but the logic is opposite!

I think there are two important observations in relation to the use of meaning here (highlighted above).

First, meaning can hardly be captured where sentences with opposite logic are considered very similar. This is because the method described above (vector embedding) does not capture meaning. It captures content (words) in context (around other words).

Second, it is not possible to fully express in numbers what text expresses or represents, or its importance or value. What the vectors capture is the representation or expression of such meaning, the representation of its value and importance through the use of those specific words in the particular order in which they are expresssed. The string of numbers is thus a second-degree representation of the meaning intended by the words; it is a numerical representation of the word representation, not a numerical representation of the meaning.

Unavoidably, there is plenty scope for loss, alteration or even inversion of meaning when it goes through multiple imperfect processes of representation. This means that the more open textured the expression in words and the less contextualised in its presentation, the more difficult it is to achieve good results.

It is important to bear in mind that the current techniques based on this or similar methods, such as those based on large language models, clearly fail on crucial aspects such as their factuality—which ultimately requires checking whether something with a given meaning is true or false.

This is a burgeoning area of technnical research but it seems that even the most accurate models tend to hover around 70% accuracy, save in highly contextual non-ambiguous contexts (see eg D Quelle and A Bovet, ‘The perils and promises of fact-checking with large language models’ (2024) 7 Front. Artif. Intell., Sec. Natural Language Processing). While this is an impressive feature of these tools, it can hardly be acceptable to extrapolate that these tools can be deployed for tasks that require precision and factuality.

Procurement big data and ‘content and context’

In some senses, the application of AI to extract insights from procurement big data is well suited to the fact that, by and large, existing procurement data is very precisely contextualised and increasingly concerns structured content—that is, that most of the procurement data that is (increasingly) available is captured in structured notices and tends to have a narrowly defined and highly contextual purpose.

From that perspective, there is potential to look for implementations of advanced comparisons of ‘content in context’. But this will most likely have a hard boundary where ‘meaning’ needs to be interpreted or analysed, as AI cannot perform that task. At most, it can help gather the information, but it cannot analyse it because it cannot ‘understand’ it.

Policy implications

In my view, the above shows that the possibility of using AI to extract insights from procurement big data needs to be approched with caution. For tasks where a ‘broad brush’ approach will do, these can be helpful tools. They can help mitigate the informational deficit procurement policy and practice tend to encounter. As put in the conference last week, these tools can help get a sense of broad trends or directions, and can thus inform policy and decision-making only in that regard and to that extent. Conversely, AI cannot be used in contexts where precision is important and where errors would affect important rights or interests.

This is important, for example, in relation to the fascination that AI ‘business insights’ seems to be triggering amongst public buyers. One of the issues that kept coming up concerns why contracting authorities cannot benefit from the same advances that are touted as being offered to (private) tenderers. The case at hand was that of identifying ‘business opportunities’.

A number of companies are using AI to support searches for contract notices to highlight potentially interesting tenders to their clients. They offer services such as ‘tender summaries’, whereby the AI creates a one-line summary on the basis of a contract notice or a tender description, and this summary can be automatically translated (eg into English). They also offer search services based on ‘capturing meaning’ from a company’s website and matching it to potentially interesting tender opportunities.

All these services, however, are at bottom a sophisticated comparison of content in context, not of meaning. And these are deployed to go from more to less information (summaries), which can reduce problems with factuality and precision except in extreme cases, and in a setting where getting it wrong has only a marginal cost (ie the company will set aside the non-interesting tender and move on). This is also an area where expectations can be managed and where results well below 100% accuracy can be interesting and have value.

The opposite does not apply from the perspective of the public buyer. For example, a summary of a tender is unlikely to have much value as, with all likelihood, the summary will simply confirm that the tender matches the advertised object of the contract (which has no value, differently from a summary suggesting a tender matches the business activities of an economic operator). Moreover, factuality is extremely important and only 100% accuracy will do in a context where decision-making is subject to good administration guarantees.

Therefore, we need to be very careful about how we think about using AI to extract insights from procurement (big) data and, as the OCP introduction highlights, one of the most important things is to clearly define the task for which AI would be used. In my view, there are much more limited tasks than one could dream up if we let our collective imagination run high on hype.

The principle of competition is dead. Long live the principle of competition (Free webinar)

Free webinar: 22 March 2024 *revised time* 1pm UK / 2pm CET / 3pm EET. Registration here.

The role of competition in public procurement regulation continues to be debated. While it is generally accepted that the proper functioning of procurement markets requires some level of competition – and the European Court of Auditors has recently pointed out that current levels of competition for public contracts in the EU are not satisfactory – the 'legal ranking' and normative weight of competition concerns are much less settled.

This has been evidenced in a recent wave of academic discussion on whether there is a general principle of competition at all in Directive 2014/24/EU, what is its normative status and how it ranks vis-à-vis sustainability and environmental considerations, and what are its practical implications for the interpretation and application of EU public procurement law.

Bringing together voices representing a wide range of views, this webinar will explore these issues and provide a space for reflective discussion on competition and public procurement. The webinar won't settle the debate, but hopefully it will allow us to take stock and outline thoughts for the next wave of discussion. It will also provide an opportunity for an interactive Q&A.

Speakers:

  • Prof Roberto Caranta, Full Professor of Administrative Law, University of Turin.

  • Mr Trygve Harlem Losnedahl, PhD researcher, University of Oslo.

  • Dr Dagne Sabockis, Senior Associate, Vinge law firm; Stockholm School of Economics.

  • Prof Albert Sanchez-Graells, Professor of Economic Law, University of Bristol.

Pre- or post-reading:

Centralised procurement for the health care sector -- bang for your pound or siphoning off scarce resources?

The National Health Service (NHS) has been running a centralised model for health care procurement in England for a few years now. The current system resulted from a redesign of the NHS supply chain that has been operational since 2019 [for details, see A Sanchez-Graells, ‘Centralisation of procurement and supply chain management in the English NHS: some governance and compliance challenges’ (2019) 70(1) NILQ 53-75.]

Given that the main driver for the implementation and redesign of the system was to obtain efficiencies (aka savings) through the exercise of the NHS’ buying power, both the UK’s National Audit Office (NAO) and the House of Commons’ Public Accounts Committee (PAC) are scrutinising the operation of the system in its first few years.

The NAO published a scathing report on 12 January 2024. Among many other concerning issues, the report highlighted how, despite the fundamental importance of measuring savings, ‘NHS Supply Chain has used different methods to report savings to different audiences, which could cause confusion.’ This triggered a clear risk of recounting (ie exaggeration) of claims of savings, as detailed below.

In my submission of written evidence to the PAC Inquiry ‘NHS Supply Chain and efficiencies in procurement’, I look in detail at the potential implications of the use of different savings reporting methods for the (mis)management of scarce NHS resources, should the recounting of savings have allowed private subcontractors to also overclaim savings in order to boost the financial return under their contracts. The full text of my submission is reproduced below, in case of interest.

nao’s findings on recounting of savings

There are three crucial findings in the NAO’s report concerning the use of different (and potentially problematic) savings reporting methods. They are as follows:

DHSC [the Department of Health and Social Care] set Supply Chain a cumulative target of making £2.4 billion savings by 2023-24. Supply Chain told us that it had exceeded this target by the end of 2022-23 although we have not validated this saving. The method for calculating this re-counted savings from each year since 2015-16. Supply Chain calculated its reported savings against the £2.4 billion target by using 2015-16 prices as its baseline. Even if prices had not reduced in any year compared with the year before, a saving was reported as long as prices were lower than that of the baseline year. This method then accumulated savings each year, by adding the difference in price as at the baseline year, for each year. This accumulation continued to re-count savings made in earlier years and did not take inflation into account. For example, if a product cost £10 in 2015-16 and reduced to £9 in 2016-17, Supply Chain would report a saving of £1. If it remained at £9 in 2017-18, Supply Chain would report a total saving of £2 (re-counting the £1 saved in 2016-17). If it then reduced to £8 in 2018-19, Supply Chain would report a total saving of £4 (re-counting the £1 saved in each of 2016-17 and 2017-18 and saving a further £2 in 2018-19) […]. DHSC could not provide us with any original sign-off or agreement that this was how Supply Chain should calculate its savings figure (para 2.4, emphasis added).

Supply Chain has used other methods for calculating savings which could cause confusion. It has used different methods for different audiences, for example, to government, trusts and suppliers (see Figure 5). When reporting progress against its £2.4 billion target it used a baseline from 2015-16 and accumulated the amount each year. To help show the savings that trusts have made individually, it also calculates in-year savings each trust has made using prices paid the previous year as the baseline. In this example, if a trust paid £10 for an item in 2015-16, and then procured it for £9 from Supply Chain in 2016-17 and 2017-18, Supply Chain would report a saving of £1 in the first year and no saving in the second year. These different methods have evolved since Supply Chain was established and there is a rationale for each. Having several methods to calculate savings has the potential to cause confusion (para 2.6, emphasis added).

When I read the report, I thought that the difference between the methods was not only problematic in itself, but also showed that the ‘main method’ for NHS Supply Chain and government to claim savings, in allowing recounting of savings, was likely to have allowed for excessive claims. This is not only a technical or political problem, but also a clear risk of siphoning off NHS scarce budgetary resources, for the reasons detailed below.

Submission to the pac inquiry

00. This brief written submission responds to the call for evidence issued by the Public Accounts Committee in relation to its Inquiry “NHS Supply Chain and efficiencies in procurement”. It focuses on the specific point of ‘Progress in delivering savings for the NHS’. This submission provides further details on the structure and functioning of NHS Supply Chain than those included in the National Audit Office’s report “NHS Supply Chain and efficiencies in procurement” (2023-24, HC 390). The purpose of this further detail is to highlight the broader implications that the potential overclaim of savings generated by NHS Supply Chain may have had in relation to payments made to private providers to whom some of the supply chain functions have been outsourced. It raises some questions that the Committee may want to explore in the context of its Inquiry.

1. NHS Supply Chain operating structure

01. The NAO report analyses the functioning and performance of NHS Supply Chain and SCCL in a holistic manner and without considering details of the complex structure of outsourced functions that underpins the model. This can obscure some of the practical impacts of some of NAO’s findings, in particular in relation with the potential overclaim of savings generated by NHS Supply Chain (paras 2.4, 2.6 and Figure 5 in the report). Approaching the analysis at a deeper level of detail on NHS Supply Chain’s operating structure can shed light on problems with the methods for calculating NHS Supply Chain savings other than the confusion caused by the use of multiple methods, and the potential overclaim of savings in relation to the original target set by DHSC.

02. NHS Supply Chain does not operate as a single entity and SCCL is not the only relevant actor in the operating structure.[1] Crucially, the operating model consists of a complex network of outsourcing contracts around what are called ‘category towers’ of products and services. SCCL coordinates a series of ‘Category Tower Service Providers’ (CTSPs), as listed in the graph below. CTSPs have an active role in developing category management strategies (that is, the ‘go to market approach’ at product level) and heavily influence the procurement strategy for the relevant category, subject to SCCL approval.

03. CTSPs are incentivised to reduce total cost in the system, not just reduce unit prices of the goods and services covered by the relevant category. They hold Guaranteed Maximum Price Target Cost (GMPTC) contracts, under which CTSPs will be paid the operational costs incurred in performing the services against an annual target set out in the contract, but will only make a profit when savings are delivered, on a gainshare basis that is capped.

Source: NHS Supply Chain - New operating model (2018).[2]

04. There are very limited public details on how the relevant targets for financial services have been set and managed throughout the operation of the system. However, it is clear that CTSPs have financial incentives tied to the generation of savings for SCCL. Given that SCCL does not carry out procurement activities without CTSP involvement, it seems plausible that SCCL’s own targets and claimed savings would (primarily) have been the result of the simple aggregation of those of CTSPs. If that is correct, the issues identified in the NAO report may have resulted in financial advantages to CTSPs if they have been allowed to overclaim savings generated.

05. NHS Supply Chain has publicly stated that[3]:

  • ‘Savings are contractual to the CTSPs. As part of the procurement, bidders were asked to provide contractual savings targets for each year. These were assessed and challenged through the process and are core to the commercial model. CTSPs cannot attain their target margins (i.e. profit) unless they are able to achieve contractual savings.’

  • ‘The CTSPs financial reward mechanism [is] based upon a gain share from the delivery of savings. The model includes savings generated across the total system, not just the price of the product. The level of gain share is directly proportional to the level of savings delivered.’

06. In view of this, if CTSPs had been allowed to use a method of savings calculation that re-counted savings in the way NAO details at para 2.4 of its report, it is likely that their financial compensation will have been higher than it should have been under alternative models of savings calculation that did not allow for such re-count. Given the volumes of savings claimed through the period covered by the report, any potential overcompensation could have been significant. As any such overcompensation would have been covered by NHS funding, the Committee may want to include its consideration within its Inquiry and in its evidence-gathering efforts.

__________________________________

[1] For a detailed account, see A Sanchez-Graells, “Centralisation of procurement and supply chain management in the English NHS: some governance and compliance challenges” (2019) 70(1) Northern Ireland Legal Quarterly 53-75.

[2] Available at https://wwwmedia.supplychain.nhs.uk/media/Customer_FAQ_November_2018.pdf (last accessed 12 January 2024).

[3] Ibid, FAQs 24 and 25.

Responsibly Buying Artificial Intelligence: A Regulatory Hallucination?

I look forward to delivering the lecture ‘Responsibly Buying Artificial Intelligence: A Regulatory Hallucination?’ as part of the Current Legal Problems Lecture Series 2023-24 organised by UCL Laws. The lecture will be this Thursday 23 November 2023 at 6pm GMT and you can still register to participate (either online or in person). These are the slides I will be using, in case you want to take a sneak peek. I will post a draft version of the paper after the lecture. Comments welcome!

Innovation procurement under the Procurement Act 2023 -- changing procurement culture on the cheap?

On 13 November 2023, the UK Government published guidance setting out its ambitions for innovation procurement under the new Procurement Act 2023 (not yet in force, of which you can read a summary here). This further expands on the ambitions underpinning the Transforming Public Procurement project that started after Brexit. The Government’s expectation is that the ‘the new legislation will allow public procurement to be done in more flexible and innovative ways’, and that this will ‘enable public sector organisations to embrace innovation more’.

The innovation procurement guidance bases its expectation that the Procurement Act will unlock more procurement of innovation and more innovative procurement on the ambition that this will be an actively supported policy by all relevant policy- and decision-makers and that there will be advocacy for the development of commercial expertise. A first hurdle here is that unless such advocacy comes with the investment of significant funds in developing skills (and this relates to both commercial and technical skills, especially where the innovation relates to digital technologies), such high-level political buy-in may not translate into any meaningful changes. The guidance itself acknowledges that the ‘overall culture, expertise and incentive structure of the public sector has led to relatively low appetite for risk and experimentation’. Therefore, that greater investment in expertise needs to be coupled with a culture change. And we know this is a process that is very difficult to push forward.

The guidance also indicates that ‘Greater transparency of procurement data will make it easier to see what approaches have been successful and encourage use of those approaches more widely across the public sector.’ This potentially points to another hurdle in unlocking this policy because generic data is not enough to support innovation procurement or the procurement of innovation. Being able to successfully replicate innovation procurement practices requires a detailed understanding of how things were done, and how they need to be adapted when replicated. However, the new transparency regime does not necessarily guarantee that such granular and detailed information will be available, especially as the practical level of transparency that will stem from the new obligations crucially hinges on the treatment of commercially sensitive information (which is exempted from disclosure in s.94 PA 2023). Unless there is clear guidance on disclosure / withholding of sensitive commercial information, it can well be that the new regime does not generate additional meaningful (publicly accessible) data to push the knowledge stock and support innovative procurement. This is an important issue that may require further discussion in a separate post.

The guidance indicates that the changes in the Procurement Act will help public buyers in three ways:

  • ‘The new rules focus more on delivering outcomes (as opposed to ‘going through the motions’ of a rigid process). Contracting authorities will be able to design their own process, tailored to the unique circumstances of the requirement and, most importantly, those who are best placed to deliver the best solution.

  • There will be clearer rules overall and more flexibility for procurers to use their commercial skills to achieve the desired outcomes.

  • Procurers will be able to better communicate their particular problem to suppliers and work with them to come up with potential solutions. Using product demonstrations alongside written tenders will help buyers get a proper appreciation of solutions being offered by suppliers. That is particularly impactful for newer, more innovative solutions which the authority may not be familiar with.’

Although the guidance document indicates that the ‘new measures include general obligations, options for preliminary market engagement, and an important new mechanism, the Competitive Flexible Procedure’, in practice, there are limited changes to what was already allowed in terms of market consultation and the general obligations— to eg publish a pipeline notice (for contracting authorities with an annual spend over £100 million), or to ‘have regard to the fact that SMEs face barriers to participation and consider whether these barriers can be removed or reduced’—are also marginal (if at all) changes from the still current regime (see regs.48 and 46 PCR 2015). Therefore, it all boils down to the new ‘innovation-friendly procurement processes’ that are enabled by the flexible (under)regulation of the competitive flexible procedure (s.20 PA 2023).

The guidance stresses that the ‘objective is that the Competitive Flexible Procedure removes some of the existing barriers to procuring new and better solutions and gives contracting authorities freedom to enable them to achieve the best fit between the specific requirement and the best the market offers.’ The example provided in the guidance provides the skeleton structure of a 3-phase procedure involving an initial ideas and feasibility phase 1, an R&D and prototype phase 2 and a final tendering leading to the award of a production/service contract (phase 3). At this level of generality, there is little to distinguish this from a competitive dialogue under the current rules (reg.30 PCR 2015). Devil will be in the detail.

Moreover, as repeatedly highlighted from the initial consultations, the under-regulation of the competitive flexible procedure will raise the information costs and risks of engaging with innovation procurement as each new approach taken by a contracting authority will require significant investment of time in its design, as well as an unavoidable risk of challenge. The incentives are not particularly geared towards facilitating risk-taking. And any more detailed guidance on ‘how to'‘ carry out an innovative competitive flexible procedure will simply replace regulation and become a de facto standard through which contracting authorities may take the same ‘going through the motions’ approach as the process detailed in teh guidance rigidifies.

The guidance acknowledges this, at least partially, when it stresses that ‘Behavioural changes will make the biggest difference’. Such behavioural changes will be supported through training, which the guidance document also describes (and there is more detail here). The training offered will consist of:

  • Knowledge drops (open to everyone): An on-demand, watchable resource up to a maximum of 45 minutes in total, providing an overview of all of the changes in legislation.

  • E-learning (for skilled practitioners within the public sector only): a learning & development self-guided course consisting of ‘10 1-hour modules and concludes with a skilled practitioner certification’.

  • Advanced course deep dives (for public sector expert practitioners only): ‘3-day, interactive, instructor-led course. It consists of virtual ‘deep dive’ webinars, which allow learners to engage with subject matter experts. This level of interaction allows a deeper insight across the full spectrum of the legislative change and support ‘hearts and minds’ change amongst the learner population (creating ‘superusers’).’

  • Communities of practice (for skilled and expert practitioners only): ‘a system of collective critical inquiry and reflection into the regime changes. Supported by the central team and superusers, they will support individuals to embed what they have learned.’

As an educator and based on my experience of training expert professionals in complex procurement, I am skeptical that this amount of training can lead to meaningful changes. The 45-minute resource can hardly cover the entirety of changes in the Procurement Act, and even the 10 hour course for public buyers only will be quite limited in how far it can go. 3 days of training are also insufficient to go much further than exploring a few examples in meaningful detail. And this is relevant because that training is not only for innovation procurement, but for all types of ‘different’ procurement under the Procurement Act 2023 (ie green, social, more robustly anti-corruption, more focused on contract performance, etc). Shifting culture and practice would require a lot more than this.

It is also unclear why this (minimal) investment in public sector understanding of the procurement framework has not taken place earlier. As I already said in the consultation, all of this could have taken place years ago and a better understanding of the current regime would have led to improvements in the practice of innovative procurement in the UK.

All in all, it seems that the aspirations of more innovation procurement and more innovative procurement are pinned on a rather limited amount of training and in (largely voluntary, in addition to the day job) collaboration for super-user experienced practitioners (who will probably see their scarce skills in high demand). It is unclear to me how this will be a game changer. Especially as most of this (and in particular collaboration and voluntary knowledge exchange) could already take place. It may be that more structure and coordination will bring better outcomes, but this would require adequate and sufficient resourcing.

Whether there will be more innovation procurement then depends on whether more money will be put into procurement structures and support. From where I stand, this is by no means a given. I guess we’ll have to wait and see.

What's the rush -- some thoughts on the UK's Foundation Model Taskforce and regulation by Twitter

I have been closely following developments on AI regulation in the UK, as part of the background research for the joint submission to the public consultation closing on Wednesday (see here and here). Perhaps surprisingly, the biggest developments do not concern the regulation of AI under the devolved model described in the ‘pro-innovation’ white paper, but its displacement outside existing regulatory regimes—both in terms of funding, and practical power.

Most of the activity and investments are not channelled towards existing resource-strained regulators to support them in their task of issuing guidance on how to deal with AI risks and harms—which stems from the white paper—but in digital industrial policy and R&D projects, including a new major research centre on responsible and trustworthy AI and a Foundation Model Taskforce. A first observation is that this type of investments can be worthwhile, but not at the expense of adequately resourcing regulators facing the tall order of AI regulation.

The UK’s Primer Minister is clearly making a move to use ‘world-leadership in AI safety’ as a major plank of his re-election bid in the coming Fall. I am not only sceptical about this move and its international reception, but also increasingly concerned about a tendency to ‘regulate by Twitter’ and to bullish approaches to regulatory and legal compliance that could well result in squandering a good part of the £100m set aside for the Taskforce.

In this blog, I offer some preliminary thoughts. Comments welcome!

Twitter announcements vs white paper?

During the preparation of our response to the AI public consultation, we had a moment of confusion. The Government published the white paper and an impact assessment supporting it, which primarily amount to doing nothing and maintaining the status quo (aka AI regulatory gap) in the UK. However, there were increasing reports of the Prime Minister’s change of heart after the emergence of a ‘doomer’ narrative peddled by OpenAI’s CEO and others. At some point, the PM sent out a tweet that made us wonder if the Government was changing policy and the abandoning the approach of the white paper even before the end of the public consultation. This was the tweet.

We could not locate any document describing the ‘Safe strategy of AI’, so the only conclusion we could reach is that the ‘strategy’ was the short twitter threat that followed that first tweet.

It was not only surprising that there was no detail, but also that there was no reference to the white paper or to any other official policy document. We were probably not the only ones confused about it (or so we hope!) as it is in general very confusing to have social media messaging pointing out towards regulatory interventions completely outside the existing frameworks—including live public consultations by the government!

It is also confusing to see multiple different documents make reference to different things, and later documents somehow reframing what previous documents mean.

For example, the announcement of the Foundation Model Taskforce came only a few weeks after the publication of the white paper, but there was no mention of it in the white paper itself. Is it possible that the Government had put together a significant funding package and related policy in under a month? Rather than whether it is possible, the question is why do things in this way? And how mature was the thinking behind the Taskforce?

For example, the initial announcement indicated that

The investment will build the UK’s ‘sovereign’ national capabilities so our public services can benefit from the transformational impact of this type of AI. The Taskforce will focus on opportunities to establish the UK as a world leader in foundation models and their applications across the economy, and acting as a global standard bearer for AI safety.

The funding will be invested by the Foundation Model Taskforce in foundation model infrastructure and public service procurement, to create opportunities for domestic innovation. The first pilots targeting public services are expected to launch in the next six months.

Less than two months later, the announcement of the appointment of the Taskforce chair (below) indicated that

… a key focus for the Taskforce in the coming months will be taking forward cutting-edge safety research in the run up to the first global summit on AI safety to be hosted in the UK later this year.

Bringing together expertise from government, industry and academia, the Taskforce will look at the risks surrounding AI. It will carry out research on AI safety and inform broader work on the development of international guardrails, such as shared safety and security standards and infrastructure, that could be put in place to address the risks.

Is it then a Taskforce and pot of money seeking to develop sovereign capabilities and to pilot public sector AI use, or a Taskforce seeking to develop R&D in AI safety? Can it be both? Is there money for both? Also, why steer the £100m Taskforce in this direction and simultaneously spend £31m in funding an academic-led research centre on ethical and trustworthy AI? Is the latter not encompassing issues of AI safety? How will all of these investments and initiatives be coordinated to avoid duplication of effort or replication of regulatory gaps in the disparate consideration of regulatory issues?

Funding and collaboration opportunities announced via Twitter?

Things can get even more confusing or worrying (for me). Yesterday, the Government put out an official announcement and heavy Twitter-based PR to announce the appointment of the Chair of the Foundation Model Taskforce. This announcement raises a few questions. Why on Sunday? What was the rush? Also, what was the process used to select the Chair, if there was one? I have no questions on the profile and suitability of the appointed Chair (have also not looked at them in detail), but I wonder … even if legally compliant to proceed without a formal process with an open call for expressions of interest, is this appropriate? Is the Government stretching the parallelism with the Vaccines Taskforce too far?

Relatedly, there has been no (or I have been unable to locate) official call for expressions of interest from those seeking to get involved with the Taskforce. However, once more, Twitter seems to have been the (pragmatic?) medium used by the newly appointed Chair of the Taskforce. On Sunday itself, this Twitter thread went out:

I find the last bit particularly shocking. A call for expressions of interest in participating in a project capable of spending up to £100m via Google Forms! (At the time of writing), the form is here and its content is as follows:

I find this approach to AI regulation rather concerning and can also see quite a few ways in which the emerging work approach can lead to breaches of procurement law and subsidies controls, or recruitment processes (depending on whether expressions of interest are corporate or individual). I also wonder what is the rush with all of this and what sort of record-keeping will be kept of all this so that it there is adequate accountability of this expenditure. What is the rush?

Or rather, I know that the rush is simply politically-driven and that this is another way in which public funds are at risk for the wrong reasons. But for the entirely arbitrary deadline of the ‘world AI safety summit’ the PM wants to host in the UK in the Fall — preferably ahead of any general election, I would think — it is almost impossible to justify the change of gear between the ‘do nothing’ AI white paper and the ‘rush everything’ approach driving the Taskforce. I hope we will not end up in another set of enquiries and reports, such as those stemming from the PPE procurement scandal or the ventilator challenge, but it is hard to see how this can all be done in a legally compliant manner, and with the serenity. clarity of view and long-term thinking required of regulatory design. Even in the field of AI. Unavoidably, more to follow.

Free registration open for two events on procurement and artificial intelligence

Registration is now open for two free events on procurement and artificial intelligence (AI).

First, a webinar where I will be participating in discussions on the role of procurement in contributing to the public sector’s acquisition of trustworthy AI, and the associated challenges, from an EU and US perspective.

Second, a public lecture where I will present the findings of my research project on digital technologies and public procurement.

Please scroll down for details and links to registration pages. All welcome!

1. ‘Can Procurement Be Used to Effectively Regulate AI?’ | Free online webinar
30 May 2023 2pm BST / 3pm CET-SAST / 9am EST (90 mins)
Co-organised by University of Bristol Law School and George Washington University Law School.

Artificial Intelligence (“AI”) regulation and governance is a global challenge that is starting to generate different responses in the EU, US, and other jurisdictions. Such responses are, however, rather tentative and politically contested. A full regulatory system will take time to crystallise and be fully operational. In the meantime, despite this regulatory gap, the public sector is quickly adopting AI solutions for a wide range of activities and public services.

This process of accelerated AI adoption by the public sector places procurement as the (involuntary) gatekeeper, tasked with ‘AI regulation by contract’, at least for now. The procurement function is expected to design tender procedures and contracts capable of attaining goals of AI regulation (such as trustworthiness, explainability, or compliance with data protection and human and fundamental rights) that are so far eluding more general regulation.

This webinar will provide an opportunity to take a hard look at the likely effectiveness of AI regulation by contract through procurement and its implications for the commercialisation of public governance, focusing on key issues such as:

  • The interaction between tender design, technical standards, and negotiations.

  • The challenges of designing, monitoring, and enforcing contractual clauses capable of delivering effective ‘regulation by contract’ in the AI space.

  • The tension between the commercial value of tailored contractual design and the regulatory value of default clauses and standard terms.

  • The role of procurement disputes and litigation in shaping AI regulation by contract.

  • The alternative regulatory option of establishing mandatory prior approval by an independent regulator of projects involving AI adoption by the public sector.

This webinar will be of interest to those working on or researching the digitalisation of the public sector and AI regulation in general, as the discussion around procurement gatekeeping mirrors the main issues arising from broader trends.

I will have the great opportunity of discussing my research with Aris Georgopoulos (Nottingham), Scott Simpson (Digital Transformation Lead at U.S. Department of Homeland Security), and Liz Chirico (Acquisition Innovation Lead at Office of the Deputy Assistant Secretary of the Army). Jessica Tillipman (GW Law) will moderate the discussion and Q&A.

Registration: https://law-gwu-edu.zoom.us/webinar/register/WN_w_V9s_liSiKrLX9N-krrWQ.

2. ‘AI in the public sector: can procurement promote trustworthy AI and avoid commercial capture?’ | Free in-person public lecture
4 July 2023 2pm BST, Reception Room, Wills Memorial Building, University of Bristol
Organised by University of Bristol Law School, Centre for Global Law and Innovation

The public sector is quickly adopting artificial intelligence (AI) to manage its interactions with citizens and in the provision of public services – for example, using chatbots in official websites, automated processes and call-centres, or predictive algorithms.

There are inherent high stakes risks to this process of public governance digitalisation, such as bias and discrimination, unethical deployment, data and privacy risks, cyber security risks, or risks of technological debt and dependency on proprietary solutions developed by (big) tech companies.

However, as part of the UK Government’s ‘light touch’ ‘pro-innovation’ approach to digital technology regulation, the adoption of AI in the public sector remains largely unregulated. 

In this public lecture, I will present the findings of my research funded by the British Academy, analysing how, in this deregulatory context, the existing rules on public procurement fall short of protecting the public interest.

An alternative approach is required to create mechanisms of external independent oversight and mandatory standards to embed trustworthy AI requirements and to mitigate against commercial capture in the acquisition of AI solutions. 

Registration: https://www.eventbrite.co.uk/e/can-procurement-promote-trustworthy-ai-and-avoid-commercial-capture-tickets-601212712407.

Procuring AI without understanding it. Way to go?

The UK’s Digital Regulation Cooperation Forum (DRCF) has published a report on Transparency in the procurement of algorithmic systems (for short, the ‘AI procurement report’). Some of DRCF’s findings in the AI procurement report are astonishing, and should attract significant attention. The one finding that should definitely not go unnoticed is that, according to DRCF, ‘Buyers can lack the technical expertise to effectively scrutinise the [algorithmic systems] they are procuring, whilst vendors may limit the information they share with buyers’ (at 9). While this is not surprising, the ‘normality’ with which this finding is reported evidences the simple fact that, at least in the UK, it is accepted that the AI field is dominated by technology providers, that all institutional buyers are ‘AI consumers’, and that regulators do not seem to see a need to intervene to rebalance the situation.

The report is not specifically about public procurement of AI, but its content is relevant to assessing the conditions surrounding the acquisition of AI by the public sector. First, the report covers algorithmic systems other than AI—that is, automation based on simpler statistical techniques—but the issues it raises can only be more acute in relation to AI than in relation to simpler algorithmic systems (as the report itself highlights, at 9). Second, the report does not make explicit whether the mix of buyers from which it draws evidence includes public as well as private buyers. However, given the public sector’s digital skills gap, there is no reason to believe that the limited knowledge and asymmetries of information documented in the AI procurement report are less acute for public buyers than private buyers.

Moreover, the AI procurement report goes as far as to suggest that public sector procurement is somewhat in a better position than private sector procurement of AI because there are multiple guidelines focusing on public procurement (notably, the Guidelines for AI procurement). Given the shortcomings in those guidelines (see here for earlier analysis), this can hardly provide any comfort.

The AI procurement report evidences that UK (public and private) buyers are procuring AI they do not understand and cannot adequately monitor. This is extremely worrying. The AI procurement report presents evidence gathered by DRCF in two workshops with 23 vendors and buyers of algorithmic systems in Autumn 2022. The evidence base is qualitative and draws from a limited sample, so it may need to be approached with caution. However, its findings are sufficiently worrying as to require a much more robust policy intervention that the proposals in the recently released White Paper ‘AI regulation: a pro-innovation approach’ (for discussion, see here). In this blog post, I summarise the findings of the AI procurement report I find more problematic and link this evidence to the failing attempt at using public procurement to regulate the acquisition of AI by the public sector in the UK.

Misinformed buyers with limited knowledge and no ability to oversee

In its report, DRCF stresses that ‘some buyers lacked understanding of [algorithmic systems] and could struggle to recognise where an algorithmic process had been integrated into a system they were procuring’, and that ‘[t]his issue may be compounded where vendors fail to note that a solution includes AI or its subset, [machine learning]’ (at 9). The report goes on to stress that ‘[w]here buyers have insufficient information about the development or testing of an [algorithmic system], there is a risk that buyers could be deploying an [algorithmic system] that is unlawful or unethical. This risk is particularly acute for high-risk applications of [algorithmic systems], for example where an [algorithmic system] determines a person's access to employment or housing or where the application is in a highly regulated sector such as finance’ (at 10). Needless to say, however, this applies to a much larger set of public sector areas of activity, and the problems are not limited to high-risk applications involving individual rights, but also to those that involve high stakes from a public governance perspective.

Similarly, DRCF stresses that while ‘vendors use a range of performance metrics and testing methods … without appropriate technical expertise or scrutiny, these metrics may give buyers an incomplete picture of the effectiveness of an [algorithmic system]’; ‘vendors [can] share performance metrics that overstate the effectiveness of their [algorithmic system], whilst omitting other metrics which indicate lower effectiveness in other areas. Some vendors raised concerns that their competitors choose the most favourable (i.e., the highest) performance metric to win procurement contracts‘, while ‘not all buyers may have the technical knowledge to understand which performance metrics are most relevant to their procurement decision’ (at 10). This demolishes any hope that buyers facing this type of knowledge gap and asymmetry of information can compare algorithmic systems in a meaningful way.

The issue is further compounded by the lack of standards and metrics. The report stresses this issue: ‘common or standard metrics do not yet exist within industry for the evaluation of [algorithmic systems]. For vendors, this can make it more challenging to provide useful information, and for buyers, this lack of consistency can make it difficult to compare different [algorithmic systems]. Buyers also told us that they would find more detail on the performance of the [algorithmic system] being procured helpful - including across a range of metrics. The development of more consistent performance metrics could also help regulators to better understand how accurate an [algorithmic system] is in a specific context’ (at 11).

Finally, the report also stresses that vendors have every incentive to withhold information from buyers, both because ‘sharing too much technical detail or knowledge could allow buyers to re-develop their product’ and because ‘they remain concerned about revealing commercially sensitive information to buyers’ (at 10). In that context, given the limited knowledge and understanding documented above, it can even be difficult for a buyer to ascertain which information it has not been given.

The DRCF AI procurement report then focuses on mechanisms that could alleviate some of the issues it identifies, such as standardisation, certification and audit mechanisms, as well as AI transparency registers. However, these mechanisms raise significant questions, not only in relation to their practical implementation, but also regarding the continued reliance on the AI industry (and thus, AI vendors) for the development of some of its foundational elements—and crucially, standards and metrics. To a large extent, the AI industry would be setting the benchmark against which their processes, practices and performance is to be measured. Even if a third party is to carry out such benchmarking or compliance analysis in the context of AI audits, the cards can already be stacked against buyers.

Not the way forward for the public sector (in the UK)

The DRCF AI procurement report should give pause to anyone hoping that (public) buyers can drive the process of development and adoption of these technologies. The AI procurement report clearly evidences that buyers with knowledge disadvantages and information asymmetries are at the merci of technology providers—and/or third-party certifiers (in the future). The evidence in the report clearly suggests that this a process driven by technology providers and, more worryingly, that (most) buyers are in no position to critically assess and discipline vendor behaviour.

The question arises why would any buyer acquire and deploy a technology it does not understand and is in no position to adequately assess. But the hype and hard-selling surrounding AI, coupled with its abstract potential to generate significant administrative and operational advantages seem to be too hard to resist, both for private sector entities seeking to gain an edge (or at least not lag behind competitors) in their markets, and by public sector entities faced with AI’s policy irresistibility.

In the public procurement context, the insights from DRCF’s AI procurement report stress that the fundamental imbalance between buyers and vendors of digital technologies undermines the regulatory role that public procurement is expected to play. Only a buyer that had equal or superior technical ability and that managed to force full disclosure of the relevant information from the technology provider would be in a position to (try to) dictate the terms of the acquisition and deployment of the technology, including through the critical assessment and, if needed, modification of emerging technical standards that could well fall short of the public interest embedded in the process of public sector digitalisation—though it would face significant limitations.

This is an ideal to which most public buyers cannot aspire. In fact, in the UK, the position is the reverse and the current approach is to try to facilitate experimentation with digital technologies for public buyers with no knowledge or digital capability whatsoever—see the Crown Commercial Service’s Artificial Intelligence Dynamic Purchasing System (CCS AI DPS), explicitly targeting inexperienced and digitally novice, to put it politely, public buyers by stressing that ‘If you are new to AI you will be able to procure services through a discovery phase, to get an understanding of AI and how it can benefit your organisation’.

Given the evidence in the DRCF AI report, this approach can only inflate the number of public sector buyers at the merci of technology providers. Especially because, while the CCS AI DPS tries to address some issues, such as ethical risks (though the effectiveness of this can also be queried), it makes clear that ‘quality, price and cultural fit (including social value) can be assessed based on individual customer requirements’. With ‘AI quality’ capturing all the problematic issues mentioned above (and, notably, AI performance), the CCS AI DPS is highly problematic.

If nothing else, the DRCF AI procurement report gives further credence to the need to change regulatory tack. Most importantly, the report evidences that there is a very real risk that public sector entities are currently buying AI they do not understand and are in no position to effectively control post-deployment. This risk needs to be addressed if the UK public is to trust the accelerating process of public sector digitalisation. As formulated elsewhere, this calls for a series of policy and regulatory interventions.

Ensuring that the adoption of AI in the public sector operates in the public interest and for the benefit of all citizens requires new legislation supported by a new mechanism of external oversight and enforcement. New legislation is required to impose specific minimum requirements of eg data governance and algorithmic impact assessment and related transparency across the public sector, to address the issue of lack of standards and metrics but without reliance on their development by and within the AI industry. Primary legislation would need to be developed by statutory guidance of a much more detailed and actionable nature than eg the current Guidelines for AI procurement. These developed requirements can then be embedded into public contracts by reference, and thus protect public buyers from vendor standard cherry-picking, as well as providing a clear benchmark against which to assess tenders.

Legislation would also be necessary to create an independent authority—eg an ‘AI in the Public Sector Authority’ (AIPSA)—with powers to enforce those minimum requirements across the public sector. AIPSA is necessary, as oversight of the use of AI in the public sector does not currently fall within the scope of any specific sectoral regulator and the general regulators (such as the Information Commissioner’s Office) lack procurement-specific knowledge. Moreover, units within Cabinet Office (such as the Office for AI or the Central Digital and Data Office) lack the required independence. The primary role of AIPSA would be to constrain the process of adoption of AI by the public sector, especially where the public buyer lacks digital capacity and is thus at risk of capture or overpowering by technological vendors.

In that regard, and until sufficient in-house capability is built to ensure adequate understanding of the technologies being procured (especially in the case of complex AI), and adequate ability to manage digital procurement governance requirements independently, AIPSA would have to approve all projects to develop, procure and deploy AI in the public sector to ensure that they meet the required legislative safeguards in terms of data governance, impact assessment, etc. This approach could progressively be relaxed through eg block exemption mechanisms, once there is sufficiently detailed understanding and guidance on specific AI use cases, and/or in relation to public sector entities that could demonstrate sufficient in-house capability, eg through a mechanism of independent certification in accordance with benchmarks set by AIPSA, or certification by AIPSA itself.

In parallel, it would also be necessary for the Government to develop a clear and sustainably funded strategy to build in-house capability in the public sector, including clear policies on the minimisation of expenditure directed at the engagement of external consultants and the development of guidance on how to ensure the capture and retention of the knowledge developed within outsourced projects (including, but not only, through detailed technical documentation).

None of this features in the recently released White Paper ‘AI regulation: a pro-innovation approach’. However, DRCF’s AI procurement report further evidences that these policy interventions are necessary. Else, the UK will be a jurisdiction where the public sector acquires and deploys technology it does not understand and cannot control. Surely, this is not the way to go.

Wishful legal analysis as a trade strategy? A rebuttal to the Minister for International Trade

In the context of the Parliamentary scrutiny of the procurement chapters of the UK’s Free Trade Agreements with Australia and New Zealand, I submitted several pieces of written evidence, which I then gathered together and reformulated in A Sanchez-Graells, ‘The Growing Thicket of Multi-Layered Procurement Liberalisation between WTO GPA Parties, as Evidenced in Post-Brexit UK’ (2022) 49(3) Legal Issues of Economic Integration 247–268. I was also invited to submit oral evidence to the Public Bills Comittee for the Trade (Australia and New Zealand) Bill.

In my research, I raised some legal issues on the way the UK-AUS and UK-NZ procurement chapters would interact with the World Trade Agreement Government Procurement Agreement (GPA)—to which UK, AUS and NZ are members—and the Comprehensive and Progressive Agreement for Trans-Pacific Partnership (CPTPP)—to which the UK seeks accession and both AUS and NZ are members. I also raised issues with the rules on remedies in particular, both in relation to UK-AUS and the CPTPP.

I have now become aware of a letter from the Minister for International Trade, where the UK Government simply dismisses my legal analysis in an unconvicing manner. In this post, I try to rebut their position—although their lack of arguments makes this rather difficult—and stress some of the misunderstandings that the letter evidences. The letter seems to me to reflect a worrying strategy of ‘wishful legal analysis’ that does not bode well for post-Brexit UK trade realignment.

Interaction between the GPA, FTAs and the CPTPP

In my analysis and submissions, I stressed how deviations in the UK’s FTAs from the substantive obligations set in the GPA generate legal uncertainty and potential problems in ‘dual regulation’ situations, where one of the contracting parties (eg the UK) would be under the impossibility of complying at the same time with the obligations resulting from the GPA with tenderers from GPA countries and those arising from the FTAs with AUS or NZ with their tenderers—without either breaching GPA obligations or, what is more likely, ignoring the deviation in the FTAs to ensure GPA compliance. It would also generate issues where compliance with the more demanding standards in the FTAs would be automatically propagated to the benefit of economic operators from other jurisdictions. I also raised how the deviations can generate legal uncertainty and make it more difficult for UK tenderers to ascertain their legal position in AUS and NZ. And I also raised how this situation can get further complicated if the UK accesses CPTPP.

My concerns were discussed in Committee and the Minister had the following to say:

The [GPA] and the [CPTPP] are plurilateral agreements between twenty-one and eleven parties respectively, including in each case Australia and New Zealand. As recognised in Committee, the [GPA] in particular establishes a global baseline for international procurement. Nonetheless, neither prevents its members from entering into bilateral free trade agreements to sit alongside the [GPA] and [CPTPP] while at the same time going further in terms of the procurement commitments between members.

These Agreements with Australia and New Zealand do just that, going beyond both the [GPA] and the [CPTPP] baselines. … Although the texts of the Agreements with Australia and New Zealand are sometimes laid out differently to the way they are in the Agreement on Government Procurement, they in no way dilute or reduce the global baseline established by the [GPA]. (emphases added).

There are two points to note, here. The first one is that the fact that the GPA and the CPTPP allow for bilateral agreements between their parties does not clarify how the overlapping treaties would operate, which is exactly what I analysed. Of note, under the 1969 Vienna Convention on the Law of Treaties (Art 30), when States conclude successive treaties relating to the same subject matter, the most recent treary prevails, and the provisions of the earlier treaty/ies only apply to the extent that they are not incompatible with those of the later treaty.

This is crucial here, especially as the Minister indicates that the UK-AUS and UK-NZ go beyond not only the GPA, but also the CPTPP. This would mean that entering into CPTPP after UK-AUS and UK-NZ—as the UK is currently in train of doing—could negate some of the aspects that go beyond CPTPP in both those FTAs. Moreover, the simple assertion that the FTAs do not dilute the GPA baseline is unconvincing, as detailed analysis shows that there are significant problems with eg the interpretation of the national treatment under the different treaties.

Secondly, the explanation provided does not resolve the practical problems arising from ‘dual regulation’ that I have identified and leaves the question open as to how the obligations under the FTAs will be interpreted and complied with in triangular situations involving tenderers not from AUS or NZ. Either the UK will apply the more demanding obligations—which will then benefit all GPA parties, not only AUS and NZ—or will stick to the GPA baseline in breach of the FTAs. There is no recognition of this issue in the letter.

The Minister also indicated that:

There was also suggestion in Committee that it would be difficult for suppliers in the United Kingdom to navigate the Agreements with Australia and New Zealand, as well as the [CPTPP] in the future. I would like to reassure the Committee that when bidding for United Kingdom procurements, the only system that British suppliers need to concern themselves with is United Kingdom’s procurement regulations. (emphasis added).

The Minister has either not understood the situation, or is seeking to obscure the analysis here. The concerns about legal uncertainty do not relate to UK businesses tendering for contracts in the UK, but to UK businesses tendering for contracts in AUS or NZ—which are the ones that would be seeking to benefit from the trade liberalisation pursued by those FTAs. Nothing in the Minister’s letter addresses this issue.

Domestic review rights under the Australian procurement chapter

One of the specific deviations from the GPA baseline that I identified in my research concerns the exclusion of access to remedies on grounds of public interest. While the GPA only allows excluding interim measures on such grounds, the AUS-UK FTA and CPTPP seem to allow for public interest to also bar access to remedies such as compensation—and, if this does not limit access to remedies as I submit, at least it does cause legal uncertainty in that respect.

My submission is met with the following response by the Minister [the mentioned annex is reproduced at the end of this post]:

The Committee also considered the evidence raised by Professor Sánchez-Graells regarding domestic review procedures … The Government respectfully disagrees with the analysis presented at that session that a provision in the government procurement chapter of the [UK-AUS FTA] ‘allows for the exclusion of legal remedies completely on the basis of public interest’.

The public interest exclusion only applies to temporary measures put in place to ensure aggrieved suppliers may continue to participate in a procurement.

…

The Government also respectfully disagrees with the suggestion in the witness evidence that this public interest exclusion is not similarly reflected in the [GPA] or the [UK-NZ FTA]. The Government acknowledges that the specific position of the exclusion differs between these agreements and is closer to the approach adopted in the [CPTPP]. Nonetheless, the Government do not consider this alters the legal effect or gives rise to legal uncertainty. For the benefit of the Committee, the relevant provisions from each of the [FTAs], the [GPA] and the [CPTPP] are set out in an annex to this letter.

The Minister’s explanations are not supported by any arguments. There is no reasoning to explain why the order of the clauses and subclauses in the relevant provisions does not alter their legal interpretation or effects. There is also no justification whatsoever for the opinion that textual differences do not give rise to legal uncertainty. The Government seems to think that it can simply wish the legal issues away.

The table included in the annex to the letter (below) is revealing of the precise issue that raises legal uncertainty and, potentially, a restriction on access to remedies other than interim measures beyond the GPA (and thus, in breach thereof). Why would treaties that seem to replicate the same rules draft them differently? How can any legal interpreter be of the opinion that the positioning of the exception clause does not have an effect on the interpretation of its scope of application? Is the fact that these agreements post-date the GPA and still deviate from it not of legal relevance?

Of course, there are arguments that could be made to counter my analysis. They could eg focus on the use of different (undefined) terms in different sub-clauses (such as ‘measures’ and ‘corrective action’). They could also focus on any preparatory works to the treaties (especially the CPTPP and UK-AUS FTA, which I have not yet been able to locate). They could even be more creative and attempt functional or customary interpretation arguments. But the letter contains no arguments at all.

Conclusion

It is a sad state of affairs where detailed legal analysis—whether correct or not—is dismissed without offering any arguments to the contrary and simply seeking to leverage the ‘authority’ of a Minister or Department. If this is the generalised approach to assessing the legal implications of the trade agreements negotiated (at speed) by the UK post-Brexit, it does not bode well for the legal certainty required to promote international investments and commercial activities.

The reassurances in the letter are void of any weight, in my view. I can only hope that the Committee is not persuaded by the empty explanations contained in the letter either.



New paper on competition and procurement regulation -- in memory of Professor Steen Treumer

Image credits: Steve Johnson.

Last year brought the saddest news with the passing of Professor Steen Treumer after a long illness. Steen was a procurement colossus and a fantastic academic. I was extremely lucky to count him amongst my mentors and champions, especially at the very early stages of my research and academic career, before he had to take a step back to focus on his health. I am particularly grateful to him for having opened the door of the European Procurement Law Group to me. And for his generosity in providing feedback, job and promotion references, and thoughtful and clever advice without ever asking for or expecting anything in return.

It is nigh impossible to do justice to the intellectual contribution Steen made to the procurement field and the influence his approach had on the research of others such as myself. It is now a humbling honour to have been invited to contribute to an edited collection in his memory (a Mindeskrift). If he could read my contribution, I am not sure Steen would agree with what I say in the paper, but we would certainly have an interesting and stimulating discussion on the basis of the sharp comments (even some devil’s advocate ones) he would surely come up with. I hope you will find the contribution worth discussing too.

Probably unsurprisingly, the paper is entitled ‘Competition and procurement regulation: a goal, a principle, a requirement, or all of the above?’ and its abstract is below. In the paper, I use the background of recent developments in UK and EU case law, as well as the UK’s procurement rulebook reform process, to reframe the issue of the role of competition in procurement regulation. While I do not provide any insights I had not already developed in earlier writing, I bring some scattered parts of my scholarship together and hopefully clarify a few things along the way. The paper may be particularly interesting to those looking for an entry point to the discussion on the role of competition in public procurement, but hopefully there is also something for those already well versed on the topic. As always, comments most welcome: a.sanchez-graells@bristol.ac.uk.

In this contribution, I reflect on the role of competition in public procurement regulation and, more specifically, on whether competition should be treated as a regulatory goal, as a general principle of public procurement law, as a specific (implicit or explicit) requirement in discrete legal provisions, or all of the above. This is an issue I had the pleasure and honour of discussing with Professor Steen Treumer back in 2009, when I was a PhD student visiting the Copenhagen Business School. While Steen never revealed to me what he really thought, his probing questions continue to help me think of this issue, which remains at the core of my research efforts. This contribution shows that the role of competition keeps cropping up in procurement regulation and litigation, as evidenced in recent UK developments. This is thus an evergreen research topic, which were Steen’s favourites.

The full citation is: Sanchez-Graells, Albert, ‘Competition and procurement regulation: a goal, a principle, a requirement, or all of the above?’, to be published in Steen Treumer’s Mindeskrift edited by Carina Risvig Hamer, Erik Bertelsen, Marta Andhov, and Roberto Caranta (Ex Tuto Publishing, forthcoming 2022). Available at SSRN: https://ssrn.com/abstract=4012022.

Interesting twist on the interpretation of extremely urgent procurement rules -- re [2022] EWHC 46 (TCC)

One of the most awaited court decisions in the PPE procurement litigation saga in the UK was handed down yesterday—see R (Good Law Project and EveryDoctor) v Secretary of State for Health and Social Care [2022] EWHC 46 (TCC). The case concerned, among other things, the interpretation of the authorisation to use a negotiated procedure without prior publication on grounds of extreme urgency, and its limits, under reg.32(2)(c) and 32(4) of the Public Contracts Regulations 2015 (‘PCR2015’), which transpose Art 32(2)(c) of Directive 2014/14/EU verbatim.

The case required an EU law conforming interpretation due to the procurement predating the end of the Brexit transition period (see para [308]). The High Court thus engaged in an analysis of CJEU case law and a functional interpretation of reg.32(2)(c) and 32(4) PCR2015 that is directly of interest regarding the interpretation of Art 32(2)(c) Dir 2014/14/EU (on which see P Bogdanowicz, ‘Article 32’ in R Caranta and A Sanchez-Graells, European Public Procurement. Commentary on Directive 2014/24/EU (Edward Elgar, 2021) 350-362]. There are two points worth highlighting in the Judgment (see also Pedro Telles’ hot take yesterday).

First, the High Court confirmed the ‘blanket approach’ interpretation that the pandemic, in its early stages, was itself sufficient justification to ‘deactivate’ procurement rules through the exception to competitive requirements in reg.32(2)(c) and 32(4) PCR2015 / Art 32(2)(c) Dir 2014/24’EU (paras [329]-[331]). This has been the position of the UK Cabinet Office and the European Commission in their ‘pandemic procurement’ guidelines of March and April 2020, respectively, and one that I share (see A Sanchez-Graells, ‘Procurement in the time of Covid-19’ (2020) 71(1) NILQ 81-87, at 83; see also Bogdanowicz, above, at 32.23, contra Telles, above).

Second, and more interesting, the High Court considered whether the authorisation to carry out a negotiated procedure without prior publication is still subject to some of the requirements of the PCR2015 (and, by analogy, Directive 2014/24/EU). The High Court found that, under certain circumstances, extremely urgent procurement is still bound to respect the equal treatment requirement of reg.18 PCR2015 / Art 18 Dir 2014/24/EU. The High Court’s reasoning was that

It is … necessary to consider whether there are any constraints on the permissible approach by a contracting authority when acting under regulation 32; in particular, whether there is an irreducible minimum standard of objective fairness that applies to such procurements, even in the absence of open competition (at [334], emphasis added).

and that

Regulation 18 provides that contracting authorities shall treat economic operators equally and without discrimination and shall act in a transparent and proportionate manner. Regulation 32 does not expressly disapply the obligations set out in regulation 18. … the question that arises is whether there is any implicit exclusion, or modification, of this provision arising from operation of the negotiated procedure without notice (at [340], emphasis added).

Within this framework, and taking into account the peculiar circumstances of the case — ie the fact that the UK Government ‘operated a high priority lane (“the High Priority Lane”, also referred to as … “the VIP Lane”), whereby suppliers who had been referred by Ministers, [Members of Parliament] and senior officials were afforded more favourable treatment, significantly increasing their prospects of being awarded a contract or contracts’ (at [4]) — the High Court established that

It is reasonably clear that where there is only one economic operator who can provide the works, supplies or services, the principle of equal treatment can have no application. Where there is no alternative source, there will be no comparative exercise carried out and no question of any discrimination arises. However, where the contracting authority considers bids from more than one economic operator, whether at the same or at different times, there is no obvious rationale for disregarding the principle of equal treatment in terms of the criteria used to decide which bidders should be awarded a contract. Dispensing with a competition does not justify arbitrary or unfair selection criteria where more than one economic operator could satisfy the demand (at [341]).

I have two comments here. The first one is that the analysis at para [341] is partially flawed when it initially refers to the existence of a single supply source, as that is covered by the grounds in reg.32(2)(b) PCR2015 / Art 32(2)(b) Dir 2014/24/EU. A proper analysis under ground (c) on extreme urgency should have triggered a different logic, as the presence of extreme urgency allows contracting authorities to simply choose a provider regardless of the existence of alternative providers, precisely because the supply, works or services are so urgent that there is no time to consider alternatives. The choice of the specific supplier to which the contract will be awarded is discretionary, and subject only to documentary requirements primarily concerned with the concurrence of the circumstances justifying the use of the negotiated procedure without prior publication (see Sanchez-Graells, above, 83).

If this premise is correct, on the basis of a maiore ad minus logic, the argument is difficult to extend to a situation where the contracting authority makes repeated choices for the direct award of contracts. That does not mean that unequal treatment is allowed, but rather that the source of the requirement for equal treatment can hardly be found in reg.18 PCR2015 / Art 18 Dir 2014/24/EU in relation to reg.32(2)(c) PCR2015 / Art 32(2)(c) Dir 2014/24/EU because its exclusion is implicit in the authorisation to directly and discretionarily choose the economic operator to be tasked with the extremely urgent supply, service provision or works—regardless of whether there is only one possible source or not, as that is covered in ground (b) of those rules instead.

The High Court dismissed this argument as follows:

The Defendant submits that, as he was not constrained to implement any competitive tender process, it was lawful for the Defendant to elect to approach an economic operator of his choice and negotiate directly with such economic operator for the purposes of awarding any individual public contract. In those circumstances, it is submitted, the principle of equal treatment did not apply. In my judgment that submission goes too far. It would be open to the Defendant to justify the selection of one economic operator but only: (i) where he could bring himself within the conditions set out in regulation 32(2)(b), for example where only one economic operator could source the required PPE; or (ii) where he could justify the extent of such derogation from the principles in regulation 18 under regulation 32(2)(c), for example where only one economic operator could source the PPE within the required timescale. That interpretation is consistent with the guidance issued by the European Commission on 1 April 2020 [at [346]).

I submit that the legal analysis of the High Court in this point is incorrect, simply because there is no single source requirement in reg.32(2)(c) PCR2015 (or in Art 32(2)(c) Dir 2014/24), even if this can be a matter of policy, as reflected in the European Commission’s guidance (at 1 and 2.3). And the absence of a sole source requirement is entirely justified on operational grounds. Imagine a situation where the contracting authority with the extremely urgent need identifies a potential provider and successfully and quickly reaches an agreement to get its urgent need satisfied. It would defy all logic to require the contracting authority to then check whether ‘only [that] undertaking is able to deliver within the technical and time constraints imposed by the extreme urgency‘ (in terms of the Commission’s guidance) and, if not, then engage with additional negotiations with the other/s, which would only generate further delay in getting the extremely urgent (public) need satisfied. Sole source requirements simply make no sense in this setting. In fact, the Commission’s guidance was (contradictorily?) clear that ‘as set out in Art. 32 of Directive 2014/24/EU (the ‘Directive’), public buyers may negotiate directly with potential contractor(s) and there are no publication requirements, no time limits, no minimum number of candidates to be consulted, or other procedural requirements. No procedural steps are regulated at EU level. In practice, this means that authorities can act as quickly as is technically/physically feasible – and the procedure may constitute a de facto direct award only subject to physical/technical constraints related to the actual availability and speed of delivery‘ (emphasis added), with this requirement logically only meaning that the awardee of the contract needs to be able to actually deliver at speed (which was the flaw with eg the ventilator challenge, see here).

Conflating both requirements constitutes an improper interpretation that runs contrary to the CJEU case law on extreme urgency grounds for the use of the negotiated procedure without prior publication. This may seem like a technical point, but I think it is important. It is also a rather unnecessary point for the High Court to have made, as the Judgment does not rest on it. At paras [348] and [350], the Court is clear that the equal treatment requirement emerged from the way in which the discretion was exercised, because the VIP Lane created a procedure that was structurally and unavoidably discriminatory.

Linked to that, my second comment is that the exclusion of reg.18 by reg.32(2)(c) PCR2015 (and EU equivalents) should not have pre-empted the finding of an ‘irreducible minimum standard of objective fairness’ in the organisation of a system to make repeated or multiple direct awards in the context of an extremely urgent need (the VIP Lane). However, such requirements should derive from general administrative law rules or principles and, in particular in the context of procurement covered (and authorised to be carried out via a negotiated procedure without prior publication) by EU law, from the duty of good administration in Article 41 of the Charter of Fundamental Rights of the EU (‘Charter’) — although, admittedly, the relevance of Art 41 Charter to procurement carried out by the Member States is controversial (in favour, AG Sharpston, Opinion in Varec, C-450/06, EU:C:2007:643, at 43; cfr. AG Bobek, Opinion in HUNGEOD, C‑496/18 and C‑497/18, EU:C:2019:1002, at 50).

And, although I am not an expert in UK public law, I would also have thought that general requirements of administrative decision-making should apply to that effect, such as the requirement for decision-makers to consider all issues which are relevant to a decision and not to consider any issues which are not [for discussion in the context of automated decision-making, and with references to case law, see J Cobbe, ‘Administrative law and the machines of government: judicial review of automated public-sector decision-making’ (2019) 39 Legal Studies 636-655, at 650]. However, the High Court also dismissed this argument, although seemingly on the specific factual circumstances of the case (at [456]-[459]).

So it could be that the stringency of the English case law’s approach to the control of objectivity in administrative decision-making provides an explanation for the, in my view, improper interpretation of the requirements that can be attached to procurement via a negotiated procedure without prior publication on grounds of extreme urgency. Whether the CJEU is likely to follow a similar approach to the imposition of equal treatment requirements in the interpretation of Art 32(2)(c) Dir 2014/24/EU in the future is thus difficult to assess.

Where does the proposed EU AI Act place procurement?

Thinking about some of the issues raised in the earlier post ‘Can the robot procure for you?,’ I have now taken a close look at the European Commission’s Proposal for an Artificial Intelligence Act (AIA) to see how it approaches the use of AI in procurement procedures. It may (not) come as a surprise that the AI Act takes an extremely light-touch approach to the regulation of AI uses in procurement and simply subjects them to (yet to be developed) voluntary codes of conduct. I will detail my analysis of why this is the case in this post, as well as some reasons why I do not find it satisfactory.

Before getting to the details, it is worth stressing that this is reflective of a broader feature of the AIA: its heavy private sector orientation. When it comes to AI uses by the public sector, other than prohibiting some massive surveillance by the State (both for law enforcement and to generate a system of social scoring) and classifying as high-risk the most obvious AI uses by the law enforcement and judicial authorities (all of which are important, of course), the AIA remains silent on the use of AI in most administrative procedures, with the only exception of those concerning social benefits.

This approach could be generally justified by the limits to EU competence and, in particular, those derived from the principle of administrative self-organisation of the Member States. However, given the very broad approach taken by the Commission on the interpretation and use of Article 114 TFEU (which is the legal basis for the AIA, more below), this is not entirely consistent. It could rather be that the specific uses of AI by the public sector covered in the proposal reflect the increasingly well-known problematic uses of (biased) AI solutions in narrow aspects of public sector activity, rather than a broader reflection on the (still unknown, or still unimplemented) uses that could be problematic.

While the AIA is ‘future-proofed’ by including criteria for the inclusion of further use cases in its ‘high-risk’ category (which determines the bulk of compliance obligations), it is difficult to see how those criteria are suited to a significant expansion of the regulatory constraints to AI uses by the public sector, including in procurement. Therefore, as a broader point, I submit that the proposed AIA needs some revision to make it more suited to the potential deployment of AI by the public sector. To reflect on that, I am co-organising a webinar on ’Digitalization and AI decision-making in administrative law proceedings’, which will take place on 15 Nov 2021, 1pm UK (save the date, registration and more details here). All welcome.

Background on the AIA

Summarising the AIA is both difficult and has already been done (see eg this quick explainer of the Centre for Data Innovation, and for an accessible overview of the rationale and regulatory architecture of the AIA, this master class by Prof Christiane Wendehorst). So, I will just highlight here a few issues linked to the analysis of procurement’s position within its regulatory framework.

The AIA seeks to establish a proportionate approach to the regulation of AI deployment and use. While its primary concern is with the consolidation of the EU Single Digital Market and the avoidance of regulatory barriers to the circulation of AI solutions, its preamble also points to the need to ensure the effectiveness of EU values and, crucially, the fundamental rights in the Charter of Fundamental Rights of the EU.

Importantly for the purposes of our discussion, recital (28) AIA stresses that ‘The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include ... right to an effective remedy and to a fair trial [Art 47 Charter] … [and] right to good administration {Art 41 Charter]’.

The AIA seeks to create such a proportionate approach to the regulation of AI by establishing four categories of AI uses: prohibited, high-risk, limited risk requiring transparency measures, and minimal risk. The two categories that carry regulatory constraints or compliance obligations are those concerning high-risk (Arts 8-15 AIA), and limited risk requiring transparency measures (Art 52 AIA, which also applies to some high-risk AI). Minimal risk AI uses are left unregulated, although the AIA (Art 69) seeks to promote the development of codes of conduct intended to foster voluntary compliance with the requirements applicable to high-risk AI systems.

Procurement within the AIA

Procurement AI practices could not be classified as prohibited uses (Art 5 AIA), except in the difficult to imagine circumstances in which they deployed subliminal techniques. It is also difficult to see how they could fall under the regime applicable to uses requiring special transparency (Art 52) because it only applies to AI systems intended to interact with natural persons, which must be ‘designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use.’ It would not be difficult for public buyers using external-facing AI solutions (eg chatbots seeking to guide tenderers through their e-submissions) to make it clear that the tenderers are interacting with an AI solution. And, even if not, the transparency obligations are rather minimal.

So, the crux of the issue rests on whether procurement-related AI uses could be classified as high-risk. This is regulated in Art 6 AIA, which cross-refers to Annex III AIA. The Annex contains a numerus clausus of high-risk AI uses, which is however susceptible of amendment under the conditions specified in Art 7 AIA. Art 6/Annex III do not contain any procurement-related AI uses. The only type of AI use linked to administrative procedures concerns ‘AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for public assistance benefits and services, as well as to grant, reduce, revoke, or reclaim such benefits and services’ (Annex III(5)(a) AIA).

Clearly, then, procurement-related AI uses are currently left to the default category of those with minimal risk and, thus, subjected only to voluntary self-regulation via codes of conduct.

Could this change in the future?

Art 7 AIA establishes the following two cumulative criteria: (a) the AI systems are intended to be used in any of the areas listed in points 1 to 8 of Annex III; and (b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

The first hurdle in getting procurement-related AI uses included in Annex III in the future is formal and concerns the interpretation of the categories listed therein. There are only two potential options: nesting them under uses related to ‘Access to and enjoyment of essential private services and public services and benefits’, or uses related to ‘Administration of justice and democratic processes’. It could (theoretically) be possible to squeeze them in one of them (perhaps the latter easier than the former), but this is by no means straightforward and, given the existing AI uses in each of the two categories, I would personally be disinclined to engage in such broad interpretation.

Even if that hurdle was cleared, the second hurdle is also challenging. Art 7(2) AIA establishes the criteria to assess that an AI use poses a sufficient ‘risk of adverse impact on fundamental rights’. Of those criteria, there are three that in my view would make it very difficult to classify procurement-related AI uses as high-risk. Those criteria require the European Commission to consider:

(c) the extent to which the use of an AI system has already caused … adverse impact on the fundamental rights or has given rise to significant concerns in relation to the materialisation of such … adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons;

(e) the extent to which potentially harmed or adversely impacted persons are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

(g) the extent to which the outcome produced with an AI system is easily reversible …;

Meeting these criteria would require for the relevant AI systems to basically be making independent or fully automated decisions (eg on award of contract, or exclusion of tenderers), so that their decisions would be seen to affect the effectiveness of Art 41 and 47 Charter rights; as well as a (practical) understanding that those decisions cannot be easily reversed. Otherwise, the regulatory threshold is so high that most likely procurement-related AI uses (screening, recommender systems, support to human decision-making (eg automated evaluation of tenders), etc) are unlikely to be considered to pose a sufficient ‘risk of adverse impact on fundamental rights’.

Could Member States go further?

As mentioned above, one of the potential explanations for the almost absolute silence on the use of AI in administrative procedures in the AIA could be that the Commission considers that this aspect of AI regulation belongs to each of the Member States. If that was true, then Member States could further than the code of conduct self-regulatory approach resulting from the AIA regulatory architecture. An easy approach would be to eg legally mandate compliance with the AIA obligations for high-risk AI systems.

However, given the internal market justification of the AIA, to be honest, I have my doubts that such a regulatory intervention would withstand challenges on the basis of general EU internal market law.

The thrust of the AIA competential justification (under Art 114 TFEU, see point 2.1 of the Explanatory memorandum) is that

The primary objective of this proposal is to ensure the proper functioning of the internal market by setting harmonised rules in particular on the development, placing on the Union market and the use of products and services making use of AI technologies or provided as stand-alone AI systems. Some Member States are already considering national rules to ensure that AI is safe and is developed and used in compliance with fundamental rights obligations. This will likely lead to two main problems: i) a fragmentation of the internal market on essential elements regarding in particular the requirements for the AI products and services, their marketing, their use, the liability and the supervision by public authorities, and ii) the substantial diminishment of legal certainty for both providers and users of AI systems on how existing and new rules will apply to those systems in the Union.

All of those issues would arise if each Member State adopted its own rules constraining the use of AI for administrative procedures not covered by the AIA (either related to procurement or not), so the challenge to that decentralised approach on grounds of internal market law by eg providers of procurement-related AI solutions capable of deployment in all Member States but burdened with uneven regulatory requirements seems quite straightforward (if controversial), especially given the high level of homogeneity in public procurement regulation resulting from the 2014 Public Procurement Package. Not to mention the possibility of challenging those domestic obligation on grounds that they go further than the AIA in breach of Art 16 Charter (freedom to conduct a business), even if this could face some issues resulting from the interpretation of Art 51 thereof.

Repositioning procurement (and other aspects of administrative law) in the AIA

In my view, there is a case to be made for the repositioning of procurement-related AI uses within the AIA, and its logic can apply to other areas of administrative law/activity with similar market effects.

The key issue is that the development of AI solutions to support decision-making in the public sector not only concerns the rights of those directly involved or affected by those decisions, but also society at large. In the case of procurement, eg the development of biased procurement evaluation or procurement recommender systems can have negative social effects via its effects on the market (eg on value for money, to mention the most obvious) that are difficult to identify in single tender procurement decisions.

Moreover, it seems that the public administration is well-placed to comply with the requirements of the AIA for high-risk AI systems as a matter of routine procedure, and the arguments on the need to take a proportionate approach to the regulation of AI so as not to stifle innovation lose steam and barely have any punch when it comes to imposing them on the public sector user. Further, to a large extent, the AIA requirements seem to me mostly aligned with the requirements for running a proper (and challenge proof) eProcurement system, and they would also facilitate compliance with duties of good administration when specific decisions are challenged.

Therefore, on balance, I see no good reason not to expand the list in Annex III AIA to include the use of AI systems in all administrative procedures, and in particular in public procurement and in other regulatory sectors where ex post interventions to correct market distortions resulting from biased AI implementations can simply be practically impossible. I submit that this should be done before its adoption.

Emerging technologies and anti-corruption efforts -- re Adam and Fazekas (2021)

(c) Sara Alaica/Flickr.

(c) Sara Alaica/Flickr.

I am working on a paper on digital technologies and corruption in procurement (or rather, trying to work on it in the midst of a challenging start of term). While researching this topic, I have come across this very interesting paper: Isabelle Adam and Mihály Fazekas, ‘Are emerging technologies helping win the fight against corruption? A review of the state of evidence’ (2021) Information Economics and Policy, available on pre-print here.

In their paper, Adam & Fazekas carry out a systematic review ‘of the academic and policy literature on the six most commonly discussed types of ICT-based anti-corruption interventions: (i) Digi- tal public services and e-government, (ii) Crowdsourcing platforms, (iii) Whistleblowing tools, (iv) Transparency portals and big data, (v) DLT and blockchain, and (vi) AI’ (at 2).

The analysis is clear and accessible and offers good insights on the positive and negative impacts that digital technologies can have for anti-corruption efforts, given that technology ‘is not per se a panacea against corruption, and it can also play into the hands of corrupt officials’ (ibid). The paper is well worth reading in full.

One of their insights I found particularly valuable is that ‘ICTs for anti-corruption operate against the background of given societal divides and power relations which are often supported by corruption. They risk further entrenching these unless their design and implementation take into account corruption and associated power imbalances. Hence, it is arguable that the success of ICT interventions against corruption hinges on their suitability for local contexts and needs, cultural backgrounds and technological experience‘ (at 1).

This directly links with Uta Kohl’s view that digital ‘technologies, whether the internet or blockchain, are tightly and on multiple levels interconnected with existing social orders and those interconnections decide upon the configurational latencies of the technological innovation within concrete settings: who uses the technological innovation in what configuration, for what purposes and against whom’ (see here for details).

To my mind, all of this stresses the need to operationalise a gatekeeping function tasked with the analysis of which digital technologies are adopted by the public sector and for what purpose, and this gatekeeping function needs not only consider downstream ethical implications in terms of impacts on citizens and service users, but also upstream implications concerning the way in which technologies will disrupt, support or entrench existing governance dynamics — and in particular those that the adoption of the technology is seeking to remedy.

Bringing this to procurement, these insights show that the public procurement function — to the extent that the adoption of these technologies is subjected to the regulatory framework of innovation procurement — is de facto playing (or failing to play) such gatekeeping function. More than in other settings, the procurement function needs to closely scrutinise the ‘use case’ of the digital technologies it is tasked with procuring. This is arguably a new regulatory function for procurement, and one that is not yet embedded in procurement theory, regulation or practice. But one that is inescapable nonetheless. So one that is worth thinking about.

Healthcare procurement: a service of general economic interest?

With thanks to Dr Mary Guy (Lancaster University) for the invitation to speak at her innovative ‘Health in Europe - Virtual Discussion Forum’, below is the recording of my presentation on the treatment of healthcare procurement as a service of general economic interest. The slides are also available.

The presentation explores the case study of the English NHS Supply Chain (for a detailed account of how it works, please see here). However, broader issues of potential relevance in EU jurisdictions considering ways of reforming (and centralising) healthcare procurement are also explored.

This is work in progress for me, so comments most welcome: a.sanchez-graells@bristol.ac.uk.

As a side note, it is worth stressing that NHS Supply Chain is currently under fire due to its failure to react properly to the PPE challenges derived from the COVID-19 crisis after a scathing National Audit Office report (on which you can watch some comments here).

Two new working papers on procurement & COVID-19

I have uploaded two new, short working papers on procurement and COVID-19 on SSRN. Comments most welcome: a.sanchez-graells@bristol.ac.uk.

  1. Procurement and Commissioning during COVID-19: Reflections and (Early) Lessons (October 8, 2020). Northern Ireland Legal Quarterly, forthcoming. Available at SSRN: https://ssrn.com/abstract=3709746.
    Abstract: This piece reflects on some common themes that are starting to emerge in the early analysis of the healthcare procurement and commissioning response to the COVID-19 pandemic. Although it largely results from the observation of the situation in the English NHS, the most salient issues are common to procurement in other EU healthcare systems, as well as more broadly across areas of the public sector that have strongly relied on the extremely urgent procurement exception in the aftermath of the first wave of the pandemic. Given the disfunction and abuse of ‘unregulated procurement’ in the context of COVID-19, the piece reflects on the longer term need for suitable procurement rules to face impending challenges, such as Brexit and, more importantly, climate change.

  2. COVID-19 PPE Extremely Urgent Procurement in England. A Cautionary Tale for an Overheating Public Governance (October 14, 2020). To be published in D Cowan and Ann Mumford (eds), Pandemic Legalities (Bristol University Press, forthcoming). Available at SSRN: https://ssrn.com/abstract=3711526.
    Abstract: In this short paper, I reflect on the case study of the procurement of personal protective equipment (PPE) for the English NHS during the first wave of the COVID-19 pandemic. I put forward two main claims. My first claim is that the UK Government not only was particularly ill-positioned to deal with the pandemic as a result of years of austerity and the institutional unsettling resulting from the continuous reform of the NHS, its internal market and its supply chain—but also due to the imminence of Brexit and its political ramifications. My second contribution is that, in its desperate reaction to the PPE fiasco, the UK Government misused and abused the disapplication of the standard procurement rules on the basis of the ‘extremely urgent need’ exemption. This resulted in the opaque award of large numbers of high value contracts to companies that would not survive basic screening under normal conditions. Overall, my goal is to lay bare the more general problems in the UK Government’s approach to the governance of public procurement and its increasing insularity as a result of Brexit, with the hope that this will show a path for change that could avert even more significant fiascos in the face of the massive challenges that climate change will bring.

NHS commissioning and procurement - 2 short lectures and a reading list

I have recorded a series of short lectures on NHS commissioning and procurement for my blended teaching at the University of Bristol Law School this coming academic year. In case they are of any interest, I am sharing two of them here.

The first one covers the organisation and regulation of NHS commissioning and procurement and primarily concentrates on the commissioning of health care services. The second lecture covers the centralisation of ‘hospital procurement’ through the NHS Supply Chain. They should be accessible through the click-through images at the end of the blog post.

The two short lectures aim to provide a (hopefully) accessible introduction to the issues covered in more detail in the accompanying reading list, which mainly comprises the following papers for each of the topics:

1. Organisation and regulation of NHS internal market, with a focus on commissioning and procurement

  • A Maynard and M Dixon, ‘Should the NHS abolish the purchaser-provider split?’, BMJ 2016;354:i3825, available at https://doi.org/10.1136/bmj.i3825.

  • C Paton, ‘Garbage-Can Policy-Making Meets Neo-Liberal Ideology: Twenty-five years of redundant reform of the English National Health Service’ (2014) 48(3) Social Policy & Administration 319-342.

  • L Jones, M Exworthy and F Frosini, ‘Implementing Market-based Reforms in the English NHS: Bureaucratic coping strategies and social embeddedness’ (2013) 111(1) Health Policy 52-59.

  • B Collins, ‘Procurement and Competition Rules. Can the NHS be exempted?’ (2015) King’s Fund briefing, available at https://www.kingsfund.org.uk/publications/nhs-procurement-competition-rules.

  • M Guy, ‘Between “Going Private” and “NHS Privatisation”: Patient choice, competition reforms and the relationship between the NHS and private healthcare in England’ (2019) 39(3) Legal Studies 479-498.

  • P Allen et al, ‘Public Contracts as Accountability Mechanisms: Assuring quality in public health care in England and Wales’ (2016) 18(1) Public Management Review 20-39.

  • D Osipovič et al, ‘Interrogating Institutional Change: Actors' Attitudes to Competition and Cooperation in Commissioning Health Services in England’ (2016) 94(3) Public Administration 823-838.

  • P Allen et al, ‘Commissioning through Competition and Cooperation in the English NHS under the Health and Social Care Act 2012: Evidence from a qualitative study of four clinical commissioning groups’, BMJ Open 2017;7:e011745, available at http://dx.doi.org/10.1136/bmjopen-2016-011745.

  • M Sanderson, P Allen and D Osipovič, ‘The Regulation of Competition in the National Health Service (NHS): what difference has the Health and Social Care Act 2012 made?’ (2017) 12(1) Health Economics, Policy and Law 1-19.

  • D Osipovič et al, ‘The Regulation of Competition and Procurement in the National Health Service 2015–2018: Enduring hierarchical control and the limits of juridification’ (2020) 15(3) Health Economics, Policy and Law 308-324.

2. Centralisation of NHS procurement

Feedback and suggestions on additional readings most welcome: a.sanchez-graells@bristol.ac.uk.

The Emergence of Trans-EU Collaborative Procurement: A 'Living Lab' for European Public Law

lab.jpg

I have uploaded a new working paper on SSRN: ‘The Emergence of Trans-EU Collaborative Procurement: A “Living Lab” for European Public Law’ (March 14, 2019) https://ssrn.com/abstract=3392228. Its abstract is as follows:

Trans-EU collaborative procurement is a fertile ‘living lab’ for the observation, theorisation and critical assessment of developments in European public law. This paper maps the emergence of this novel type of cross-border administrative collaboration and scrutinises the new rules of Directive 2014/24/EU, which evidence the tension between promoting economic co-operation across borders within the internal market and the concern to respect the Member States’ administrative autonomy. The paper critically assesses the EU legislative competence in this area, extracts consequences for balancing trans-EU collaboration with ‘mandatory public law requirements’ at Member State level and proposes minimum functional guarantees to be expected in the implementation of trans-EU collaborative procurement.

Rejection of tenders for EU research funding, any lessons for procurement? (T-76/15)

horizon2020.png

In its Judgment of 18 January 2018 in Kenup Foundation and Others v EIT, T-76/15, EU:T:2018:9, the General Court of the Court of Justice of the EU (GC) assessed the compatibility with EU law of the rejection of a tender for funding under the Horizon 2020 framework programme for research and innovation. Some of the GC's analysis in this context can provide interesting insights for the rejection of tenders in procedures controlled by the EU public procurement rules, if evaluation and award decisions are adopted through two-tier bodies (eg a technical evaluation and an overall 'political' decision-making). This could be particularly relevant in the context of competitive dialogues or innovation partnerships. To be sure, the Kenup case hinges on EU administrative law, but I think it raises issues that can be comparable in some domestic settings in the Member States.

In Kenup, the European Institute of Innovation and Technology (EIT) issued a call for the designation of a Knowledge and Innovation Community (KIC) in the field of innovation for healthy living and active ageing. The cooperation established between the EIT and the KIC would take the form of a framework partnership, with an initial duration of seven years, in the course of which grants could be paid by the EIT on the basis of the conclusion of specific agreements. The call for proposals established specific criteria for the exclusion, eligibility and selection of proposals, which were to be undertaken under EIT's responsibility. The decision on the designation of the KIC was subjected to a three-tier process, as follows:

According to the rules in ... that call for tenders, which the parties agree were complied with both by the independent experts and by the EIT, eligible proposals were to be evaluated by high-level independent external experts. Each proposal was thus examined by five experts, that is to say three thematic experts and two ‘horizontal’ experts, each responsible for an evaluation report for each proposal. The panel of experts was then required to draw up a consolidated evaluation report for each proposal. Next, the three proposals with the best rankings were evaluated by a second panel of high-level independent experts responsible for making a final recommendation containing an overview of those three proposals as well as recommendations for their improvement or reinforcement. Finally, representatives of the three proposals with the best scores were to be heard by the governing board before it designated the selected KIC (T-76/15, para 58).

Therefore, the decision on the designation of the KIC was to be made by the EIT's governing board, on the basis of the recommendation made by the second panel of experts, which only had to take into consideration the three top proposals as 'filtered' by the first panel of experts. In principle, this seems like a rather robust evaluation mechanism, in particular of the technical aspects of the proposals. However, it can also raise issues of compliance with rules on 'fully-informed' decision-making or 'unrestricted' executive discretion, as the Kenup case evidences.

In response to the call for proposals, the Kenup consortium submitted a tender under the coordination of the Stiftung Universität Lüneburg. After assessment of the proposals received in accordance with the evaluation mechanism described above, the governing board of EIT selected a  proposal for the KIC and rejected the other proposals, including Kenup's. Kenup then challenged the EIT's decision on several grounds, including issues concerned with an alleged failure by EIT to state the reasons for its decision, as well as a breach of the principles of transparency and equal treatment of tenderers.

All of these arguments are common place in the challenge of procurement decisions and their analysis would have been interesting. However, the case was decided solely on an issue concerning the implicit constraints on the exercise of executive discretion by the EIT's governing board due to the initial 'filtering' of proposals by a panel of high-level experts. This merits some close analysis.

As presented by the GC,

It follows from the evaluation process for the proposals ... that the panel of experts responsible for the final recommendation only had to examine the three proposals with the best scores following the evaluation by the first panel of experts. In addition, only representatives of those three proposals were to be heard by the governing board. In that regard, it should be noted that the call for proposals clearly indicated that the KIC would be selected by the EIT on the basis, first, of the consolidated evaluation reports relating to the three best proposals, as established by the panel of experts, secondly, of the report drawn up by the panel responsible for the final recommendation and, thirdly, of the outcome of the hearings. Accordingly, the EIT was required to make its selection only on the basis of the work carried out by the independent experts on the three proposals with the best scores and the outcome of the hearings carried out with the representatives of those proposals.

... the members of the governing board had access, via a protected website, to all the proposals submitted for the KIC on ‘Innovation for healthy living and active ageing’, including the Kenup consortium’s proposal. Furthermore, before the hearings, the director of the EIT indicated to the governing board the various stages of the evaluation procedure, including the various scores awarded overall and for sub-criteria to the five proposals submitted. However, none of the analyses of the Kenup consortium’s proposal carried out by the independent experts were submitted to the members of the governing board. Annex 1 to the information note of 1 December 2014 drawn up by the director of the EIT for the members of the governing board, produced by the EIT at the Court’s request, included merely a summary of the evaluation reports drawn up by the panel of experts relating solely to the proposals selected for the hearings. In addition, it does not follow from the procedure for the call for proposals, nor is it claimed, that members of the governing board attended the experts’ working sessions.

It is true, as the EIT maintains in its defence, that members of the governing board were free to raise questions and to request additional information concerning all the proposals and their evaluation by the experts. However, ... the members of the governing board did not possess any of the evaluations or a summary of the evaluations carried out by the panel of experts concerning the two proposals not selected for the hearings.

In any event, any initiatives of the governing board were unlikely to call into question the fact that only the three proposals with the best scores awarded by the experts could have been designated as the KIC on ‘Innovation for healthy living and active ageing’. The procedure established by the call for proposals entirely ruled out any possibility of the governing board’s selecting the Kenup consortium’s proposal and inviting its representatives to participate in the hearings, since that proposal was ranked in fourth position by the independent experts. That finding is confirmed by the wording of the letter of 10 December 2014 informing the coordinator of the Kenup consortium that its proposal had been rejected, which clearly links that exclusion with the ranking of the consortium’s proposal below third place. On that point, it may be noted ... that, in its reply to their request for further information, the EIT stated that the experts had been granted, by the call for proposals, a delegated power to preselect proposals.

Therefore, in accordance with the procedure defined in the call for proposals, the governing board could, following the hearings, only alter the ranking of the three best proposals selected by the experts ... The fact that, according to Article 15 of Regulation No 1290/2013, the selection of a KIC is made on the basis of the ranking of the proposals, in accordance with the evaluation carried out by independent experts, cannot mean that the EIT is bound, even in part, as regards the order of the proposals thus selected.

67      It follows from all the foregoing considerations that the applicants are justified in maintaining that the governing board failed fully to exercise its powers in respect of the selection of proposals, in breach of the provisions of Article 4 of Regulation No 294/2008, those powers having been delegated in part to experts without that board having, at any time, had the opportunity to make a proper assessment of the work they carried out on the proposals which were not ranked in the first three places (T-76/15, paras 61-65 & 67, emphases added).

As mentioned above, the Kenup Judgment is largely conditioned by a point of EU administrative law concerning the implicit delegation of the power to preselect proposals to the initial high-level expert panel. However, I find the case troubling in that context, and for any implications it could have in the context of procurement covered by the 2014 Public Procurement Package. I have two main issues with this Judgment.

First, and foremost, that it seems to follow the worrying trend of disrespect for expert opinion. Implicit in the GC Judgment, there is an assumption that the governing board of EIT would have been able to challenge expert reports prepared in a seemingly robust manner. This seems difficult to share. Either the independent technical evaluation was needed because the governing board does not have the expertise (or time) to sift through all proposals--in which case the assumption that the governing board will look at all documents and sort of reassess all proposals from scratch is ludicrous--or it was not needed at all, and should be abandoned--which seems equally unpersuasive. More generally, it seems that the GC misunderstands the context and boundaries of the executive discretion given to EIT's governing board by the relevant EU provisions, as well as the fact that EIT had endorsed the specific evaluation mechanism (thus potentially self-constraining any broader discretion it may have had, in a manner that the GC hardly demonstrates to run contrary to any relevant constraints). From that perspective, this Judgment is at best extremely formalistic and, at worse, simply misguided.

Second, and also of importance, depending on the rules applicable under the general administrative law of the Member States, the thrust of the Kenup Judgment can result in significant difficulties (and potential challenges) in the context of complex procurement procedures where the overall (political) decision-making is supported by one or several rounds of technical evaluation aimed at filtering the initial proposals into shortlists or recommendations. If the logic in the Kenup Judgment was adopted, and the ultimate decision-makers of the contracting authorities and entities covered by the 2014 Public Procurement Package were required to have before them (and effectively engage with) the entirety of the documentation with a view to (potentially) challenging technical evaluations, complex procurement procedures could become exceedingly burdensome and/or (even more of a) box-ticking exercise. Moreover, it would be possible to generate inadvertent corruption risks if the non-expert (ie political) board could second-guess or deviate from robust technical assessments and have unfettered discretion. This would run in stark contrast with the case law of the CJEU on award criteria and unlimited freedom to choose a tender.

Consequently, my overall view of the Kenup Judgment is that it does not offer any valuable (or at least useful) lesson for procurement, and that the GC would have been well-advised to have followed the opposite direction of travel. By taking into consideration the case law on procurement that requires discretion to be constrained by solid technical evaluation, the decision in Kenup could (and should) have been the opposite. I can only hope that this case is limited to the way EU research funding is administered, and that the Kenup Judgment results in a change of EIT's internal governance rules in a way that preserves and enhances the role of independent high-level technical evaluations against the erosion that the GC's Judgment has generated.

Another conversation on procurement developments with the EFTA Surveillance Authority

ESA_logo_Col_600.gif

Last week, I had the great honor and pleasure of speaking at another EFTA Surveillance Authority lunchtime staff seminar two years after (see here for the 2016 edition), and to benefit again from the insights and challenging questions of its community of enforcers. This time, the conversation concentrated on recent policy and case law trends, and the impact they can have for the enforcement of EEA/EU internal market rules.

These are the slides I used. If time allows, I will try to publish a more complete account of my speech, although most of the issues have been covered in previous posts already, so hopefully the slides will be easy to follow. As always, feedback and comments more than welcome.