Implementation Challenges for the Procurement Act 2023

I have put together a consolidated overview of the primary challenges for the implementation of the Procurement Act 2023, to be included as a country report in a forthcoming issue of the European Procurement & Public Private Partnership Law Review.

It brings together developments discussed in the blog over the last year or so, including the transparency ambition, the innovation ambition, and the training offer linked to the Transforming Public Procurement project.

In case of interest, it can be downloaded from SSRN: https://ssrn.com/abstract=4692660.

It contains nothing new, though, so assiduous readers may want to skip this one!

Responsibly Buying Artificial Intelligence: A ‘Regulatory Hallucination’ -- draft paper for comment

© Matt Lowe/LinkedIn.

Following yesterday’s Current Legal Problems Lecture, I have uploaded the current full draft of the paper on SSRN. I would be very grateful for any comments in the next few weeks, as I plan to do a final revision and to submit it for peer-review in early 2024. Thanks in advance for those who take the time. As always, you can reach me at a.sanchez-graells@bristol.ac.uk.

The abstract of the paper is as follows:

Here, I focus on the UK’s approach to regulating public sector procurement and use of artificial intelligence (AI) in the context of the broader ‘pro-innovation’ approach to AI regulation. Borrowing from the description of AI ‘hallucinations’ as plausible but incorrect answers given with high confidence by AI systems, I argue that UK policymaking is trapped in a ‘regulatory hallucination.’ Despite having embraced the plausible ‘pro-innovation’ regulatory approach with high confidence, that is the incorrect answer to the challenge of regulating AI procurement and use by the public sector. I conceptualise the current strategy as one of ‘regulation by contract’ and identify two of its underpinning presumptions that make its deployment in the digital context particularly challenging. I show how neither the presumption of superiority of the public buyer over the public contractor, nor the related presumption that the public buyer is the rule-maker and the public contractor is the rule-taker, necessarily hold in this context. Public buyer superiority is undermined by the two-sided gatekeeping required to simultaneously discipline the behaviour of the public sector AI user and the tech provider. The public buyer’s rule-making role is also undermined by its reliance on industry-led standards, as well as by the tech provider’s upper hand in setting contractual benchmarks and controlling the ensuing self-assessments. In view of the ineffectiveness of regulating public sector AI use by contract, I then sketch an alternative strategy to boost the effectiveness of the goals of AI regulation and the protection of individual rights and collective interests through the creation of an independent authority.

Sanchez-Graells, Albert, ‘Responsibly Buying Artificial Intelligence: A “Regulatory Hallucination”’ (November 24, 2023). Current Legal Problems 2023-24, Available at SSRN: https://ssrn.com/abstract=4643273.

Public procurement (entry for an Encyclopaedia)

I was invited to provide an entry on ‘public procurement’ for the forthcoming Elgar Encyclopedia of European Law co-edited by Andrea Biondi and Oana Stefan. I must say I struggled to decide what to write about, as the entry was limited to 4,000 words and there are so many (!!) things going on in procurement. Below is my draft entry with perhaps an eclectic choice of content. Comments most welcome!

The draft entry is also available on SSRN if you prefer a pdf version: A Sanchez-Graells, ‘Public procurement’ in A Biondi and O Stefan, Elgar Encyclopedia of European Law (forthcoming) available at https://ssrn.com/abstract=4621399.

Public Procurement

I. Introduction

From up close, public procurement law can be seen as the set of mostly procedural rules controlling the way in which the public sector buys goods, services, and works from the market. Procurement would thus be a set of administrative law requirements concerned with the design and advertisement of tenders for public contracts, the decision-making process leading to the award of those contracts, and the advertisement and potential challenge of such decisions. To a more limited extent, some requirements would extend to the contract execution phase, and control in particular the modification and eventual termination of public contracts. From this narrow perspective, procurement would be primarily concerned with ensuring the integrity and probity of decision-making processes involving the management of public funds, as well as fostering the generation of value for money through effective reliance on competition for public contracts.

The importance and positive contribution of public procurement law to the adequate management of public funds may seem difficult to appreciate in ordinary times, and there are recurrent calls for a reduction of the administrative burden and bureaucracy related to procurement procedures, checks and balances. However, as the pervasive abuses of direct awards under the emergency conditions generated by the covid pandemic evidenced in virtually all jurisdictions, dispensing with those requirements, checks and balances comes with a very high price tag for taxpayers in terms of corruption, favouritism, and wastage of public funds.

Even from this relatively narrow perspective of procurement as a process-based mechanism of public governance, procurement attracts a significant amount of attention from EU legislators and from the EU Courts and is an area of crucial importance in the development of the European administrative space. As procurement regulation has been developed through successive generations of directives, and as many Member States had long traditions on the regulation of public procurement prior to the emergence of EU law on the topic, procurement offers a fertile ground for comparative public law scholarship. More recently, as EU procurement policy increasingly seeks to promote cross-border collaboration, procurement is also becoming a driver (or an irritant) for the transnational regulation of administrative processes and a living lab for experimentation and legal innovation.

From a slightly broader perspective, public procurement can be seen as a tool for the self-organisation of the State and as a primary conduit for the privatisation and outsourcing of State functions. A decision preceding procurement concerns the size and shape of the State, especially in relation to which functions and activities the State carries out in-house (including through public-public collaboration mechanisms), and which other are contracted out to the market (‘make or buy’ decisions). Procurement then controls the design and award of contracts involving the exercise of public powers, or the direct provision of public services to citizens where market agents are called upon to do so (including in the context of quasi-markets). Procurement thus heavily influences the interaction between the State’s contractual agents and citizens, and becomes a tool for the regulation of public service delivery. The more the State relies on markets for the provision of public services, the larger the potential influence (both positive and negative) of procurement mechanisms on citizens’ experience of their (indirect) interaction with the State. On this view, procurement is a tool of public governance and a conduit for public-private cooperation, as well as a regulatory mechanism for delegated public-public and public-private interactions. From this perspective, procurement is often seen as a neoliberal tool closely linked to new public management (NPM), although it should be stressed that procurement rules only activate once the decision to resort to contracting out or outsourcing has been made, as EU law does not mandate ‘going to market’.

From an even broader perspective, public procurement represents a more complex and multi-layered regulatory instrument. Given the enormous amounts of public funds channelled through public procurement, and the market-shaping effects that can follow from the exercise of such buying power, procurement regulation is often used as a lever for the promotion of policies and goals well beyond the narrower confines of procurement as a regulated administrative process. In the EU, procurement has always been an instrument of internal market regulation and sought to dismantle barriers to cross-border competition for the award of public contracts. More recently, and in line with developments in other jurisdictions, procurement has been increasingly singled out as a tool to promote environmental and sustainability goals, as well as social goals, or as a tool to foster innovation. Procurement is also increasingly identified as a tool to foster compliance with human rights along increasingly complex supply chains, or to address social inequality, such as through gender responsive procurement. In the face of the challenges posed by the mainstreaming of digital technologies, and artificial intelligence in particular, procurement is also increasingly identified as a tool of digital regulation. And, against the background of rule of law challenges within the EU, procurement conditionality has added to the fiscal control effect traditionally linked to the use of EU funds to subsidise procurement projects at Member State level. From this perspective, procurement is either an enforcement (or reinforcement) mechanism, or a self-standing regulatory tool for the pursuit of an increasingly diverse array of horizontal policies seeking to steer market activities.

Relatedly, given the importance of procurement as an economic activity, its regulation is of crucial importance in the context of industrial and trade policies. The interaction between procurement and industrial policy is not entirely straightforward, and neither is the position of procurement in the context of trade liberalisation. While there have been waves of policy efforts seeking to minimise the use of procurement for industrial policy purposes (ie the award of public contracts to national champions), in particular given the State aid implications of such uses of public contracts under EU law, and while there is a general push for the liberalisation of international trade through procurement—there are also periodic waves of protectionism where procurement is used as a tool of international economic regulation or, more broadly, geopolitics. Most recently, the EU has aggressively (re)regulated access to its procurement markets on grounds of such considerations.

It would be impossible to address all the issues that arise from the regulation of public procurement in all these (and other potential) dimensions within a single entry. Here, I will touch upon some the issues highlighted by recent developments in EU law and policy, and in relation to contemporary debates around the salient grand challenges encapsulated in the need for procurement to support the ‘twin transition’ to green and digital. I will not focus on the detail of procurement rules, which is better left to in-depth analysis (eg Arrowsmith [2014] and [2018], Steinicke and Vesterdorf [2018], or Caranta and Sanchez-Graells [2021]). There are a few common threats in the developments discussed below, especially in relation to the increasing complexity of procurement policymaking and administration, or the crucial role of expertise and capability, as well as some challenges in coordinating them in a way that generates meaningful outcomes. I will briefly return to these issues in the conclusion.

II. Procurement, Trade, and Geopolitics

A constant tension in the regulation of procurement concerns the openness of procurement markets. On the one hand, procurement can be a catalyst for trade liberalisation and there are many economic advantages stemming from increased (international) competition for public contracts—as evidenced in the context of the World Trade Organisation Government Procurement Agreement (WTO GPA) (Georgopoulos et al [2017]). In the narrower context of the EU’s internal market, public procurement openness is taken to its logical extremes and barriers to cross-border tendering are systematically dismantled through legislation, such as the most recent 2014 Public Procurement Package, and its interpretation by the Court of Justice. While there is disparity in national practice, the (complete) openness of procurement markets in the EU tends to not only benefit EU tenderers, but also those of third countries, who tend to be treated equally with EU ‘domestic’ tenderers.

On the other hand, the same (international) competition that can bring economic advantages can also put pressure on (less competitive) domestic industries or create risks of uneven playing field—especially where (foreign national champion) tenderers are propped up by their States. In some industries and in relation to some critical infrastructure, the award of oftentimes large and sensitive public contracts to foreign undertakings also generates concerns around safety and sovereignty.

A mechanism to mediate this tension is to make procurement-related trade liberalisation conditional on reciprocity, which in turn leverages multilateral instruments such as the WTO GPA. This is an area where EU law has recently generated significant developments. After protracted negotiations, EU procurement law now comprises a set of three instruments seeking to rebalance the (complete) openness of EU procurement markets.

As a starting point, under EU law, only foreign economic operators covered by an existing international agreement (such as the WTO GPA, or bilateral or multilateral trade agreements concluded with the EU that include commitments on access to public procurement) are entitled to equal treatment. However, differential treatment or outright exclusion of economic operators not covered by such equal treatment obligation tends (or has historically tended to) be rare. This can be seen to weaken the hand of the European Commission in international negotiations, as EU procurement markets are de facto almost entirely open, regardless of the much more limited legal openness resulting from those international agreements.

To nudge contracting authorities to enforce differential treatment, in 2020, the European Commission issued guidance on the participation of third country bidders and goods in EU procurement markets, stressing the several ways in which public buyers could address concerns regarding unfair competitive advantages of foreign tenderers. This should be seen as a first step towards ramping up the ‘rebalancing’ of access to EU procurement markets, though it is a soft (law) step and one that would still hinge on coordinated decision-making by a very large number of public buyers making tender-by-tender decisions.

A second and crucial step was taken in 2022 with the adoption of the EU’s International Procurement Instrument (IPI), which empowers the European Commission to carry out investigations where there are concerns about measures or practices negatively affecting the access of EU businesses, goods and services to non-EU procurement markets and, eventually, to impose (centralised) IPI measures to restrict access to EU public procurement procedures for businesses, goods and services from the non-EU countries concerned. The main effect of the IPI can be expected to be twofold. Outwardly, the IPI will lead to the European Commission having ‘a stick’ to push for reciprocity in procurement liberalisation as a complement to ‘the carrot’ used to persuade more and more countries to enter into bilateral trade deals, or for them to join the WTO GPA. Internally, the IPI will allow the Commission to mandate Member States to implement the relevant restrictions or exclusions from the EU procurement markets in relation to the jurisdictions concerned. This is expected to address the issue of de facto openness beyond existing (international) legal requirements, and therefore galvanise the ability of the Commission to control access to ‘the EU procurement market’ and thus bolster its ability to use procurement reciprocity as a tool for trade liberalisation more effectively.

A third and final crucial step came with the adoption in 2023 of the Regulation on foreign subsidies distorting the internal market, which creates a mechanism for the control of potential foreign subsidies in tenders for contracts with an estimated value above EUR 250 million, and can also result in the imposition of (centralised) measures curving access to the relevant contracts by the beneficiaries of those foreign subsidies. This comes to somehow create an international functional equivalent to the State aid control in place for domestic tenders, as well as a mechanism for the EU to enforce international anti-dumping standards within its own jurisdiction.

This trend of evolution in EU public procurement regulation evidences that public buyers are increasingly constrained by geopolitical and international economic considerations administered by the European Commission in a centralised manner (Andhov and Kania [2023]). Whether this will create friction between the Commission and Member States, perhaps in relation to particularly critical or sensitive procurement projects, remains to be seen. In any case, this line of policy and legal developments generates increased complexity in the administration of procurement processes on a day-to-day basis, and will require public buyers to develop expertise in the assessment of the relevant trade-related instruments and associated documentation, which will be a theme in common with other developments discussed below.

III. Procurement and Sustainability

It is relatively uncontroversial that public expenditure has a crucial role to play in supporting (or driving) the transition towards a more sustainable economy, and most jurisdictions explicitly consider how to harness public expenditure to decarbonise their economy and achieve net zero targets—sometimes in the broader context of efforts to achieve interlinked sustainable development goals. However, the details on the specific sustainability goals to be pursued through procurement (as compared to other means of public finances, such as subsidies or tax incentives), and on how to design and implement sustainable procurement are more contested.

Green procurement has been a primary focus of EU public procurement policy for a long time now, and it has received even further increased attention in recent years, culminating in the attribution of a prominent role for the implementation of the EU’s Green Deal. EU procurement law has been increasingly permissive and facilitative of the inclusion of environmental considerations in procurement decision-making and the European Commission has developed sets of guidance and technical documentation that are kept under permanent review and update. Overall, EU procurement law offers a diverse toolkit for public buyers to embed sustainability requirements.

However, the uptake of green procurement is much lower than would be desirable and progress is very uneven across jurisdictions and in different sectors of the economy. There is a growing realisation that facilitative or permissive approaches will not result in the quick generalisation of sustainability concerns across procurement practice required to contribute to mitigating the devastating effects of climate change in a timely fashion, or with sufficient scale. Informational and skills barriers, difficult economic assessments and competing (political) priorities necessarily slow down the uptake of sustainable procurement. In this context, it seems clear that technical complexity in the administration of procurement on a day-to-day basis, and limited technical skills in relation to sustainability assessments, are the primary obstacle in the road to mainstreaming sustainable public procurement. It is hard for public buyers to identify the relevant sustainability requirements and to embed them in their decision-making, especially where the inclusion of such requirements is bound to be checked against its suitability, proportionality, and its effect on potential competition for the relevant public contract.

To overcome this obstacle, it seems clear that a more proactive or prescriptive approach is required and that sustainability requirements must be embedded in legislation that binds public buyers—so that their role becomes one of (reinforced) compliance assessment or indirect enforcement. The question that arises, and which reopens age old discussions, is whether such legislation should solely target public procurement (Janssen and Caranta [2023]) or rather be of general application across the economy (Halonen [2021]).

This controversy evidences different understandings of the role of procurement-specific legislation and different levels of concern with the partitioning of markets. While the passing of procurement-specific legislation could be easier and politically more palatable—as it would be perceived to ultimately impose the relevant burden on economic operators seeking to gain public business (and so embed a certain element of opt-in or balanced regulatory burden against the prospect of accessing public funds), and the cost would ultimately fall on public buyers as ‘responsible (sustainable) buyers’—it would partition markets and eg potentially prevent the generation of economies of scale where public demand is not majoritarian. Moreover, such market partitioning would raise entry barriers for entities new to bidding for public contracts, as well as facilitate the emergence of anticompetitive and collusive practices in the more concentrated and partly isolated from potential competition ‘public markets’ (Sanchez-Graells [2015]) in ways that general legislation would not. More generally, advances in mandating sustainable procurement could deactivate the pressure for developments in more general sustainability mandates, as policymakers could claim to already be doing significant efforts (in the narrow setting of procurement).

A narrow sectoral approach to legislating for public procurement only would probably also over-rely on the hopes that procurement practices can become best practices and thus disseminate themselves across the economy through some understanding of mimicking, or race to the top. This relates to discussions in other areas and to the broader expectation that procurement can be a trend setter and influence industry practice and standards. However, as the discussion on digitalisation will show, the direction of influence tends to be on reverse and there are very limited mechanisms to promote or force industry adaptation to procurement standards other than in relation to direct access to procurement.

IV. Procurement and the ‘Digital Transformation’ of the State

Another area of growing consensus is that public procurement has a key role to play in the ‘digital transformation’ of the State, as the process of digitalisation is bound to rely on the acquisition of technology from market providers to a large or sole extent (depending on each jurisdiction’s make or buy decisions). This can in turn facilitate the role of procurement as a tool of digital industrial policy, especially because procurement expenditure can be a way of ensuring demand for innovation, and because public sector technology adoption can be used as a domain for experimentation with new technologies and new forms of technology-enabled governance.

The European Union has set very high expectations in its Digital Agenda 2030, and the Commission has recently stressed that achieving them would require roughly doubling the predicted level of public procurement expenditure in digital technologies, and artificial intelligence (AI) in particular. It can thus be expected that the procurement of digital technologies will quickly gain practical importance even in jurisdictions that have been lagging so far.

However, echoing some of the issues concerning sustainable procurement, in this second stream of the ‘twin transition’, the uptake of procurement of digital technologies is slowed down by the complexity of procuring unregulated immature technologies, and the (digital) skills gaps in the public sector—which are exacerbated by the absence of a toolkit of regulatory and practical resources equivalent to that of green procurement. In such a context of technological fluidity and hype, given the skills and power imbalances between technology providers and public buyers, the shortcomings of the use of public procurement as a regulatory mechanism become stark and the flaws in the logic or expectation that procurement can be an effective tool of market steering are laid bare (Sanchez-Graells [2024]).

Public buyers are expected to act as responsible AI buyers and to ensure the ‘responsible use of AI’ in the public sector. The EU AI Act will soon establish specific requirements in that regard, although solely in relation to high-risk AI uses as defined therein. Implementing the requirements of the EU AI Act—and their extension to other types of uses of digital technology or algorithms as a matter of ‘best practice’—will leverage procurement processes and, in particular, the ensuing public contracts to impose the relevant obligations on technology providers. In that connection, the European Commission has promoted the development of model contractual AI clauses that seek to regulate the technology to be procured and their future use by the relevant public sector deployer.

However, an analysis of the model clauses and broader guidance on the procurement of AI shows that public buyers will still face a very steep knowledge gap as it will be difficult to set the detail of the relevant contracts, which will tend to be highly context dependent. In other words, the model clauses are not ‘plug and play’ and implementing meaningful safeguards in the procurement and use of AI and other digital technologies will require advanced digital skills and sufficient commercial leverage—which are not to be taken as a given. Crucially, all obligations under the model clauses (and the EU AI Act itself) hinge on (self-assessment) processes controlled by the technology provider and/or refer back to technical standards or the state-of-the-art, which are driven and heavily influenced (or entirely controlled) by the technology industry. Public buyers are at a significant disadvantage not only to set, but also to monitor compliance with relevant requirements.

This shows that, in the absence of mandatory requirements and binding (general) legislation, the use of procurement for regulatory purposes has a high risk of commercial determination and regulatory tunnelling as public buyers with limited skills and capabilities struggle to impose requirements on technology providers, and where references to standards also displace regulatory decision-making. This means that public procurement can no longer be expected to ‘monitor itself’, and that new forms of institutional oversight are required to ensure that the procurement of digital technologies works in the broader public interest.

V. Conclusion

Although the issues discussed above may seem rather disparate, they share a few common threads. First, in all areas, the regulatory use of procurement generates complexity and makes the day-to-day administration of procurement processes more complex. It can be hard for a public buyer to navigate socio-political, sustainability and digitalisation concerns—and these are only some of the ‘non-strictly procurement-related’ concerns and considerations to be taken into account. Such difficulty can be compounded by limited capabilities and by gaps in the required skills. While this is particularly clear in the digital context, the issue of limited (technical) capability is also highly relevant in relation to sustainable procurement. An imbalance in skills and commercial leverage between the public buyer and technology providers undermines the logic of using procurement as a regulatory tool. Implementation issues thus require much further thought and investment than they currently receive.

Ultimately, the effectiveness of the regulatory goals underpinning the leveraging of procurement hinges on the ability of public buyers to meaningfully implement them. This raises the further question whether all goals can be achieved at the same time, especially where there can be difficult trade-offs. And there can be many of those. For example, it can well be that the offeror of the most attractive technology comes from a ‘black-listed’ jurisdiction. It can also be that the most attractive technology is also the most polluting, or one that raises significant other risks or harms from a social perspective, etc. Navigating these risks and making the (implicit) political choices may be too taxing a task for public buyers, as well as raise issues of democratic accountability more generally. Moreover, enabling public buyers to deal with these issues and to exercise judgement and discretion reopens the door to risks of eg bias, capture or corruption, as well as maladministration and error, which are some of the core concerns in the narrow approach to the regulation of procurement as an administrative procedure to being with. Those trade-offs are also pervasive and hard to assess.

It is difficult to foresee the future, but my intuition is that the trend of piling up of regulatory goals on procurement’s shoulders will need to slow down or reverse if it is meant to remain operational, and that a return to a more paired down understanding of the role of procurement will need to be enabled by the emergence of (generally applicable) legislation and external oversight mechanisms that can discharge procurement of these regulatory roles. Or, at least, that is the way I would like to see the broader regulation and policymaking around procurement to evolve.

Bibliography

Andhov, Marta and Michal Andrzej Kania, ‘Restricting Freedom of Contract – the EU Foreign Subsidies Regulation and its Consequences for Public Procurement’ (2023) Journal of Public Procurement.

Arrowsmith, Sue, The Law of Public and Utilities Procurement. Regulation in the EU and the UK, vols 1 & 2 (3rd edn, Sweet & Maxwell 2014 and 2018).

Caranta, Roberto and Albert Sanchez-Graells (eds), European Public Procurement. Commentary on Directive 2014/24/EU (Edward Elgar 2021).

Georgopoulos, Aris, Bernard Hoekman and Petros C Mavroidis (eds), The Internationalization of Government Procurement Regulation (OUP 2017).

Halonen, Kirsi-Maria, ‘Is public procurement fit for reaching sustainability goals? A law and economics approach to green public procurement’ (2021) 28(4) Maastricht Journal of European and Comparative Law 535-555.

Janssen, Willem and Roberto Caranta (eds), Mandatory Sustainability Requirements in EU Public Procurement Law. Reflections on a Paradigm Shift (Hart 2023).

Sanchez-Graells, Albert, Public Procurement and the EU Competition rules (2nd end, Hart, 2015).

Sanchez-Graells, Albert, Digital Technologies and Public Procurement. Gatekeeping and Experimentation in Digital Public Governance (OUP 2024).

Steinicke, Michael and Peter L Vesterdorf (eds), Brussels Commentary on EU Public Procurement Law (C H Beck, Hart & Nomos 2018).

Some thoughts on the US' Executive Order on the Safe, Secure, and Trustworthy Development and Use of AI

On 30 October 2023, President Biden adopted the Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (the ‘AI Executive Order’, see also its Factsheet). The use of AI by the US Federal Government is an important focus of the AI Executive Order. It will be subject to a new governance regime detailed in the Draft Policy on the use of AI in the Federal Government (the ‘Draft AI in Government Policy’, see also its Factsheet), which is open for comment until 5 December 2023. Here, I reflect on these documents from the perspective of AI procurement as a major plank of this governance reform.

Procurement in the AI Executive Order

Section 2 of the AI Executive Order formulates eight guiding principles and priorities in advancing and governing the development and use of AI. Section 2(g) refers to AI risk management, and states that

It is important to manage the risks from the Federal Government’s own use of AI and increase its internal capacity to regulate, govern, and support responsible use of AI to deliver better results for Americans. These efforts start with people, our Nation’s greatest asset. My Administration will take steps to attract, retain, and develop public service-oriented AI professionals, including from underserved communities, across disciplines — including technology, policy, managerial, procurement, regulatory, ethical, governance, and legal fields — and ease AI professionals’ path into the Federal Government to help harness and govern AI. The Federal Government will work to ensure that all members of its workforce receive adequate training to understand the benefits, risks, and limitations of AI for their job functions, and to modernize Federal Government information technology infrastructure, remove bureaucratic obstacles, and ensure that safe and rights-respecting AI is adopted, deployed, and used.

Section 10 then establishes specific measures to advance Federal Government use of AI. Section 10.1(b) details a set of governance reforms to be implemented in view of the Director of the Office of Management and Budget (OMB)’s guidance to strengthen the effective and appropriate use of AI, advance AI innovation, and manage risks from AI in the Federal Government. Section 10.1(b) includes the following (emphases added):

The Director of OMB’s guidance shall specify, to the extent appropriate and consistent with applicable law:

(i) the requirement to designate at each agency within 60 days of the issuance of the guidance a Chief Artificial Intelligence Officer who shall hold primary responsibility in their agency, in coordination with other responsible officials, for coordinating their agency’s use of AI, promoting AI innovation in their agency, managing risks from their agency’s use of AI …;

(ii) the Chief Artificial Intelligence Officers’ roles, responsibilities, seniority, position, and reporting structures;

(iii) for [covered] agencies […], the creation of internal Artificial Intelligence Governance Boards, or other appropriate mechanisms, at each agency within 60 days of the issuance of the guidance to coordinate and govern AI issues through relevant senior leaders from across the agency;

(iv) required minimum risk-management practices for Government uses of AI that impact people’s rights or safety, including, where appropriate, the following practices derived from OSTP’s Blueprint for an AI Bill of Rights and the NIST AI Risk Management Framework: conducting public consultation; assessing data quality; assessing and mitigating disparate impacts and algorithmic discrimination; providing notice of the use of AI; continuously monitoring and evaluating deployed AI; and granting human consideration and remedies for adverse decisions made using AI;

(v) specific Federal Government uses of AI that are presumed by default to impact rights or safety;

(vi) recommendations to agencies to reduce barriers to the responsible use of AI, including barriers related to information technology infrastructure, data, workforce, budgetary restrictions, and cybersecurity processes;

(vii) requirements that [covered] agencies […] develop AI strategies and pursue high-impact AI use cases;

(viii) in consultation with the Secretary of Commerce, the Secretary of Homeland Security, and the heads of other appropriate agencies as determined by the Director of OMB, recommendations to agencies regarding:

(A) external testing for AI, including AI red-teaming for generative AI, to be developed in coordination with the Cybersecurity and Infrastructure Security Agency;

(B) testing and safeguards against discriminatory, misleading, inflammatory, unsafe, or deceptive outputs, as well as against producing child sexual abuse material and against producing non-consensual intimate imagery of real individuals (including intimate digital depictions of the body or body parts of an identifiable individual), for generative AI;

(C) reasonable steps to watermark or otherwise label output from generative AI;

(D) application of the mandatory minimum risk-management practices defined under subsection 10.1(b)(iv) of this section to procured AI;

(E) independent evaluation of vendors’ claims concerning both the effectiveness and risk mitigation of their AI offerings;

(F) documentation and oversight of procured AI;

(G) maximizing the value to agencies when relying on contractors to use and enrich Federal Government data for the purposes of AI development and operation;

(H) provision of incentives for the continuous improvement of procured AI; and

(I) training on AI in accordance with the principles set out in this order and in other references related to AI listed herein; and

(ix) requirements for public reporting on compliance with this guidance.

Section 10.1(b) of the AI Executive Order establishes two sets or types of requirements.

First, there are internal governance requirements and these revolve around the appointment of Chief Artificial Intelligence Officers (CAIOs), AI Governance Boards, their roles, and support structures. This set of requirements seeks to strengthen the ability of Federal Agencies to understand AI and to provide effective safeguards in its governmental use. The crucial set of substantive protections from this internal perspective derives from the required minimum risk-management practices for Government uses of AI, which is directly placed under the responsibility of the relevant CAIO.

Second, there are external (or relational) governance requirements that revolve around the agency’s ability to control and challenge tech providers. This involves the transfer (back to back) of minimum risk-management practices to AI contractors, but also includes commercial considerations. The tone of the Executive Order indicates that this set of requirements is meant to neutralise risks of commercial capture and commercial determination by imposing oversight and external verification. From an AI procurement governance perspective, the requirements in Section 10.1(b)(viii) are particularly relevant. As some of those requirements will need further development with a view to their operationalisation, Section 10.1(d)(ii) of the AI Executive Order requires the Director of OMB to develop an initial means to ensure that agency contracts for the acquisition of AI systems and services align with its Section 10.1(b) guidance.

Procurement in the Draft AI in Government Policy

The guidance required by Section 10.1(b) of the AI Executive Order has been formulated in the Draft AI in Government Policy, which offers more detail on the relevant governance mechanisms and the requirements for AI procurement. Section 5 on managing risks from the use of AI is particularly relevant from an AI procurement perspective. While Section 5(d) refers explicitly to managing risks in AI procurement, given that the primary substantive obligations will arise from the need to comply with the required minimum risk-management practices for Government uses of AI, this specific guidance needs to be read in the broader context of AI risk-management within Section 5 of the Draft AI in Government Policy.

Scope

The Draft AI in Government Policy relies on a tiered approach to AI risk by imposing specific obligations in relation to safety-impacting and rights-impacting AI only. This is an important element of the policy because these two categories are defined (in Section 6) and in principle will cover pre-established lists of AI use, based on a set of presumptions (Section 5(b)(i) and (ii)). However, CAIOs will be able to waive the application of minimum requirements for specific AI uses where, ‘based upon a system-specific risk assessment, [it is shown] that fulfilling the requirement would increase risks to safety or rights overall or would create an unacceptable impediment to critical agency operations‘ (Section 5(c)(iii)). Therefore, these are not closed lists and the specific scope of coverage of the policy will vary with such determinations. There are also some exclusions from minimum requirements where the AI is used for narrow purposes (Section 5(c)(i))—notably the ‘Evaluation of a potential vendor, commercial capability, or freely available AI capability that is not otherwise used in agency operations, solely for the purpose of making a procurement or acquisition decision’; AI evaluation in the context of regulatory enforcement, law enforcement or national security action; or research and development.

This scope of the policy may be under-inclusive, or generate risks of under-inclusiveness at the boundary, in two respects. First, the way AI is defined for the purposes of the Draft AI in Government Policy, excludes ‘robotic process automation or other systems whose behavior is defined only by human-defined rules or that learn solely by repeating an observed practice exactly as it was conducted’ (Section 6). This could be under-inclusive to the extent that the minimum risk-management practices for Government uses of AI create requirements that are not otherwise applicable to Government use of (non-AI) algorithms. There is a commonality of risks (eg discrimination, data governance risks) that would be better managed if there was a joined up approach. Moreover, developing minimum practices in relation to those means of automation would serve to develop institutional capability that could then support the adoption of AI as defined in the policy. Second, the variability in coverage stemming from consideration of ‘unacceptable impediments to critical agency operations‘ opens the door to potentially problematic waivers. While these are subject to disclosure and notification to OMB, it is not entirely clear on what grounds OMB could challenge those waivers. This is thus an area where the guidance may require further development.

extensions and waivers

In relation to covered safety-impacting or rights-impacting AI (as above), Section 5(a)(i) establishes the important principle that US Federal Government agencies have until 1 August 2024 to implement the minimum practices in Section 5(c), ‘or else stop using any AI that is not compliant with the minimum practices’. This type of sunset clause concerning the currently implicit authorisation for the use of AI is a potentially powerful mechanism. However, the Draft also establishes that such obligation to discontinue non-compliant AI use must be ‘consistent with the details and caveats in that section [5(c)]’, which includes the possibility, until 1 August 2024, for agencies to

request from OMB an extension of limited and defined duration for a particular use of AI that cannot feasibly meet the minimum requirements in this section by that date. The request must be accompanied by a detailed justification for why the agency cannot achieve compliance for the use case in question and what practices the agency has in place to mitigate the risks from noncompliance, as well as a plan for how the agency will come to implement the full set of required minimum practices from this section.

Again, the guidance does not detail on what grounds OMB would grant those extensions or how long they would be for. There is a clear interaction between the extension and waiver mechanism. For example, an agency that saw its request for an extension declined could try to waive that particular AI use—or agencies could simply try to waive AI uses rather than applying for extensions, as the requirements for a waiver seem to be rather different (and potentially less demanding) than those applicable to a waiver. In that regard, it seems that waiver determinations are ‘all or nothing’, whereas the system could be more flexible (and protective) if waiver decisions not only needed to explain why meeting the minimum requirements would generate the heightened overall risks or pose such ‘unacceptable impediments to critical agency operations‘, but also had to meet the lower burden of mitigation currently expected in extension applications, concerning detailed justification for what practices the agency has in place to mitigate the risks from noncompliance where they can be partly mitigated. In other words, it would be preferable to have a more continuous spectrum of mitigation measures in the context of waivers as well.

general minimum practices

Both in relation to safety- and rights-impact AI uses, the Draft AI in Government Policy would require agencies to engage in risk management both before and while using AI.

Preventative measures include:

  • completing an AI Impact Assessment documenting the intended purpose of the AI and its expected benefit, the potential risks of using AI, and and analysis of the quality and appropriateness of the relevant data;

  • testing the AI for performance in a real-world context—that is, testing under conditions that ‘mirror as closely as possible the conditions in which the AI will be deployed’; and

  • independently evaluate the AI, with the particularly important requirement that ‘The independent reviewing authority must not have been directly involved in the system’s development.’ In my view, it would also be important for the independent reviewing authority not to be involved in the future use of the AI, as its (future) operational interest could also be a source of bias in the testing process and the analysis of its results.

In-use measures include:

  • conducting ongoing monitoring and establish thresholds for periodic human review, with a focus on monitoring ‘degradation to the AI’s functionality and to detect changes in the AI’s impact on rights or safety’—‘human review, including renewed testing for performance of the AI in a real-world context, must be conducted at least annually, and after significant modifications to the AI or to the conditions or context in which the AI is used’;

  • mitigating emerging risks to rights and safety—crucially, ‘Where the AI’s risks to rights or safety exceed an acceptable level and where mitigation is not practicable, agencies must stop using the affected AI as soon as is practicable’. In that regard, the draft indicates that ‘Agencies are responsible for determining how to safely decommission AI that was already in use at the time of this memorandum’s release without significant disruptions to essential government functions’, but it would seem that this is also a process that would benefit from close oversight by OMB as it would otherwise jeopardise the effectiveness of the extension and waiver mechanisms discussed above—in which case additional detail in the guidance would be required;

  • ensuring adequate human training and assessment;

  • providing appropriate human consideration as part of decisions that pose a high risk to rights or safety; and

  • providing public notice and plain-language documentation through the AI use case inventory—however, this is subject a large number of caveats (notice must be ‘consistent with applicable law and governmentwide guidance, including those concerning protection of privacy and of sensitive law enforcement, national security, and other protected information’) and more detailed guidance on how to assess these issues would be welcome (if it exists, a cross-reference in the draft policy would be helpful).

additional minimum practices for rights-impacting ai

In relation to rights-affecting AI only, the Draft AI in Government Policy would require agencies to take additional measures.

Preventative measures include:

  • take steps to ensure that the AI will advance equity, dignity, and fairness—including proactively identifying and removing factors contributing to algorithmic discrimination or bias; assessing and mitigating disparate impacts; and using representative data; and

  • consult and incorporate feedback from affected groups.

In-use measures include:

  • conducting ongoing monitoring and mitigation for AI-enabled discrimination;

  • notifying negatively affected individuals—this is an area where the draft guidance is rather woolly, as it also includes a set of complex caveats, as individual notice that ‘AI meaningfully influences the outcome of decisions specifically concerning them, such as the denial of benefits’ must only be given ‘[w]here practicable and consistent with applicable law and governmentwide guidance’. Moreover, the draft only indicates that ‘Agencies are also strongly encouraged to provide explanations for such decisions and actions’, but not required to. In my view, this tackles two of the most important implications for individuals in Government use of AI: the possibility to understand why decisions are made (reason giving duties) and the burden of challenging automated decisions, which is increased if there is a lack of transparency on the automation. Therefore, on this point, the guidance seems too tepid—especially bearing in mind that this requirement only applies to ‘AI whose output serves as a basis for decision or action that has a legal, material, or similarly significant effect on an individual’s’ civil rights, civil liberties, or privacy; equal opportunities; or access to critical resources or services. In these cases, it seems clear that notice and explainability requirements need to go further.

  • maintaining human consideration and remedy processes—including ‘potential remedy to the use of the AI by a fallback and escalation system in the event that an impacted individual would like to appeal or contest the AI’s negative impacts on them. In developing appropriate remedies, agencies should follow OMB guidance on calculating administrative burden and the remedy process should not place unnecessary burden on the impacted individual. When law or governmentwide guidance precludes disclosure of the use of AI or an opportunity for an individual appeal, agencies must create appropriate mechanisms for human oversight of rights-impacting AI’. This is another crucial area concerning rights not to be subjected to fully-automated decision-making where there is no meaningful remedy. This is also an area of the guidance that requires more detail, especially as to what is the adequate balance of burdens where eg the agency can automate the undoing of negative effects on individuals identified as a result of challenges by other individuals or in the context of the broader monitoring of the functioning and effects of the rights-impacting AI. In my view, this would be an opportunity to mandate automation of remediation in a meaningful way.

  • maintaining options to opt-out where practicable.

procurement related practices

In addition to the need for agencies to be able to meet the above requirements in relation to procured AI—which will in itself create the need to cascade some of the requirements down to contractors, and which will be the object of future guidance on how to ensure that AI contracts align with the requirements—the Draft AI in Government Policy also requires that agencies procuring AI manage risks by:

  • aligning to National Values and Law by ensuring ‘that procured AI exhibits due respect for our Nation’s values, is consistent with the Constitution, and complies with all other applicable laws, regulations, and policies, including those addressing privacy, confidentiality, copyright, human and civil rights, and civil liberties’;

  • taking ‘steps to ensure transparency and adequate performance for their procured AI, including by: obtaining adequate documentation of procured AI, such as through the use of model, data, and system cards; regularly evaluating AI-performance claims made by Federal contractors, including in the particular environment where the agency expects to deploy the capability; and considering contracting provisions that incentivize the continuous improvement of procured AI’;

  • taking ‘appropriate steps to ensure that Federal AI procurement practices promote opportunities for competition among contractors and do not improperly entrench incumbents. Such steps may include promoting interoperability and ensuring that vendors do not inappropriately favor their own products at the expense of competitors’ offering’;

  • maximizing the value of data for AI; and

  • responsibly procuring Generative AI.

These high level requirements are well targeted and compliance with them would go a long way to fostering ‘responsible AI procurement’ through adequate risk mitigation in ways that still allow the procurement mechanism to harness market forces to generate value for money.

However, operationalising these requirements will be complex and the further OMB guidance should be rather detailed and practical.

Final thoughts

In my view, the AI Executive Order and the Draft AI in Government Policy lay the foundations for a significant strengthening of the governance of AI procurement with a view to embedding safeguards in public sector AI use. A crucially important characteristic in the design of these governance mechanisms is that it imposes significant duties on the agencies seeking to procure and use the AI, and it explicitly seeks to address risks of commercial capture and commercial determination. Another crucially important characteristic is that, at least in principle, use of AI is made conditional on compliance with a rather comprehensive set of preventative and in-use risk mitigation measures. The general aspects of this governance approach thus offer a very valuable blueprint for other jurisdictions considering how to boost AI procurement governance.

However, as always, the devil is in the details. One of the crucial risks in this approach to AI governance concerns a lack of independence of the entities making the relevant assessments. In the Draft AI in Government Policy, there are some risks of under-inclusion and/or excessive waivers of compliance with the relevant requirements (both explicit and implicit, through protracted processes of decommissioning of non-compliant AI), as well as a risk that ‘practical considerations’ will push compliance with the risk mitigation requirements well past the (ambitious) 1 August 2024 deadline through long or rolling extensions.

To mitigate for this, the guidance should be much clearer on the role of OMB in extension, waiver and decommissioning decisions, as well as in relation to the specific criteria and limits that should form part of those decisions. Only by ensuring adequate OMB intervention can a system of governance that still does not entirely (organisationally) separate procurement, use and oversight decisions reach the levels of independent verification required not only to neutralise commercial determination, but also operational dependency and the ‘policy irresistibility’ of digital technologies.

Response to the UK’s March 2023 White Paper "A pro-innovation approach to AI regulation"

Together with colleagues at the Centre for Global Law and Innovation of the University of Bristol Law School, I submitted a response to the UK Government’s public consultation on its ‘pro-innovation’ approach to AI regulation. For an earlier assessment, see here.

The full submission is available at https://ssrn.com/abstract=4477368, and this is the executive summary:

The white paper ‘A pro-innovation approach to AI regulation’ (the ‘AI WP’) claims to advance a ‘pro-innovation, proportionate, trustworthy, adaptable, clear and collaborative’ model that leverages the capabilities and skills of existing regulators to foster AI innovation. This model, we are told, would be underpinned by a set of principles providing a clear, unified, and flexible framework improving upon the current ‘complex patchwork of legal requirements’ and striking ‘the right balance between responding to risks and maximising opportunities.’

In this submission, we challenge such claims in the AI WP. We argue that:

  • The AI WP does not advance a balanced and proportionate approach to AI regulation, but rather, an “innovation first” approach that caters to industry and sidelines the public. The AI WP primarily serves a digital industrial policy goal ‘to make the UK one of the top places in the world to build foundational AI companies’. The public interest is downgraded and building public trust is approached instrumentally as a mechanism to promote AI uptake. Such an approach risks breaching the UK’s international obligations to create a legal framework that effectively protects fundamental rights in the face of AI risks. Additionally, in the context of public administration, poorly regulated AI could breach due process rules, putting public funds at risk.

  • The AI WP does not embrace an agile regulatory approach, but active deregulation. The AI WP stresses that the UK ‘must act quickly to remove existing barriers to innovation’ without explaining how any of the existing safeguards are no longer required in view of identified heightened AI risks. Coupled with the “innovation first” mandate, this deregulatory approach risks eroding regulatory independence and the effectiveness of the regulatory regimes the AI WP claims to seek to leverage. A more nuanced regulatory approach that builds on, rather than threatens, regulatory independence is required.

  • The AI WP builds on shaky foundations, including the absence of a mapping of current regulatory remits and powers. This makes it near impossible to assess the effectiveness and comprehensiveness of the proposed approach, although there are clear indications that regulatory gaps will remain. The AI WP also presumes continuity in the legal framework, which ignores reforms currently promoted by Government and further reforms of the overarching legal regime repeatedly floated. It seems clear that some regulatory regimes will soon see their scope or stringency limited. The AI WP does not provide clear mechanisms to address these issues, which undermine its core claim that leveraging existing regulatory regimes suffices to address potential AI harms. This is perhaps particularly evident in the context of AI use for policing, which is affected by both the existence of regulatory gaps and limitations in existing legal safeguards.

  • The AI WP does not describe a full, workable regulatory model. Lack of detail on the institutional design to support the central function is a crucial omission. Crucial tasks are assigned to such central function without clarifying its institutional embedding, resourcing, accountability mechanisms, etc.

  • The AI WP foresees a government-dominated approach that further risks eroding regulatory independence, in particular given the “innovation first” criteria to be used in assessing the effectiveness of the proposed regime.

  • The principles-based approach to AI regulation suggested in the AI WP is undeliverable due to lack of detail on the meaning and regulatory implications of the principles, barriers to translation into enforceable requirements, and tensions with existing regulatory frameworks. The minimalistic legislative intervention entertained in the AI WP would not equip regulators to effectively enforce the general principles. Following the AI WP would also result in regulatory fragmentation and uncertainty and not resolve the identified problem of a ‘complex patchwork of legal requirements’.

  • The AI WP does not provide any route towards sufficiently addressing the digital capabilities gap, or towards mitigating new risks to capabilities, such as deskilling—which create significant constraints on the likely effectiveness of the proposed approach.

Full citation: A Charlesworth, K Fotheringham, C Gavaghan, A Sanchez-Graells and C Torrible, ‘Response to the UK’s March 2023 White Paper "A pro-innovation approach to AI regulation"’ (June 19, 2023). Available at SSRN: https://ssrn.com/abstract=4477368.

Free registration open for two events on procurement and artificial intelligence

Registration is now open for two free events on procurement and artificial intelligence (AI).

First, a webinar where I will be participating in discussions on the role of procurement in contributing to the public sector’s acquisition of trustworthy AI, and the associated challenges, from an EU and US perspective.

Second, a public lecture where I will present the findings of my research project on digital technologies and public procurement.

Please scroll down for details and links to registration pages. All welcome!

1. ‘Can Procurement Be Used to Effectively Regulate AI?’ | Free online webinar
30 May 2023 2pm BST / 3pm CET-SAST / 9am EST (90 mins)
Co-organised by University of Bristol Law School and George Washington University Law School.

Artificial Intelligence (“AI”) regulation and governance is a global challenge that is starting to generate different responses in the EU, US, and other jurisdictions. Such responses are, however, rather tentative and politically contested. A full regulatory system will take time to crystallise and be fully operational. In the meantime, despite this regulatory gap, the public sector is quickly adopting AI solutions for a wide range of activities and public services.

This process of accelerated AI adoption by the public sector places procurement as the (involuntary) gatekeeper, tasked with ‘AI regulation by contract’, at least for now. The procurement function is expected to design tender procedures and contracts capable of attaining goals of AI regulation (such as trustworthiness, explainability, or compliance with data protection and human and fundamental rights) that are so far eluding more general regulation.

This webinar will provide an opportunity to take a hard look at the likely effectiveness of AI regulation by contract through procurement and its implications for the commercialisation of public governance, focusing on key issues such as:

  • The interaction between tender design, technical standards, and negotiations.

  • The challenges of designing, monitoring, and enforcing contractual clauses capable of delivering effective ‘regulation by contract’ in the AI space.

  • The tension between the commercial value of tailored contractual design and the regulatory value of default clauses and standard terms.

  • The role of procurement disputes and litigation in shaping AI regulation by contract.

  • The alternative regulatory option of establishing mandatory prior approval by an independent regulator of projects involving AI adoption by the public sector.

This webinar will be of interest to those working on or researching the digitalisation of the public sector and AI regulation in general, as the discussion around procurement gatekeeping mirrors the main issues arising from broader trends.

I will have the great opportunity of discussing my research with Aris Georgopoulos (Nottingham), Scott Simpson (Digital Transformation Lead at U.S. Department of Homeland Security), and Liz Chirico (Acquisition Innovation Lead at Office of the Deputy Assistant Secretary of the Army). Jessica Tillipman (GW Law) will moderate the discussion and Q&A.

Registration: https://law-gwu-edu.zoom.us/webinar/register/WN_w_V9s_liSiKrLX9N-krrWQ.

2. ‘AI in the public sector: can procurement promote trustworthy AI and avoid commercial capture?’ | Free in-person public lecture
4 July 2023 2pm BST, Reception Room, Wills Memorial Building, University of Bristol
Organised by University of Bristol Law School, Centre for Global Law and Innovation

The public sector is quickly adopting artificial intelligence (AI) to manage its interactions with citizens and in the provision of public services – for example, using chatbots in official websites, automated processes and call-centres, or predictive algorithms.

There are inherent high stakes risks to this process of public governance digitalisation, such as bias and discrimination, unethical deployment, data and privacy risks, cyber security risks, or risks of technological debt and dependency on proprietary solutions developed by (big) tech companies.

However, as part of the UK Government’s ‘light touch’ ‘pro-innovation’ approach to digital technology regulation, the adoption of AI in the public sector remains largely unregulated. 

In this public lecture, I will present the findings of my research funded by the British Academy, analysing how, in this deregulatory context, the existing rules on public procurement fall short of protecting the public interest.

An alternative approach is required to create mechanisms of external independent oversight and mandatory standards to embed trustworthy AI requirements and to mitigate against commercial capture in the acquisition of AI solutions. 

Registration: https://www.eventbrite.co.uk/e/can-procurement-promote-trustworthy-ai-and-avoid-commercial-capture-tickets-601212712407.

External oversight and mandatory requirements for public sector digital technology adoption

© Mateo Mulder-Graells (2023).

I thought the time would never come, but the last piece of my book project puzzle is now more or less in place. After finding that procurement is not the right regulatory actor and does not have the best tools of ‘digital regulation by contract’, in this last draft chapter, I explore how to discharge procurement of the assigned digital regulation role to increase the likelihood of effective enforcement of desirable goals of public sector digital regulation.

I argue that this should be done through two inter-related regulatory interventions consisting of developing (1) a regulator tasked with the external oversight of the adoption of digital technologies by the public sector, as well as (2) a suite of mandatory requirements binding both public entities seeking to adopt digital technologies and technology providers, and both in relation to the digital technologies to be adopted by the public sector and the applicable governance framework.

Detailed analysis of these issues would require much more extensive treatment than this draft chapter can offer. The modest goal here is simply to stress the key attributes and functions that each of these two regulatory interventions should have to make a positive contribution to governing the transition towards a new model of public digital governance. In this blog post, I summarise the main arguments.

As ever, I would be most grateful for feedback: a.sanchez-graells@bristol.ac.uk. Especially as I will now turn my attention to seeing how the different pieces of the puzzle fit together, while I edit the manuscript for submission before end of July 2023.

Institutional deficit and risk of capture

In the absence of an alternative institutional architecture (or while it is put in place), procurement is expected to develop a regulatory gatekeeping role in relation to the adoption of digital technologies by the public sector, which is in turn expected to have norm-setting and market-shaping effects across the economy. This could be seen as a way of bypassing or postponing decisions on regulatory architecture.

However, earlier analysis has shown that the procurement function is not the right institution to which to assign a digital regulation role, as it cannot effectively discharge such a duty. This highlights the existence of an institutional deficit in the process of public sector digitalisation, as well as in relation to digital technology regulation more broadly. An alternative approach to institutional design is required, and it can be delivered through the creation of a notional ‘AI in Public Sector Authority’ (AIPSA).

Earlier analysis has also shown that there are pervasive risks of regulatory capture and commercial determination of the process of public sector digitalisation stemming from reliance on standards and benchmarks created by technology vendors or by bodies heavily influenced by the tech industry. AIPSA could safeguard against such risk through controls over the process of standard adoption. AIPSA could also guard against excessive experimentation with digital technologies by creating robust controls to counteract their policy irresistibility.

Overcoming the institutional deficit through AIPSA

The adoption of digital technologies in the process of public sector digitalisation creates regulatory challenges that require external oversight, as procurement is unable to effectively regulate this process. A particularly relevant issue concerns whether such oversight should be entrusted to a new regulator (broad approach), or whether it would suffice to assign new regulatory tasks to existing regulators (narrow approach).

I submit that the narrow approach is inadequate because it perpetuates regulatory fragmentation and can lead to undesirable spillovers or knock-on effects, whether the new regulatory tasks are assigned to data protection authorities, (quasi)regulators with a ‘sufficiently close’ regulatory remit in relation with information and communications technologies (ICT) (such as eg the Agency for Digital Italy (AgID), or the Dutch Advisory Council on IT assessment (AcICT)), or newly created centres of expertise in algorithmic regulation (eg the French PEReN). Such ‘organic’ or ‘incremental’ approach to institutional development could overshadow important design considerations, as well embed biases due to the institutional drivers of the existing (quasi)regulators.

To avoid these issues, I advocate a broader or more joined up approach in the proposal for AIPSA. AIPSA would be an independent authority with the statutory function of promoting overarching goals of digital regulation, and specifically tasked with regulating the adoption and use of digital technologies by the public sector, whether through in-house development or procurement from technology providers. AIPSA would also absorb regulatory functions in cognate areas, such as the governance of public sector data, and integrate work in areas such as cyber security. It would also serve a coordinating function with the data protection authority.

In the draft chapter, I stress three fundamental aspects of AIPSA’s institutional design: regulatory coherence, independence and expertise. Independence and expertise would be the two most crucial factors. AIPSA would need to be designed in a way that ensured both political and industry independence, with the issue of political independence having particular salience and requiring countervailing accountability mechanisms. Relatedly, the importance of digital capabilities to effectively exercise a digital regulation role cannot be overemphasised. It is not only important in relation to the active aspects of the regulatory role—such as control of standard setting or permissioning or licencing of digital technology use (below)—but also in relation to the passive aspects of the regulatory role and, in particular, in relation to reactive engagement with industry. High levels of digital capability would be essential to allow AIPSA to effectively scrutinise claims from those that sought to influence its operation and decision-making, as well as reduce AIPSA’s dependence on industry-provided information.

safeguard against regulatory capture and policy irresistibility

Regulating the adoption of digital technologies in the process of public sector digitalisation requires establishing the substantive requirements that such technology needs to meet, as well as the governance requirements need to ensure its proper use. AIPSA’s role in setting mandatory requirements for public sector digitalisation would be twofold.

First, through an approval or certification mechanism, it would control the process of standardisation to neutralise risks of regulatory capture and commercial determination. Where no standards were susceptible of approval or certification, AIPSA would develop them.

Second, through a permissioning or licencing process, AIPSA would ensure that decisions on the adoption of digital technologies by the public sector are not driven by ‘policy irresistibility’, that they are supported by clear governance structures and draw on sufficient resources, and that adherence to the goals of digital regulation is sustained throughout the implementation and use of digital technologies by the public sector and subject to proactive transparency requirements.

The draft chapter provides more details on both issues.

If not AIPSA … then clearly not procurement

There can be many objections to the proposals developed in this draft chapter, which would still require further development. However, most of the objections would likely also apply to the use of procurement as a tool of digital regulation. The functions expected of AIPSA closely match those expected of the procurement function under the approach to ‘digital regulation by contract’. Challenges to AIPSA’s ability to discharge such functions would be applicable to any public buyer seeking to achieve the same goals. Similarly, challenges to the independence or need for accountability of AIPSA would be similarly applicable to atomised decision-making by public buyers.

While the proposal is necessarily imperfect, I submit that it would improve upon the emerging status quo and that, in discharging procurement of the digital regulation role, it would make a positive contribution to the governance of the transition to a new model of digital public governance.

The draft chapter is available via SSRN: Albert Sanchez-Graells, ‘Discharging procurement of the digital regulation role: external oversight and mandatory requirements for public sector digital technology adoption’.

UK's 'pro-innovation approach' to AI regulation won't do, particularly for public sector digitalisation

Regulating artificial intelligence (AI) has become the challenge of the time. This is a crucial area of regulatory development and there are increasing calls—including from those driving the development of AI—for robust regulatory and governance systems. In this context, more details have now emerged on the UK’s approach to AI regulation.

Swimming against the tide, and seeking to diverge from the EU’s regulatory agenda and the EU AI Act, the UK announced a light-touch ‘pro-innovation approach’ in its July 2022 AI regulation policy paper. In March 2023, the same approach was supported by a Report of the Government Chief Scientific Adviser (the ‘GCSA Report’), and is now further developed in the White Paper ‘AI regulation: a pro-innovation approach’ (the ‘AI WP’). The UK Government has launched a public consultation that will run until 21 June 2023.

Given the relevance of the issue, it can be expected that the public consultation will attract a large volume of submissions, and that the ‘pro-innovation approach’ will be heavily criticised. Indeed, there is an on-going preparatory Parliamentary Inquiry on the Governance of AI that has already collected a wealth of evidence exploring the pros and cons of the regulatory approach outlined there. Moreover, initial reactions eg by the Public Law Project, the Ada Lovelace Institute, or the Royal Statistical Society have been (to different degrees) critical of the lack of regulatory ambition in the AI WP—while, as could be expected, think tanks closely linked to the development of the policy, such as the Alan Turing Institute, have expressed more positive views.

Whether the regulatory approach will shift as a result of the expected pushback is unclear. However, given that the AI WP follows the same deregulatory approach first suggested in 2018 and is strongly politically/policy entrenched—for the UK Government has self-assessed this approach as ‘world leading’ and claims it will ‘turbocharge economic growth’—it is doubtful that much will necessarily change as a result of the public consultation.

That does not mean we should not engage with the public consultation, but the opposite. In the face of the UK Government’s dereliction of duty, or lack of ideas, it is more important than ever that there is a robust pushback against the deregulatory approach being pursued. Especially in the context of public sector digitalisation and the adoption of AI by the public administration and in the provision of public services, where the Government (unsurprisingly) is unwilling to create regulatory safeguards to protect citizens from its own action.

In this blogpost, I sketch my main areas of concern with the ‘pro-innovation approach’ in the GCSA Report and AI WP, which I will further develop for submission to the public consultation, building on earlier views. Feedback and comments would be gratefully received: a.sanchez-graells@bristol.ac.uk.

The ‘pro-innovation approach’ in the GCSA Report — squaring the circle?

In addition to proposals on the intellectual property (IP) regulation of generative AI, the opening up of public sector data, transport-related, or cyber security interventions, the GCSA Report focuses on ‘core’ regulatory and governance issues. The report stresses that regulatory fragmentation is one of the key challenges, as is the difficulty for the public sector in ‘attracting and retaining individuals with relevant skills and talent in a competitive environment with the private sector, especially those with expertise in AI, data analytics, and responsible data governance‘ (at 5). The report also further hints at the need to boost public sector digital capabilities by stressing that ‘the government and regulators should rapidly build capability and know-how to enable them to positively shape regulatory frameworks at the right time‘ (at 13).

Although the rationale is not very clearly stated, to bridge regulatory fragmentation and facilitate the pooling of digital capabilities from across existing regulators, the report makes a central proposal to create a multi-regulator AI sandbox (at 6-8). The report suggests that it could be convened by the Digital Regulatory Cooperation Forum (DRCF)—which brings together four key regulators (the Information Commissioner’s Office (ICO), Office of Communications (Ofcom), the Competition and Markets Authority (CMA) and the Financial Conduct Authority (FCA))—and that DRCF should look at ways of ‘bringing in other relevant regulators to encourage join up’ (at 7).

The report recommends that the AI sandbox should operate on the basis of a ‘commitment from the participant regulators to make joined-up decisions on regulations or licences at the end of each sandbox process and a clear feedback loop to inform the design or reform of regulatory frameworks based on the insights gathered. Regulators should also collaborate with standards bodies to consider where standards could act as an alternative or underpin outcome-focused regulation’ (at 7).

Therefore, the AI sandbox would not only be multi-regulator, but also encompass (in some way) standard-setting bodies (presumably UK ones only, though), without issues of public-private interaction in decision-making implying the exercise of regulatory public powers, or issues around regulatory capture and risks of commercial determination, being considered at all. The report in general is extremely industry-orientated, eg in stressing in relation to the overarching pacing problem that ‘for emerging digital technologies, the industry view is clear: there is a greater risk from regulating too early’ (at 5), without this being in any way balanced with clear (non-industry) views that the biggest risk is actually in regulating too late and that we are collectively frog-boiling into a ‘runaway AI’ fiasco.

Moreover, confusingly, despite the fact that the sandbox would be hosted by DRCF (of which the ICO is a leading member), the GCSA Report indicates that the AI sandbox ‘could link closely with the ICO sandbox on personal data applications’ (at 8). The fact that the report is itself unclear as to whether eg AI applications with data protection implications should be subjected to one or two sandboxes, or the extent to which the general AI sandbox would need to be integrated with sectoral sandboxes for non-AI regulatory experimentation, already indicates the complexity and dubious practical viability of the suggested approach.

It is also unclear why multiple sector regulators should be involved in any given iteration of a single AI sandbox where there may be no projects within their regulatory remit and expertise. The alternative approach of having an open or rolling AI sandbox mechanism led by a single AI authority, which would then draw expertise and work in collaboration with the relevant sector regulator as appropriate on a per-project basis, seems preferable. While some DRCF members could be expected to have to participate in a majority of sandbox projects (eg CMA and ICO), others would probably have a much less constant presence (eg Ofcom, or certainly the FCA).

Remarkably, despite this recognition of the functional need for a centralised regulatory approach and a single point of contact (primarily for industry’s convenience), the GCSA Report implicitly supports the 2022 AI regulation policy paper’s approach to not creating an overarching cross-sectoral AI regulator. The GCSA Report tries to create a ‘non-institutionalised centralised regulatory function’, nested under DRCF. In practice, however, implementing the recommendation for a single AI sandbox would create the need for the further development of the governance structures of the DRCF (especially if it was to grow by including many other sectoral regulators), or whichever institution ‘hosted it’, or else risk creating a non-institutional AI regulator with the related difficulties in ensuring accountability. This would add a layer of deregulation to the deregulatory effect that the sandbox itself creates (see eg Ranchordas (2021)).

The GCSA Report seems to try to square the circle of regulatory fragmentation by relying on cooperation as a centralising regulatory device, but it does this solely for the industry’s benefit and convenience, without paying any consideration to the future effectiveness of the regulatory framework. This is hard to understand, given the report’s identification of conflicting regulatory constraints, or in its terminology ‘incentives’: ‘The rewards for regulators to take risks and authorise new and innovative products and applications are not clear-cut, and regulators report that they can struggle to trade off the different objectives covered by their mandates. This can include delivery against safety, competition objectives, or consumer and environmental protection, and can lead to regulator behaviour and decisions that prioritise further minimising risk over supporting innovation and investment. There needs to be an appropriate balance between the assessment of risk and benefit’ (at 5).

This not only frames risk-minimisation as a negative regulatory outcome (and further feeds into the narrative that precautionary regulatory approaches are somehow not legitimate because they run against industry goals—which deserves strong pushback, see eg Kaminski (2022)), but also shows a main gap in the report’s proposal for the single AI sandbox. If each regulator has conflicting constraints, what evidence (if any) is there that collaborative decision-making will reduce, rather than exacerbate, such regulatory clashes? Are decisions meant to be arrived at by majority voting or in any other way expected to deactivate (some or most) regulatory requirements in view of (perceived) gains in relation to other regulatory goals? Why has there been no consideration of eg the problems encountered by concurrency mechanisms in the application of sectoral and competition rules (see eg Dunne (2014), (2020) and (2021)), as an obvious and immediate precedent of the same type of regulatory coordination problems?

The GCSA report also seems to assume that collaboration through the AI sandbox would be resource neutral for participating regulators, whereas it seems reasonable to presume that this additional layer of regulation (even if not institutionalised) would require further resources. And, in any case, there does not seem to be much consideration as to the viability of asking of resource-strapped regulators to create an AI sandbox where they can (easily) be out-skilled and over-powered by industry participants.

In my view, the GCSA Report already points at significant weaknesses in the resistance to creating any new authorities, despite the obvious functional need for centralised regulation, which is one of the main weaknesses, or the single biggest weakness, in the AI WP—as well as in relation to a lack of strategic planning around public sector digital capabilities, despite well-recognised challenges (see eg Committee of Public Accounts (2021)).

The ‘pro-innovation approach’ in the AI WP — a regulatory blackhole, privatisation of ai regulation, or both

The AI WP envisages an ‘innovative approach to AI regulation [that] uses a principles-based framework for regulators to interpret and apply to AI within their remits’ (para 36). It expects the framework to ‘pro-innovation, proportionate, trustworthy, adaptable, clear and collaborative’ (para 37). As will become clear, however, such ‘innovative approach’ solely amounts to the formulation of high-level, broad, open-textured and incommensurable principles to inform a soft law push to the development of regulatory practices aligned with such principles in a highly fragmented and incomplete regulatory landscape.

The regulatory framework would be built on four planks (para 38): [i] an AI definition (paras 39-42); [ii] a context-specific approach (ie a ‘used-based’ approach, rather than a ‘technology-led’ approach, see paras 45-47); [iii] a set of cross-sectoral principles to guide regulator responses to AI risks and opportunities (paras 48-54); and [iv] new central functions to support regulators to deliver the AI regulatory framework (paras 70-73). In reality, though, there will be only two ‘pillars’ of the regulatory framework and they do not involve any new institutions or rules. The AI WP vision thus largely seems to be that AI can be regulated in the UK in a world-leading manner without doing anything much at all.

AI Definition

The UK’s definition of AI will trigger substantive discussions, especially as it seeks to build it around ‘the two characteristics that generate the need for a bespoke regulatory response’: ‘adaptivity’ and ‘autonomy’ (para 39). Discussing the definitional issue is beyond the scope of this post but, on the specific identification of the ‘autonomy’ of AI, it is worth highlighting that this is an arguably flawed regulatory approach to AI (see Soh (2023)).

No new institutions

The AI WP makes clear that the UK Government has no plans to create any new AI regulator, either with a cross-sectoral (eg general AI authority) or sectoral remit (eg an ‘AI in the public sector authority’, as I advocate for). The Ministerial Foreword to the AI WP already stresses that ‘[t]o ensure our regulatory framework is effective, we will leverage the expertise of our world class regulators. They understand the risks in their sectors and are best placed to take a proportionate approach to regulating AI’ (at p2). The AI WP further stresses that ‘[c]reating a new AI-specific, cross-sector regulator would introduce complexity and confusion, undermining and likely conflicting with the work of our existing expert regulators’ (para 47). This however seems to presume that a new cross-sector AI regulator would be unable to coordinate with existing regulators, despite the institutional architecture of the regulatory framework foreseen in the AI WP entirely relying on inter-regulator collaboration (!).

No new rules

There will also not be new legislation underpinning regulatory activity, although the Government claims that the WP AI, ‘alongside empowering regulators to take a lead, [is] also setting expectations‘ (at p3). The AI WP claims to develop a regulatory framework underpinned by five principles to guide and inform the responsible development and use of AI in all sectors of the economy: [i] Safety, security and robustness; [ii] Appropriate transparency and explainability; [iii] Fairness; [iv] Accountability and governance; and [v] Contestability and redress (para 10). However, they will not be put on a statutory footing (initially); ‘the principles will be issued on a non-statutory basis and implemented by existing regulators’ (para 11). While there is some detail on the intended meaning of these principles (see para 52 and Annex A), the principles necessarily lack precision and, worse, there is a conflation of the principles with other (existing) regulatory requirements.

For example, it is surprising that the AI WP describes fairness as implying that ‘AI systems should (sic) not undermine the legal rights of individuals or organisations, discriminate unfairly against individuals or create unfair market outcomes‘ (emphasis added), and stresses the expectation ‘that regulators’ interpretations of fairness will include consideration of compliance with relevant law and regulation’ (para 52). This encapsulates the risks that principles-based AI regulation ends up eroding compliance with and enforcement of current statutory obligations. A principle of AI fairness cannot modify or exclude existing legal obligations, and it should not risk doing so either.

Moreover, the AI WP suggests that, even if the principles are supported by a statutory duty for regulators to have regard to them, ‘while the duty to have due regard would require regulators to demonstrate that they had taken account of the principles, it may be the case that not every regulator will need to introduce measures to implement every principle’ (para 58). This conflates two issues. On the one hand, the need for activity subjected to regulatory supervision to comply with all principles and, on the other, the need for a regulator to take corrective action in relation to any of the principles. It should be clear that regulators have a duty to ensure that all principles are complied with in their regulatory remit, which does not seem to entirely or clearly follow from the weaker duty to have due regard to the principles.

perpetuating regulatory gaps, in particular regarding public sector digitalisation

As a consequence of the lack of creation of new regulators and the absence of new legislation, it is unclear whether the ‘regulatory strategy’ in the AI WP will have any real world effects within existing regulatory frameworks, especially as the most ambitious intervention is to create ‘a statutory duty on regulators requiring them to have due regard to the principles’ (para 12)—but the Government may decide not to introduce it if ‘monitoring of the effectiveness of the initial, non-statutory framework suggests that a statutory duty is unnecessary‘ (para 59).

However, what is already clear that there is no new AI regulation in the horizon despite the fact that the AI WP recognises that ‘some AI risks arise across, or in the gaps between, existing regulatory remits‘ (para 27), that ‘there may be AI-related risks that do not clearly fall within the remits of the UK’s existing regulators’ (para 64), and the obvious and worrying existence of high risks to fundamental rights and values (para 4 and paras 22-25). The AI WP is naïve, to say the least, in setting out that ‘[w]here prioritised risks fall within a gap in the legal landscape, regulators will need to collaborate with government to identify potential actions. This may include identifying iterations to the framework such as changes to regulators’ remits, updates to the Regulators’ Code, or additional legislative intervention’ (para 65).

Hoping that such risk identification and gap analysis will take place without assigning specific responsibility for it—and seeking to exempt the Government from such responsibility—seems a bit too much to ask. In fact, this is at odds with the graphic depiction of how the AI WP expects the system to operate. As noted in (1) in the graph below, it is clear that the identification of risks that are cross-cutting or new (unregulated) risks that warrant intervention is assigned to a ‘central risk function’ (more below), not the regulators. Importantly, the AI WP indicates that such central function ‘will be provided from within government’ (para 15 and below). Which then raises two questions: (a) who will have the responsibility to proactively screen for such risks, if anyone, and (b) how has the Government not already taken action to close the gaps it recognises exists in the current legal landscape?

AI WP Figure 2: Central risks function activities.

This perpetuates the current regulatory gaps, in particular in sectors without a regulator or with regulators with very narrow mandates—such as the public sector and, to a large extent, public services. Importantly, this approach does not create any prohibition of impermissible AI uses, nor sets any (workable) set of minimum requirements for the deployment of AI in high-risk uses, specially in the public sector. The contrast with the EU AI Act could not be starker and, in this aspect in particular, UK citizens should be very worried that the UK Government is not committing to any safeguards in the way technology can be used in eg determining access to public services, or by the law enforcement and judicial system. More generally, it is very worrying that the AI WP does not foresee any safeguards in relation to the quickly accelerating digitalisation of the public sector.

Loose central coordination leading to ai regulation privatisation

Remarkably, and in a similar functional disconnect as that of the GCSA Report (above), the decision not to create any new regulator/s (para 15) is taken in the same breath as the AI WP recognises that the small coordination layer within the regulatory architecture proposed in the 2022 AI regulation policy paper (ie, largely, the approach underpinning the DRCF) has been heavily criticised (para 13). The AI WP recognises that ‘the DRCF was not created to support the delivery of all the functions we have identified or the implementation of our proposed regulatory framework for AI’ (para 74).

The AI WP also stresses how ‘[w]hile some regulators already work together to ensure regulatory coherence for AI through formal networks like the AI and digital regulations service in the health sector and the Digital Regulation Cooperation Forum (DRCF), other regulators have limited capacity and access to AI expertise. This creates the risk of inconsistent enforcement across regulators. There is also a risk that some regulators could begin to dominate and interpret the scope of their remit or role more broadly than may have been intended in order to fill perceived gaps in a way that increases incoherence and uncertainty’ (para 29), which points at a strong functional need for a centralised approach to AI regulation.

To try and mitigate those regulatory risks and shortcomings, the AI WP proposes the creation of ‘a number of central support functions’, such as [i} a central monitoring function of overall regulatory framework’s effectiveness and the implementation of the principles; [ii] central risk monitoring and assessment; [iii] horizon scanning; [iv] supporting testbeds and sandboxes; [v] advocacy, education and awareness-raising initiatives; or [vi] promoting interoperability with international regulatory frameworks (para 14, see also para 73). Cryptically, the AI WP indicates that ‘central support functions will initially be provided from within government but will leverage existing activities and expertise from across the broader economy’ (para 15). Quite how this can be effectively done outwith a clearly defined, adequately resourced and durable institutional framework is anybody’s guess. In fact, the AI WP recognises that this approach ‘needs to evolve’ and that Government needs to understand how ‘existing regulatory forums could be expanded to include the full range of regulators‘, what ‘additional expertise government may need’, and the ‘most effective way to convene input from across industry and consumers to ensure a broad range of opinions‘ (para 77).

While the creation of a regulator seems a rather obvious answer to all these questions, the AI WP has rejected it in unequivocal terms. Is the AI WP a U-turn waiting to happen? Is the mention that ‘[a]s we enter a new phase we will review the role of the AI Council and consider how best to engage expertise to support the implementation of the regulatory framework’ (para 78) a placeholder for an imminent project to rejig the AI Council and turn it into an AI regulator? What is the place and role of the Office for AI and the Centre for Data Ethics and Innovation in all this?

Moreover, the AI WP indicates that the ‘proposed framework is aligned with, and supplemented by, a variety of tools for trustworthy AI, such as assurance techniques, voluntary guidance and technical standards. Government will promote the use of such tools’ (para 16). Relatedly, the AI WP relies on those mechanisms to avoid addressing issues of accountability across AI life cycle, indicating that ‘[t]ools for trustworthy AI like assurance techniques and technical standards can support supply chain risk management. These tools can also drive the uptake and adoption of AI by building justified trust in these systems, giving users confidence that key AI-related risks have been identified, addressed and mitigated across the supply chain’ (para 84). Those tools are discussed in much more detail in part 4 of the AI WP (paras 106 ff). Annex A also creates a backdoor for technical standards to directly become the operationalisation of the general principles on which the regulatory framework is based, by explicitly identifying standards regulators may want to consider ‘to clarify regulatory guidance and support the implementation of risk treatment measures’.

This approach to the offloading of tricky regulatory issues to the emergence of private-sector led standards is simply an exercise in the transfer of regulatory power to those setting such standards, guidance and assurance techniques and, ultimately, a privatisation of AI regulation.

A different approach to sandboxes and testbeds?

The Government will take forward the GCSA recommendation to establish a regulatory sandbox for AI, which ‘will bring together regulators to support innovators directly and help them get their products to market. The sandbox will also enable us to understand how regulation interacts with new technologies and refine this interaction where necessary’ (p2). This thus is bound to hardwire some of the issues mentioned above in relation to the GCSA proposal, as well as being reflective of the general pro-industry approach of the AI WP, which is obvious in the framing that the regulators are expected to ‘support innovators directly and help them get their products to market’. Industrial policy seems to be shoehorned and mainstreamed across all areas of regulatory activity, at least in relation to AI (but it can then easily bleed into non-AI-related regulatory activities).

While the AI WP indicates the commitment to implement the AI sandbox recommended in the GCSA Report, it is by no means clear that the implementation will be in the way proposed in the report (ie a multi-regulator sandbox nested under DRCF, with an expectation that it would develop a crucial coordination and regulatory centralisation effect). The AI WP indicates that the Government still has to explore ‘what service focus would be most useful to industry’ in relation to AI sandboxes (para 96), but it sets out the intention to ‘focus an initial pilot on a single sector, multiple regulator sandbox’ (para 97), which diverges from the approach in the GCSA Report, which would be that of a sandbox for ‘multiple sectors, multiple regulators’. While the public consultation intends to gather feedback on which industry sector is the most appropriate, I would bet that the financial services sector will be chosen and that the ‘regulatory innovation’ will simply result in some closer cooperation between the ICO and FCA.

Regulator capabilities — ai regulation on a shoestring?

The AI WP turns to the issue of regulator capabilities and stresses that ‘While our approach does not currently involve or anticipate extending any regulator’s remit, regulating AI uses effectively will require many of our regulators to acquire new skills and expertise’ (para 102), and that the Government has ‘identified potential capability gaps among many, but not all, regulators’ (para 103).

To try to (start to) address this fundamental issue in the context of a devolved and decentralised regulatory framework, the AI WP indicates that the Government will explore, for example, whether it is ‘appropriate to establish a common pool of expertise that could establish best practice for supporting innovation through regulatory approaches and make it easier for regulators to work with each other on common issues. An alternative approach would be to explore and facilitate collaborative initiatives between regulators – including, where appropriate, further supporting existing initiatives such as the DRCF – to share skills and expertise’ (para 105).

While the creation of ‘common regulatory capacity’ has been advocated by the Alan Turing Institute, and while this (or inter-regulator secondments, for example) could be a short term fix, it seems that this tries to address the obvious challenge of adequately resourcing regulatory bodies without a medium and long-term strategy to build up the digital capability of the public sector, and to perpetuate the current approach to AI regulation on a shoestring. The governance and organisational implications arising from the creation of common pool of expertise need careful consideration, in particular as some of the likely dysfunctionalities are only marginally smaller than current over-reliance on external consultants, or the ‘salami-slicing’ approach to regulatory and policy interventions that seems to bleed from the ’agile’ management of technological projects into the realm of regulatory activity, which however requires institutional memory and the embedding of knowledge and expertise.

Two roles of procurement in public sector digitalisation: gatekeeping and experimentation

In a new draft chapter for my monograph, I explore how, within the broader process of public sector digitalisation, and embroiled in the general ‘race for AI’ and ‘race for AI regulation’, public procurement has two roles. In this post, I summarise the main arguments (all sources, included for quoted materials, are available in the draft chapter).

This chapter frames the analysis in the rest of the book and will be fundamental in the review of the other drafts, so comments would be most welcome (a.sanchez-graells@bristol.ac.uk).

Public sector digitalisation is accelerating in a regulatory vacuum

Around the world, the public sector is quickly adopting digital technologies in virtually every area of its activity, including the delivery of public services. States are not solely seeking to digitalise their public sector and public services with a view to enhance their operation (internal goal), but are also increasingly willing to use the public sector and the construction of public infrastructure as sources of funding and spaces for digital experimentation, to promote broader technological development and boost national industries in a new wave of (digital) industrial policy (external goal). For example, the European Commission clearly seeks to make the ‘public sector a trailblazer for using AI’. This mirrors similar strategic efforts around the globe. The process of public sector digitalisation is thus embroiled in the broader race for AI.

Despite the fact that such dynamic of public sector digitalisation raises significant regulatory risks and challenges, well-known problems in managing uncertainty in technology regulation—ie the Collingridge dilemma or pacing problem (‘cannot effectively regulate early on, so will probably regulate too late’)—and different normative positions, interact with industrial policy considerations to create regulatory hesitation and side-line anticipatory approaches. This creates a regulatory gap —or rather a laissez faire environment—whereby the public sector is allowed to experiment with the adoption of digital technologies without clear checks and balances. The current strategy is by and large one of ‘experiment first, regulate later’. And while there is little to no regulation, there is significant experimentation and digital technology adoption by the public sector.

Despite the emergence of a ‘race for AI regulation’, there are very few attempts to regulate AI use in the public sector—with the EU’s proposed EU AI Act offering a (partial) exception—and general mechanisms (such as judicial review) are proving slow to adapt. The regulatory gap is thus likely to remain, at least partially, in the foreseeable future—not least, as the effective functioning of new rules such as the EU AI Act will not be immediate.

Procurement emerges as a regulatory gatekeeper to plug that gap

In this context, proposals have started to emerge to use public procurement as a tool of digital regulation. Or, in other words, to use the acquisition of digital technologies by the public sector as a gateway to the ‘regulation by contract’ of their use and governance. Think tanks, NGOs, and academics alike have stressed that the ‘rules governing the acquisition of algorithmic systems by governments and public agencies are an important point of intervention in ensuring their accountable use’, and that procurement ‘is a central policy tool governments can deploy to catalyse innovation and influence the development of solutions aligned with government policy and society’s underlying values’. Public procurement is thus increasingly expected to play a crucial gatekeeping role in the adoption of digital technologies for public governance and the delivery of public services.

Procurement is thus seen as a mechanism of ‘regulation by contract’ whereby the public buyer can impose requirements seeking to achieve broad goals of digital regulation, such as transparency, trustworthiness, or explainability, or to operationalise more general ‘AI ethics’ frameworks. In more detail, the Council of Europe has recommended using procurement to: (i) embed requirements of data governance to avoid violations of human rights norms and discrimination stemming from faulty datasets used in the design, development, or ongoing deployment of algorithmic systems; (ii) ‘ensure that algorithmic design, development and ongoing deployment processes incorporate safety, privacy, data protection and security safeguards by design’; (iii) require ‘public, consultative and independent evaluations of the lawfulness and legitimacy of the goal that the [procured algorithmic] system intends to achieve or optimise, and its possible effects in respect of human rights’; (iv) require the conduct of human rights impact assessments; or (v) promote transparency of the ‘use, design and basic processing criteria and methods of algorithmic systems’.

Given the absence of generally applicable mandatory requirements in the development and use of digital technologies by the public sector in relation to some or all of the stated regulatory goals, the gatekeeping role of procurement in digital ‘regulation by contract’ would mostly involve the creation of such self-standing obligations—or at least the enforcement of emerging non-binding norms, such as those developed by (voluntary) standardisation bodies or, more generally, by the technology industry. In addition to creating risks of regulatory capture and commercial determination, this approach may overshadow the difficulties in using procurement for the delivery of the expected regulatory goals. A closer look at some selected putative goals of digital regulation by contract sheds light on the issue.

Procurement is not at all suited to deliver incommensurable goals of digital regulation

Some of the putative goals of digital regulation by contract are incommensurable. This is the case in particular of ‘trustworthiness’ or ‘responsibility’ in AI use in the public sector. Trustworthiness or responsibility in the adoption of AI can have several meanings, and defining what is ‘trustworthy AI’ or ‘responsible AI’ is in itself contested. This creates a risk of imprecision or generality, which could turn ‘trustworthiness’ or ‘responsibility’ into mere buzzwords—as well as exacerbate the problem of AI ethics-washing. As the EU approach to ‘trustworthy AI’ evidences, the overarching goals need to be broken down to be made operational. In the EU case, ‘trustworthiness’ is intended to cover three requirements for lawful, ethical, and robust AI. And each of them break down into more detailed or operationalizable requirements.

In turn, some of the goals into which ‘trustworthiness’ or ‘responsibility’ breaks down are also incommensurable. This is notably the case of ‘explainability’ or interpretability. There is no such thing as ‘the explanation’ that is required in relation to an algorithmic system, as explanations are (technically and legally) meant to serve different purposes and consequently, the design of the explainability of an AI deployment needs to take into account factors such as the timing of the explanation, its (primary) audience, the level of granularity (eg general or model level, group-based, or individual explanations), or the level of risk generated by the use of the technical solution. Moreover, there are different (and emerging) approaches to AI explainability, and their suitability may well be contingent upon the specific intended use or function of the explanation. And there are attributes or properties influencing the interpretability of a model (eg clarity) for which there are no evaluation metrics (yet?). Similar issues arise with other putative goals, such as the implementation of a principle of AI minimisation in the public sector.

Given the way procurement works, it is ill-suited for the delivery of incommensurable goals of digital regulation.

Procurement is not well suited to deliver other goals of digital regulation

There are other goals of digital regulation by contract that are seemingly better suited to delivery through procurement, such as those relating to ‘technical’ characteristics such as neutrality, interoperability, openness, or cyber security, or in relation to procurement-adjacent algorithmic transparency. However, the operationalisation of such requirements in a procurement context will be dependent on a range of considerations, such as judgements on the need to keep information confidential, judgements on the state of the art or what constitutes a proportionate and economically justified requirement, the generation of systemic effects that are hard to evaluate within the limits of a procurement procedure, or trade-offs between competing considerations. The extent to which procurement will be able to operationalise the desired goals of digital regulation will depend on its institutional embeddedness and on the suitability of procurement tools to impose specific regulatory approaches. Additional analysis conducted elsewhere (see here and here) suggests that, also in relation to these regulatory goals, the emerging approach to AI ‘regulation by contract’ cannot work well.

Procurement digitalisation offers a valuable case study

The theoretical analysis of the use of procurement as a tool of digital ‘regulation by contract’ (above) can be enriched and further developed with an in-depth case study of its practical operation in a discrete area of public sector digitalisation. To that effect, it is important to identify an area of public sector digitalisation which is primarily or solely left to ‘regulation by contract’ through procurement—to isolate it from the interaction with other tools of digital regulation (such as data protection, or sectoral regulation). It is also important for the chosen area to demonstrate a sufficient level of experimentation with digitalisation, so that the analysis is not a mere concretisation of theoretical arguments but rather grounded on empirical insights.

Public procurement is itself an area of public sector activity susceptible to digitalisation. The adoption of digital tools is seen as a potential source of improvement and efficiency in the expenditure of public funds through procurement, especially through the adoption of digital technology solutions developed in the context of supply chain management and other business operations in the private sector (or ‘ProcureTech’), but also through the adoption of digital tools tailored to the specific goals of procurement regulation, such as the prevention of corruption or collusion. There is emerging evidence of experimentation in procurement digitalisation, which is shedding light on regulatory risks and challenges.

In view of its strategic importance and the current pace of procurement digitalisation, it is submitted that procurement is an appropriate site of public sector experimentation in which to explore the shortcomings of the approach to AI ‘regulation by contract’. Procurement is an adequate case study because, being a ‘back-office’ function, it does not concern (likely) high-risk uses of AI or other digital technologies, and it is an area where data protection regulation is unlikely to provide a comprehensive regulatory framework (eg for decision automation) because the primary interactions are between public buyers and corporate institutions.

Procurement therefore currently represents an unregulated digitalisation space in which to test and further explore the effectiveness of the ‘regulation by contract’ approach to governing the transition to a new model of digital public governance.

* * * * * *

The full draft is available on SSRN as: Albert Sanchez-Graells, ‘The two roles of procurement in the transition towards digital public governance: procurement as regulatory gatekeeper and as site for public sector experimentation’ (March 10, 2023): https://ssrn.com/abstract=4384037.

Micro-purchases as political football? -- some thoughts on the UK's GPC files and needed regulatory reform

The issue of public micro-purchases has just gained political salience in the UK. The opposition Labour party has launched a dedicated website and an aggressive media campaign calling citizens to scrutinise the use of government procurement cards (GPCs). The analysis revealed so far and the political spin being put on it question the current government’s wastefulness and whether ‘lavish’ GPC expenses are adequate and commensurate with the cost of living crisis and other social pressures. Whether this will yield the political results Labour hopes for is anybody’s guess (I am sceptical), but this is an opportunity to revisit GPC regulation and to action long-standing National Audit Office recommendations on transparency and controls, as well as to reconsider the interaction between GPCs and procurement vehicles based on data analysis. The political football around the frugality expected of a government in times of economic crisis should not obscure the clear need to strengthen GPC regulation in the UK.

Background

GPCs are debit or credit cards that allow government officials to pay vendors directly. In the UK, their issue is facilitated by a framework agreement run by the Crown Commercial Service. These cards are presented as a means to accelerate payment to public vendors (see eg current UK policy). However, their regulatory importance goes beyond their providing an (agile) means of payment, as they generate the risk of public purchases bypassing procurement procedures. If a public official can simply interact with a vendor of their choice and ‘put it on the card’, this can be a way to funnel public funds and engage with direct awards outside procurement procedures. There is thus a clear difference between the use of GPCs within procurement transactions (eg to pay for call-offs within a pre-existing framework agreement) and their use instead of procurement transactions (eg a public official buying something off your preferred online retailer and paying with a card).

Uses within procurement seem rather uncontroversial and the specific mechanism used to pay invoices should be driven by administrative efficiency considerations. There are also good reasons for (some) government officials to hold a GPC to cover the types of expenses that are difficult to procure (eg those linked to foreign travel, or unavoidably ‘spontaneous’ expenses, such as those relating to hospitality). In those cases, GPCs substitute for either the need to provide officials with cash advances (and thus create much sounder mechanisms to control the expenditure, as well as avoiding the circulation of cash with its own corruption and other risks), or to force them to pay in advance from their private pockets and then claim reimbursement (which can put many a public sector worker in financial difficulties, as eg academics know all too well).

The crucial issue then becomes how to control the expenditure under the GPCs and how to impose limits that prevent the bypassing of procurement rules and existing mechanisms. From this perspective, procurement cards are not a new phenomenon at all, and the challenges they pose from a procurement and government contracting perspective have long been understood and discussed—see eg Steven L Schooner and Neil S Whiteman, ‘Purchase Cards and Micro-Purchases: Sacrificing Traditional United States Procurement Policies at the Alter of Efficiency’ (2000) 9 Public Procurement Law Review 148. The UK’s National Audit Office (NAO) also carried out an in-depth investigation and published a report on the issue in 2012.

The regulatory and academic recommendations seeking to ensure probity and value for money in the use of GPCs as a (procurement) mechanism generally address three issues: (1) limits on expenditure, (2) (internal) expenditure control, and (3) expenditure transparency. I would add a fourth issue, which relates to (4) bypassing existing (or easy to set up) procurement frameworks. It is worth noting that the GPC files report provides useful information on each of these issues, all of which requires rethinking in the context of the UK’s current process of reforming procurement law.

Expenditure limits

The GPC files show how there are three relevant value thresholds: the threshold triggering expenditure transparency (currently £500), the maximum single transaction limit (currently £20,000, which raised the pre-pandemic £10,000), and the maximum monthly expenditure (currently £100,000, which raised the pre-pandemic limits if they were lower). It is worth assessing these limits from the perspective of their interaction with procurement rules, as well as broader considerations.

The first consideration is that the £500 threshold triggering expenditure transparency has remained fixed since 2011. Given a cumulative inflation of close to 30% in the period 2011-2022, this means that the threshold has constantly been lower in comparative purchase parity. This should make us reconsider the relevance of some of the findings in the GPC files. Eg the fact that, within its scope, there were ‘65,824 transactions above £500 in 2021, compared to 35,335 in 2010-11’ is not very helpful. This raises questions on the adequacy of having a (fixed) threshold below which expenditure is not published. While the NAO was reluctant to recommend full transparency in 2011, it seems that the administrative burden of providing such transparency has massively lowered in the intervening period, so this may be the time to scrape the transparency threshold. As below, however, this does not mean that the information should be immediately published in open data (as below).

The single transaction limit is the one with the most relevance from a procurement perspective. If a public official can use a GPC for a value exceeding the threshold of regulated procurement, then the rules are not well aligned and there is a clear regulatory risk. Under current UK law, central government contracts with a value above £12,000 must be advertised. This would be kept as the general rule in the Procurement Bill (clause 86(4)), unless there are further amendments prior to its entry into force. This evidences a clear regulatory risk of bypassing procurement (advertising) obligations through GPC use. The single transaction limit should be brought back to pre-pandemic levels (£10,000) or, at least, to the value threshold triggering procurement obligations (£12,000).

The maximum monthly expenditure should be reassessed from an (internal) control perspective (as below), but the need to ensure that GPCs cannot be used to fraction (above threshold) direct awards over short periods of time should also be taken into consideration. From that perspective, ensuring that a card holder cannot spend more than eg £138,760 in a given category of goods or services per month (which is the relevant threshold under both current rules and the foreseen Procurement Bill). Current data analytics in basic banking applications should facilitate such classification and limitation.

(internal) expenditure controls

The GPC files raise questions not only on the robustness of internal controls, but also on the accounting underpinning them (see pp 11-12). Most importantly, there seems to be no meaningful internal post-expenditure control to check for accounting problems or suspected fraudulent use, or no willingness to disclose how any such mechanisms operate. This creates expenditure control opacity that can point to a big governance gap. Expenditure controls should not only apply at the point of deciding who to authorise to hold and use a GPC and up to which expenditure limit, but also (and perhaps more importantly), to how expenditure is being carried out. From a regulatory theory perspective, it is very clear that the use of GPCs is framed under an agency relationship and it is very important to continuously signal to the agent that the principal is monitoring the use of the card and that there are serious (criminal) consequences to misuse. As things stand, it seems that ex post internal controls may operate in some departments (eg those that report recovery for inappropriately used funds) but not (effectively) in others. This requires urgent review of the mechanisms of pre- and post-expenditure control. An update of the 2012 NAO report seems necessary.

Expenditure transparency

The GPC files (pp 10-11) show clear problems in the implementation of the policy of disclosing all expenditure in transactions exceeding £500, which should be published published monthly, 2 months in arrears, despite (relatively clear) guidance to that effect. In addition to facilitating the suppression of the transparency threshold, developments in the collection and publication of open data should also facilitate the rollout of a clear plan to ensure effective publication without the gaps identified in the GPC files (and other problems in practice). However, this is also a good time to carefully consider the purpose of these publications and the need to harmonise them with the publication of other procurement information.

There are conflicting issues at hand. First, the current policy of publishing 2 months in arrears does not seem justified in relation to some qualified users of that information, such as those with an oversight role (or fraud investigation powers). Second, in relation to the general public, publication in full of all details may not be adequate within that time period in some cases, and the publication of some information may not be appropriate at all. There are, of course, intermediate situations, such as data access for journalists of research academics. In relation to this data, as well as all procurement data, this is an opportunity to create a sophisticated data-management architecture that can handle of multi-tiered access to different types of information at different times, by different stakeholders and under different conditions (see here and here).

bypassing procurement frameworks

A final consideration is that the GPC files evidence a risk that GPCs may be used in ways that bypass existing procurement frameworks, or in ways that would require setting up new frameworks (or other types of procurement vehicle, such as dynamic purchasing systems). The use of GPCs to buy goods off Amazon is the clearest example (see pp 24-25), as there is nothing in the functioning of Amazon that could not be replicated through pre-procured frameworks supported by electronic catalogues. In that regard, GPC data should be used to establish the (administrative) efficiency of creating (new) frameworks and to inform product (and service) selection for inclusion therein. There should also be a clear prohibition of using GPCs outside existing frameworks unless better value for money for identical products can be documented, in which case this should also be reported to the entity running the relevant framework (presumably, the Crown Commercial Service) for review.

Conclusion

In addition to discussions about the type and level of expenditure that (high-raking) public officials should be authorised to incur as a political and policy matter, there is clearly a need and opportunity to engage in serious discussions on the tightening of the regulation of GPCs in the UK, and these should be coordinated with the passage of the Procurement Bill through the House of Commons. I have identified the following areas for action:

  • Suppression of the value threshold triggering transparency of specific transactions, so that all transactions are subjected to reporting.

  • Coordination of the single transaction threshold with that triggering procurement obligations for central government (which is to also apply to local and other contracting authorities).

  • Coordination of the maximum monthly spend limit with the threshold for international advertising of contract opportunities, so that no public official can spend more than the relevant amount in a given category of goods or services per month.

  • Launch of a new investigation and report by NAO on the existing mechanisms of pre- and post-expenditure control.

  • Creation of a sophisticated data-management architecture that can handle of multi-tiered access to different types of information at different times, by different stakeholders and under different conditions. This needs to be in parallel or jointly with proposals under the Procurement Bill.

  • There should also be a clear prohibition of using GPCs outside existing frameworks unless better value for money for identical products can be documented. GPC data should be used to inform the creation and management of procurement frameworks and other commercial vehicles.

Regulating public and private interactions in public sector digitalisation through procurement

As discussed in previous entries in this blog (see here, here, here, here or here), public procurement is progressively being erected as the gatekeeper of the public interest in the process of digital technology adoption by the public sector, and thus positioned as digital technology regulator—especially in the EU and UK context.

In this gatekeeping role, procurement is expected to ensure that the public sector only acquires and adopts trustworthy technologies, and that (private) technology providers adhere to adequate technical, legal, and ethical standards to ensure that this is the case. Procurement is also expected to operate as a lever for the propagation of (soft) regulatory tools, such as independently set technical standards or codes of conduct, to promote their adoption and harness market dynamics to generate effects beyond the public sector (ie market-shaping). Even further, where such standards are not readily available or independently set, the procurement function is expected to formulate specific (contractual) requirements to ensure compliance with the overarching regulatory goals identified at higher levels of policymaking. The procurement function is thus expected to leverage the design of public tenders and public contracts as tools of digital technology regulation to plug the regulatory gap resulting from the absence of binding (legal) requirements. This is a tall order.

Analysing this gatekeeping role and whether procurement can adequately perform it is the focus of the last part of my current research project. In this latest draft book chapter, I focus on an analysis of the procurement function as a regulatory actor. The following chapter will focus on an analysis of procurement rules on the design of tender procedures and some elements of contractual design as regulatory tools. Combined, the analyses will shed light on the unsuitability of procurement to carry out this gatekeeping role in the absence of minimum mandatory requirements and external oversight, which will also be explored in detail in later chapters. This draft book chapter is giving me a bit of a hard time and some of the ideas there are still slightly tentative, so I would more than ever welcome any and all feedback.

In ‘Regulating public and private interactions in public sector digitalisation through procurement: the clash between agency and gatekeeping logics’, my main argument is that the proposals to leverage procurement to regulate public sector digitalisation, which seek to use public sector market power and its gatekeeping role to enforce standards of technological regulation by embedding them in public contracts, are bound to generate significant dysfunction due to a break in regulatory logic. That regulatory logic results from an analysis of the procurement function from an agency theory and a gatekeeping theory perspective, which in my view evidence the impossibility for procurement to carry out conflicting roles. To support this claim, I explore: 1) the position of the procurement function amongst the public and private actors involved in public sector digitalisation; 2) the governance implications of the procurement function’s institutional embeddedness; and 3) the likely (in)effectiveness of public contracts in disciplining private and public behaviour, as well as behaviour that is mutually influenced or coproduced by public and private actors during the execution of public contracts.

My analysis finds that, in the regulation of public-private interactions, the regulatory logic underpinning procurement is premised on the existence of a vertical relationship between the public buyer and (potential) technology providers and an expectation of superiority of the public buyer, which is thus (expected to be) able to dictate the terms of the market interaction (through tender requirements), to operate as gatekeeper (eg by excluding potential providers that fall short of pre-specified standards), and to dictate the terms of the future contract (eg through contract performance clauses with a regulatory component). This regulatory logic hits obvious limitations when the public buyer faces potential providers with market power, an insufficient offer of (regulated) goods and services, or significant information asymmetries, which result in a potential ‘weak public buyer’ problem. Such problem has generally been tried to be addressed through procurement centralisation and upskilling of the (centralised) procurement workforce, but those measures create additional governance challenges (especially centralisation) and are unlikely to completely re-establish the balance of power required for the effective regulation by contract of public sector digitalisation, as far as the provider side is concerned.

Parking the ‘weak public buyer’ problem, my analysis then focuses on the regulation of public-public interactions between the adopting public sector entity and the procurement function. I separate them for the purposes of the analysis, to point out that at theoretical level, there is a tension between the expectations of agency and gatekeeping theories in this context. While both of them conceptualise the relationship as vertical, they operate on an opposite understanding of who holds a predominant position. Under agency theory, the public buyer is the agent and thus subject to the instructions of the public entity that will ultimately adopt the digital technology. Conversely, under gatekeeping theory, the public buyer is the (independent) guarantor of a set of goals or attributes in public sector digitalisation projects and is thus tasked with ensuring compliance therewith. This would place the public buyer in a position of (functional) superiority, in that it would (be expected to) be able to dictate (some of) the terms of the technological adoption. This conflict in regulatory logics creates a structural conflict of interest for the procurement function as both agent and gatekeeper.

The analysis then focuses on how the institutional embeddedness of procurement exacerbates this problem. Where the procurement function is embedded in the same administrative unit or entity that is seeking to adopt the technology, it is subjected to hierarchical governance and thus lacks the independence required to carry out the gatekeeping role. Similarly, where the procurement function is separate (eg in the case of centralised or collaborative procurement), in the absence of mandatory requirements (eg to use the centralised procurement vehicle), the adopting public entity retains discretion whether to subject itself to the (gatekeeper) procurement function or to carry out its own procurement. Moreover, even when it uses centralised procurement vehicles, it tends to retain discretion (eg on the terms of mini-competitions or for the negotiation of some contractual clauses), which also erodes the position of the procurement function to effectively carry out its gatekeeping role.

On the whole, the procurement function is not in a good position to discipline the behaviour of the adopting public entity and this creates another major obstacle to the effectiveness of the proposed approach to the regulation by contract of public sector digitalisation. This is exacerbated by the fact that the adopting public entity will be the principal of the regulatory contract with the (chosen) technology provider, which means that the contractual mechanisms designed to enforce regulatory goals will be left to interpretation and enforcement by those actors whose behaviour it seeks to govern.

In such decentred interactions, procurement lacks any meaningful means to challenge deviations from the contract that are in the mutual interest of both the adopting entity and the technology provider. The emerging approach to regulation by contract cannot properly function where the adopting public entity is not entirely committed to maximising the goals of digital regulation that are meant to be enforced by contract, and where the public contractor has a concurring interest in deviating from those goals by reducing the level of demand of the relevant contractual clauses. In the setting of digital technology regulation, this seems a likely common case, especially if we consider that the main regulatory goals (eg explainability, trustworthiness) are open-ended and thus the question is not whether the goals in themselves are embraced in abstracto by the adopting entity and the technology provider, but the extent to which effective (and costly or limiting) measures are put in place to maximise the realisation of such goals. In this context, (relational) contracts seem inadequate to prevent behaviour (eg shirking) that is the mutual interest of the contractual parties.

This generates what I label as a ‘two-sided gatekeeping’ challenge. This challenge encapsulates the difficulties for the procurement function to effectively influence regulatory outcomes where it needs to discipline both the behaviour of technology providers and adopting entities, and where contract implementation depends on the decentred interaction of those two agents with the procurement function as a (toothless) bystander.

Overall, then, the analysis shows that agency and gatekeeping theory point towards a disfunction in the leveraging of procurement to regulate public sector digitalisation by contract. There are two main points of tension or rupture with the regulatory logic. First, the regulatory approach cannot effectively operate in the absence of a clear set of mandatory requirements to bind the discretion of the procurement function during the tendering and contract formation phase, as well as the discretion of the adopting public entity during contract implementation phase, and which are also enforceable on the technology provider regardless of the terms of the contract. Second, the regulatory approach cannot effectively operate in the absence of an independent actor capable of enforcing those standards and monitoring continuous compliance during the lifecycle of technological adoption and use by the public sector entity. As things stand, the procurement function is affected by structural and irresolvable conflicts between its overlaid roles. Moreover, even if the procurement function was not caught by the conflicting logics and requirements of agency and gatekeeping (eg as a result of the adoption of the mandatory requirements mentioned above), it would still not be in an adequate position to monitor and discipline the behaviour of the adopting public entity—and, relatedly, of the technology provider—after the conclusion of the procurement phase.

The regulatory analysis thus points to the need to discharge the procurement function from its newest gatekeeping role, to realign it with agency theory as appropriate. This would require both the enactment of mandatory requirements and the subjection to external oversight of the process of technological adoption by the public sector. This same conclusion will be further supported by an analysis of the limitations of procurement law to effectively operate as a regulatory tool, which will be the focus of the next chapter in the book.

UK REGULATION AFTER BREXIT REVISITED -- PUBLIC PROCUREMENT

Negotiating the Future’ and ‘UK in a Changing Europe’ have published a second edition of their interesting report on ‘UK Regulation after Brexit - Revisited’. I had contributed a procurement chapter to the first edition (which has recently been cited in this interesting report for the European Committee of the Regions on the impact on regions and cities of the new trade and economic relations between EU-UK). So I was invited to update the analysis, paying special attention to the (slow) progress of reform of the UK procurement rulebook with the Procurement Bill.

The procurement analysis is below, but I would recommend reading the report in full, as it gives a rather comprehensive picture of how regulation is moving in the UK. For more targeted analysis on regulatory divergence with the EU, this other UK in a Changing Europe ‘Divergence Tracker’ (v5.0) will be of interest.

Public procurement

Public procurement regulation is the set of rules and policies that controls the award of public contracts for works, supplies, and services. Its main goal is to ensure probity and value for money in the spending of public funds – to prevent corruption, collusion, and wastage of taxpayers’ money. It does so by establishing procedural requirements leading to the award of a public contract, and by constraining discretion through requirements of equal treatment, competition, and proportionality. From a trade perspective, procurement law prevents favouritism and protectionism of domestic businesses by facilitating international competition.

In the UK, procurement rules have long been considered an excessive encumbrance on the discretion and flexibility of the public sector, as well as on its ability to deploy ambitious policies with social value to buy British products made by British workers. The EU origin of UK domestic rules, which ‘copied out’ EU Directives before Brexit, has long been blamed for perceived rigidity and constraint in the allocation of public contracts, even though a ‘WTO regime’ would look very similar.

Capitalising on that perception during the Brexit process, public procurement was ear-marked for reform. Boris Johnson promised a ‘bonfire of procurement red tape to give small firms a bigger slice of Government contracts’. The Johnson Government proposed to significantly rewrite and simplify the procurement rulebook, and to adopt an ambitious ‘Buy British’ policy, which would reserve some public contracts to British firms. However, although one of the flagship areas for regulatory reform, not much has changed in practical terms. Reforms are perhaps on the horizon in 2023 or 2024, but the extent to which they will result in material divergence from the pre-Brexit EU regulatory baseline remains to be seen.

Post-Brexit changes so far, plus ça change…

To avoid a regulatory cliff edge and speed up its realignment under international trade law, the UK sought independent membership of the World Trade Organisation Government Procurement Agreement (GPA) from 1 January 2021 on terms that replicate and give continuity to its previously indirect membership as an EU Member State. The UK’s current individual obligations under the GPA are the same as before Brexit. Moreover, to maintain market access, the EU-UK Trade and Cooperation Agreement (TCA) replicates obligations under EU law that go beyond the GPA in substantive and procedural elements (‘GPA+’), with only the exception of some contracts for healthcare services. The Free Trade Agreements (FTAs) with Australia and New Zealand, and the envisioned accession of the UK to the Comprehensive and Progressive Agreement for Trans-Pacific Partnership (CPTPP) foresee further GPA+ market access obligations and increasingly complicated constraints related to trade.

These commitments prevent the adoption of an expansive ‘Buy British’ policy and could in fact restrict it in some industries, although healthcare is explicitly excluded from procurement-related trade negotiations. Despite misleading claims to the contrary in UK governments reports, such as the January 2022 Benefits of Brexit report, which gives the impression that Brexit ‘enabled goods and services contracts below £138,760 (central government), £213,477 (sub-central authorities) and £5.3 million (construction throughout the public sector) to be reserved for UK suppliers’ (art 8), official procurement guidance makes clear that the situation remains unchanged. Contracts above the values quoted above – those covered by the GPA, the TCA, and Free Trade Agreements – remain open to international competition. In other words, the government has not achieved its stated Brexit aspiration of reserving ‘a bigger slice’ of procurement to domestic businesses.

A similar picture emerges in relation to procedural requirements under procurement law. While the UK Government declared that its aim was to ‘rewrite the rulebook’ (as discussed below), the pre-Brexit ‘copy out’ of EU procurement rules remains in effect as retained EU law. Brexit required some marginal technical adjustments, such as a change in the digital platform where contract opportunities are advertised and where high value contract opportunities are published in the Find a Tender portal rather than the EU’s official journal, or the substitution of the European Single Procurement Document (ESPD) with a near-identical Single Procurement Document (SPD). The main practical change following Brexit is the UK being disconnected from the e-Certis database. The lack of direct access to documentary evidence makes it more difficult and costly for businesses and public sector entities to complete pre-award checks, especially in cases of cross-border EU-UK tendering. However, TCA provisions seek to minimise these documentary requirements (Art 280) and could mitigate the practical implications of the UK no longer being part of the e-Certis system.

With Brexit, the Minister for the Cabinet Office assumed the powers and functions relating to compliance with procurement rules. Even if the bar was already quite low before Brexit, since virtually no infringement procedures had been opened against the UK for procurement breaches, this change is likely to result in a weakening of enforcement due to the lack of separation between Cabinet Office and other central government departments. The shortcomings of current oversight mechanisms are reflected in the proposed reforms discussed below, which include a proposal to create a dedicated Procurement Review Unit.

Future change

The government has been promoting the reform of the UK’s procurement rulebook. Its key elements were included in the 2020 Green Paper Transforming Public Procurement. The aim was ‘to speed up and simplify [UK] procurement processes, place value for money at their heart, and unleash opportunities for small businesses, charities and social enterprises to innovate in public service delivery’, through greater procedural flexibility, commercial discretion, data transparency, centralisation of a debarment mechanism, and regulatory space for non-economic considerations. The Green Paper envisaged the creation of a new Procurement Review Unit with oversight powers, as well as measures to facilitate the judicial review of procurement decisions. Despite the rhetoric, the proposals did not mark a significant departure from the current rules. They were ‘EU law+’, at best. However, a deregulatory approach that introduces more discretion and less procedural limitations carries potential for significantly complicating procurement practice by reducing procedural standardisation and increasing tendering costs.

The 2021’s government response to the consultation mostly confirmed the approach in the Green Paper and, on 11 May 2022, the Procurement Bill was introduced in the House of Lords, the day after the Queen’s Speech. The Procurement Bill is hardly an exemplar of legislative drafting and it was soon clear that it would need very significant amending. As of 1 September 2022, the Bill had reached its committee stage in the Lords. Five hundred amendments have been put forward with over three hundred of those originating from the government itself. The amendments affect the ‘transformative’ elements of the Bill, and sometimes there are competing amendments over the same clause that would result in different outcomes. It is difficult to gauge whether the government’s proposals will result in a legislative text that materially deviates from the current rules. It is also unclear to what extent the new Procurement Review Unit will have effective oversight powers, or enforcement powers.

The Procurement Bill, moreover, contains only the bare bones of a future regime. Secondary legislation and volumes of statutory guidance will be adopted and developed once the final legislation is in place. Given the uncertainty, the government has committed to provide at least six months’ notice of the new system. It is therefore unlikely that the new rules will be in place before mid-2023. The roll-out of the new rules will require a major training exercise, but most of the government’s training programme is directed towards the public sector. Business can expect to shoulder significant costs associated with the introduction of the new rules.

These legislative changes will not apply UK-wide. Scotland has decided to keep its own separate (EU-derived) procurement rules in place. Divergence between the rules in Scotland and those that apply in the rest of the UK is governed by the 2022 revised Common Framework for Public Procurement. The Common Framework allows for policy divergence, and has already resulted in different national procurement strategies for England, Wales and Scotland, as well as keeping in place a pre-existing policy for Northern Ireland. It is too early to judge, but different policy approaches may in the medium term fragment the UK internal market for public contracts, especially non-central government procurement.

Conclusion

The process of UK procurement reform may be the ‘perfect Brexit story’. Perceived pre-Brexit problems and dissatisfaction were largely a result of long-lasting underinvestment in public sector capacity and training and constraints that mostly derive from international treaties rather than EU law. As an EU member state, the UK could have decided to transpose EU rules other than copying them, thereby building a more comprehensive set of procurement rules that could address some of the shortcomings in the EU framework. It could have funded a better public sector training programme, implemented open procurement data standards and developed analytical dashboards, or centralised debarment decisions. It decided not to opt for any of these measures but blamed the EU for the issues that arose from that decision.

When Brexit rhetoric had to be translated into legal change, reality proved rather stubborn. International trade commitments were simply rolled over, thereby reducing any prospect of a ‘Buy British’ policy. Moreover, the ongoing reform of procurement law is likely to end up introducing more complexity, while only deviating marginally from EU standards in practice. Despite all the effort expended and resource invested, a Brexit dividend in public procurement remains elusive.

Flexibility, discretion and corruption in procurement: an unavoidable trade-off undermining digital oversight?

Magic; Stage Illusions and Scientific Diversions, Including Trick Photography (1897), written by Albert Allis Hopkins and Henry Ridgely Evan.

As the dust settles in the process of reform of UK public procurement rules, and while we await for draft legislation to be published (some time this year?), there is now a chance to further reflect on the likely effects of the deregulatory, flexibility- and discretion-based approach to be embedded in the new UK procurement system.

An issue that may not have been sufficiently highlighted, but which should be of concern, is the way in which increased flexibility and discretion will unavoidably carry higher corruption risks and reduce the effectiveness of potential anti-corruption tools, in particular those based on the implementation of digital technologies for procurement oversight [see A Sanchez-Graells, ‘Procurement Corruption and Artificial Intelligence: Between the Potential of Enabling Data Architectures and the Constraints of Due Process Requirements’ in S Williams-Elegbe & J Tillipman (eds), Routledge Handbook of Public Procurement Corruption (Routledge, forthcoming)].

This is an inescapable issue, for there is an unavoidable trade-off between flexibility, discretion and corruption (in procurement, and more generally). And this does not bode well for the future of UK procurement integrity if the experience during the pandemic is a good predictor.

The trade-off between flexibility, discretion and corruption underpins many features of procurement regulation, such as the traditional distrust of procedures involving negotiations or direct awards, which may however stifle procurement innovation and limit value for money [see eg F Decarolis et al, ‘Rules, Discretion, and Corruption in Procurement: Evidence from Italian Government Contracting’ (2021) NBER Working Paper 28209].

The trade-off also underpins many of the anti-corruption tools (eg red flags) that use discretionary elements in procurement practice as a potential proxy for corruption risk [see eg M Fazekas, L Cingolani and B Tóth, ‘Innovations in Objectively Measuring Corruption in Public Procurement’ in H K Anheier, M Haber and M A Kayser (eds) Governance Indicators: Approaches, Progress, Promise (OUP 2018) 154-180; or M Fazekas, S Nishchal and T Søreide, ‘Public procurement under and after emergencies’ in O Bandiera, E Bosio and G Spagnolo (eds), Procurement in Focus – Rules, Discretion, and Emergencies (CEPR Press 2022) 33-42].

Moreover, economists and political scientists have clearly stressed that one way of trying to strike an adequate balance between the exercise of discretion and corruption risks, without disproportionately deterring the exercise of judgement or fostering laziness or incompetence in procurement administration, is to increase oversight and monitoring, especially through auditing mechanisms based on open data (see eg Procurement in a crisis: how to mitigate the risk of corruption, collusion, abuse and incompetence).

The difficulty here is that the trade-off is inescapable and the more dimensions on which there is flexibility and discretion in a procurement system, the more difficult it will be to establish a ‘normalcy benchmark’ or ‘integrity benchmark’ from which deviations can trigger close inspection. Taking into account that there is a clear trend towards seeking to automate integrity checks on the basis of big data and machine learning techniques, this is a particularly crucial issue. In my view, there are two main sources of difficulties and limitations.

First, that discretion is impossible to code for [see S Bratus and A Shubina, Computerization, Discretion, Freedom (2015)]. This both means that discretionary decisions cannot be automated, and that it is impossible to embed compliance mechanisms (eg through the definition of clear pathways based on business process modelling within an e-procurement system, or even in blockchain and smart contract approaches: Neural blockchain technology for a new anticorruption token: towards a novel governance model) where there is the possibility of a ‘discretion override’.

The more points along the procurement process where discretion can be exercised (eg choice of procedure, design of procedure, award criteria including weakening of link to subject matter of the contract and inclusion of non(easily)measurable criteria eg on social value, displacement of advantage analysis beyond sphere of influence of contracting authority, etc) the more this difficulty matters.

Second, the more deviations there are between the new rulebook and the older one, the lower the value of existing (big) data (if any is available or useable) and of any indicators of corruption risk, as the regulatory confines of the exercise of discretion will not only have shifted, but perhaps even lead to a displacement of corruption-related exercise of discretion. For example, focusing on the choice of procedure, data on the extent to which direct awards could be a proxy for corruption may be useless in a new context where that type of corruption can morph into ‘custom-made’ design of a competitive flexible procedure—which will be both much more difficult to spot, analyse and prove.

Moreover, given the inherent fluidity of that procedure (even if there is to be a template, which is however not meant to be uncritically implemented), it will take time to build up enough data to be able to single out specific characteristics of the procedure (eg carrying out negotiations with different bidders in different ways, such as sequentially or in parallel, with or without time limits, the inclusion of any specific award criterion, etc) that can be indicative of corruption risk reliably. And that intelligence may not be forthcoming if, as feared, the level of complexity that comes with the exercise of discretion deters most contracting authorities from exercising it, which would mean that only a small number of complex procedures would be carried out every year, potentially hindering the accumulation of data capable of supporting big data analysis (or even meaningful econometrical treatment).

Overall, then, the issue I would highlight again is that there is an unavoidable trade-off between increasing flexibility and discretion, and corruption risk. And this trade-off will jeopardise automation and data-based approaches to procurement monitoring and oversight. This will be particularly relevant in the context of the design and implementation of the tools at the disposal of the proposed Procurement Review Unit (PRU). The Response to the public consultation on the Transforming Public Procurement green paper emphasised that

‘… the PRU’s main focus will be on addressing systemic or institutional breaches of the procurement regulations (i.e. breaches common across contracting authorities or regularly being made by a particular contracting authority). To deliver this service, it will primarily act on the basis of referrals from other government departments or data available from the new digital platform and will have the power to make formal recommendations aimed at addressing these unlawful breaches’ (para [48]).

Given the issues raised above, and in particular the difficulty or impossibility of automating the analysis of such data, as well as the limited indicative value and/or difficulty of creating reliable red flags in a context of heightened flexibility and discretion, quite how effective this will be is difficult to tell.

Moreover, given the floating uncertainty on what will be identified as suspicious of corruption (or legal infringement), it is also possible that the PRU (initially) operates on the basis of indicators or thresholds arbitrarily determined (much like the European Commission has traditionally arbitrarily set thresholds to consider procurement practices problematic under the Single Market Scorecard; see eg here). This could have a signalling effect that could influence decision-making at contracting authority level (eg to avoid triggering those red flags) in a way that pre-empts, limits or distorts the exercise of discretion—or that further displaces corruption-related exercise of discretion to areas not caught by the arbitrary indicators or thresholds, thus making it more difficult to detect.

Therefore, these issues can be particularly relevant in establishing both whether the balance between discretion and corruption risk is right under the new rulebook’s regulatory architecture and approach, as well as whether there are non-statutory determinants of the (lack of) exercise of discretion, other than the complexity and potential litigation and challenge risk already stressed in earlier analysis and reflections on the green paper.

Another ‘interesting’ area of development of UK procurement law and practice post-Brexit when/if it materialises.

New paper on competition and procurement regulation -- in memory of Professor Steen Treumer

Image credits: Steve Johnson.

Last year brought the saddest news with the passing of Professor Steen Treumer after a long illness. Steen was a procurement colossus and a fantastic academic. I was extremely lucky to count him amongst my mentors and champions, especially at the very early stages of my research and academic career, before he had to take a step back to focus on his health. I am particularly grateful to him for having opened the door of the European Procurement Law Group to me. And for his generosity in providing feedback, job and promotion references, and thoughtful and clever advice without ever asking for or expecting anything in return.

It is nigh impossible to do justice to the intellectual contribution Steen made to the procurement field and the influence his approach had on the research of others such as myself. It is now a humbling honour to have been invited to contribute to an edited collection in his memory (a Mindeskrift). If he could read my contribution, I am not sure Steen would agree with what I say in the paper, but we would certainly have an interesting and stimulating discussion on the basis of the sharp comments (even some devil’s advocate ones) he would surely come up with. I hope you will find the contribution worth discussing too.

Probably unsurprisingly, the paper is entitled ‘Competition and procurement regulation: a goal, a principle, a requirement, or all of the above?’ and its abstract is below. In the paper, I use the background of recent developments in UK and EU case law, as well as the UK’s procurement rulebook reform process, to reframe the issue of the role of competition in procurement regulation. While I do not provide any insights I had not already developed in earlier writing, I bring some scattered parts of my scholarship together and hopefully clarify a few things along the way. The paper may be particularly interesting to those looking for an entry point to the discussion on the role of competition in public procurement, but hopefully there is also something for those already well versed on the topic. As always, comments most welcome: a.sanchez-graells@bristol.ac.uk.

In this contribution, I reflect on the role of competition in public procurement regulation and, more specifically, on whether competition should be treated as a regulatory goal, as a general principle of public procurement law, as a specific (implicit or explicit) requirement in discrete legal provisions, or all of the above. This is an issue I had the pleasure and honour of discussing with Professor Steen Treumer back in 2009, when I was a PhD student visiting the Copenhagen Business School. While Steen never revealed to me what he really thought, his probing questions continue to help me think of this issue, which remains at the core of my research efforts. This contribution shows that the role of competition keeps cropping up in procurement regulation and litigation, as evidenced in recent UK developments. This is thus an evergreen research topic, which were Steen’s favourites.

The full citation is: Sanchez-Graells, Albert, ‘Competition and procurement regulation: a goal, a principle, a requirement, or all of the above?’, to be published in Steen Treumer’s Mindeskrift edited by Carina Risvig Hamer, Erik Bertelsen, Marta Andhov, and Roberto Caranta (Ex Tuto Publishing, forthcoming 2022). Available at SSRN: https://ssrn.com/abstract=4012022.

Doing procurement differently after Brexit? [update]

The UK in a Changing Europe (UKICE) has published a new report: ‘Doing things differently? Policy after Brexit‘. The report provides an update on last year’s ‘UK regulation after Brexit', as well as additional analysis.

‘Doing things differently? Policy after Brexit’ brings together a number experts in their respective fields to investigate how policy and policymaking have changed in a range of sectors. UKICE asked them to consider how changes so far compare to what was promised before Brexit, and to analyse what changes lie ahead and what their impact might be.

I contributed a section on public procurement. For more details and broader developments in UK procurement regulation, you can also see my recent country report for EPPPL.

What changes were promised after Brexit?

Public procurement regulation is a set of rules and policies controlling the award of public contracts for works, supplies, and services. Its main goal is to ensure probity and value for money in the spending of public funds, to prevent corruption, collusion, and wastage of taxpayers’ money. As pandemic-related procurement has shown, the absence of procurement rules (or their disapplication due to an emergency), all too often leads to the improper award of public contracts. Nonetheless, the benefits of constraining discretion in the award of public contracts are easily forgotten in ‘normal times’, and procurement regulation is permanently challenged for creating an administrative burden on both the public sector and on companies tendering for public contracts, and for stifling innovation.

Procurement has long been heavily influenced by international and regional agreements, which constrain domestic choices to facilitate cross-border tendering for public contracts. Before Brexit, the UK was directly bound by the procurement rules of the European Union (EU), and indirectly by those of the World Trade Organisation’s Government Procurement Agreement (GPA), to which EU rules are aligned. As a result, UK regulatory autonomy was limited to the spaces left by general EU rules requiring domestic transposition. The UK decided not to exercise that limited discretion and consistently took a copy-out approach to the transposition of EU rules, so pre-Brexit UK procurement regulation was virtually identical to the EU’s.

During the Brexit process, public procurement was ear-marked for reform. Boris Johnson promised a ‘bonfire of procurement red tape to give small firms a bigger slice of Government contracts’ and his Government proposed to significantly rewrite the procurement rulebook, and to adopt an ambitious ‘Buy British’ policy to reserve some public contracts to British firms.

What has changed so far?

Despite those promises, the UK Government has made big efforts to replicate international and regional procurement agreements post-Brexit, which means it will continue to be hard to introduce an effective ‘Buy British’ policy. The UK gained GPA membership in its own right on 1 January 2021. This now directly constrains domestic choices on procurement regulation. The EU-UK Trade and Cooperation Agreement (TCA) also includes a chapter on public procurement that leaves mutual market access commitments virtually unchanged.

The UK Government was slow to understand (or at least clearly communicate) the implications of this continuity in the trade-related aspects of procurement regulation. On 15 December 2020, the Cabinet Office issued a Procurement Policy Note (PPN) on ‘Reserving below threshold procurements’ that formulated the new ‘Buy British’ policy in terms of reserving contracts by supplier location (either UK-wide, or by county) and/or reserving them for small and medium sized enterprises (SMEs) or voluntary, community and social enterprises (VCSEs). Aggressive implementation could have contravened international agreements to which the UK had signed up. This led to the publication on 19 February 2021 of a new PPN on ‘The WTO GPA and the UK-EU TCA,’ stressing that the pre-Brexit limits on a ‘Buy British’ policy remain in place and virtually unchanged post-Brexit.

On 15 December 2020, the UK Government published the green paper ‘Transforming Public Procurement’ to consult on planned legislative changes to the procurement rulebook. The original timeline envisaged the introduction of a Procurement Bill in Parliament after summer 2021. However, the volume of responses to the public consultation (over 600) and the complex issues they raised, as well as the intrinsic difficulty in seeking to significantly change procurement law in a manner that is compliant with international obligations led the Cabinet Office to adjust the timeline. The 6 December 2021 Government response to the public consultation clarified that the new regime will not come into force until 2023 at the earliest.

So far, then, the Brexit-related changes have been modest. There have been some policy developments, such as the adoption of a National Procurement Policy Statement seeking to embed government goals such as growth and jobs and climate change in procurement decision-making; a push for a fresh approach to assessing social value in the award of government contracts; new requirements for firms applying for major contracts to have Carbon Reduction plans; and to also require those firms to have systems in place that ensure prompt, fair and effective payments to their supply chains. None of these will reduce procurement red tape and most, if not all, would have been possible pre-Brexit.

What are the possibilities for the future?

Given the commitments in the GPA and TCA, there is virtually no scope for a Buy British policy. The UK could be more aggressive in the exclusion of tenderers from non-GPA jurisdictions such as China, India or Brazil (something the EU is increasingly doing) as a practical way of seeking to boost contract awards to UK companies.

By contrast, the process of reform of the UK’s procurement rulebook is likely to result in a new set of streamlined regulations, as well as a voluminous body of guidance. Despite the Government’s prioritisation of simplification as a primary goal of legislative reform, the extent to which procurement can be significantly deregulated is unclear, as a result both of international commitments and, more importantly, the need to create a legislative framework fit for purpose that does not overwhelm the public sector in its complexity.

There is an opportunity for the Procurement Bill to make some progress on the modernisation and digitalisation of procurement systems, which has been slow in the UK despite it being a shared strategic goal with the EU. It is likely that the new rules will bring a clearer focus on open procurement data, which could enable a change of approach to the practice and management of procurement and offer some benefits from a red tape perspective. However, the green paper was criticised, among other things, for a lack of ambition in the automation of public procurement, so the extent to which tech will be a pillar of procurement ‘transformation’ in the UK remains unclear.

Overall, not much has changed and, rhetoric apart, there is limited scope for further change.

The 'NHS Food Scanner' app as a springboard to explore the regulation of public sector recommender systems

In England, the Department of Health and Social Care (DHSC) offers an increasingly wide range of public health-related apps. One of the most recently launched is the ‘Food Scanner’, which aims to provide ‘swap suggestions, which means finding healthier choices for your family is easier than ever!’.

This is part of a broader public health effort to tackle, among other issues, child obesity, and is currently supported by a strong media push aimed primarily at parents. As the parent of two young children, this clearly caught my attention.

The background for this public health intervention is clear:

Without realising it, we are all eating too much sugar, saturated fat and salt. Over time this can lead to harmful changes on the inside and increases the risk of serious diseases in the future. Childhood obesity is a growing issue with figures showing that in England, more than 1 in 4 children aged 4-to 5-years-old and more than 1 in 3 children aged 10 and 11-years-old are overweight or obese.

The Be Food Smart campaign empowers families to take control of their diet by making healthier food and drink choices. The free app works by scanning the barcode of products, revealing the total sugar, saturated fat and salt inside and providing hints and tips adults plus fun food detectives activities for kids.

No issues with that. My family and myself could do with a few healthier choices. So I downloaded the app and started playing around.

As I scanned a couple of (unavoidably) branded products from the cupboard, I realised that the swaps were not for generic, alternative, healthier products, but also for branded products (often of a different brand). While this has the practical advantage of specifying the recommended healthier alternative in an ‘actionable’ manner for the consumer, this made my competition lawyer part of the brain uneasy.

The proposed swaps were (necessarily) ranked and limited, with a ‘top 3’ immediately on display, and with a possibility to explore further swaps not too easy to spot (unless you scrolled down to the bottom). The different offered swaps also had a ‘liked’ button with a counter (still in very low numbers, probably because the app is very new), but those ‘likes’ did not seem to establish ranking (or alter it?), as lower ranked items could have higher like counts (in my limited experiment).

I struggled to make sense of how products are chosen and presented. This picked my interest, so I looked at how the swaps ‘work’.

The in-app information explained that:

How do we do this?

We look into 3 aspects of the product that you have scanned:
1) Product name; so we can try and find similar products based on the words used within the name.
2) Ingredients list; so we can try and find similar products based on the ingredients of the product you have scanned.
3) Pack size; finally we look into the size of the product you have scanned so that, if have scanned a 330ml can, we can try and show you another can-sized product rather than a 1 litre bottle.

How are they ordered?

We have a few rules as to what we show within the top 3. We reserve spaces for:
1) The same manufacturer; if you have scanned a particular brand we will do our best to try and find a healthier version of that same brand which qualifies for a good choice badge.
2) The same supermarket; if you have scanned a supermarket product we will again do our best to show you an alternative from the same store.
3) Partner products; there are certain products which team up with Change4life that we will try and show if they match the requirements of the products you have scanned.

I could see that convenience and a certain element of ‘competition neutrality’ were clearly at play, but a few issues bothered me, especially as the interaction between manufacturer/supermarket is not too clear and there is a primary but nebulous element of preferencing that I was not expecting in an app meant to provide product-based information. I could see myself spending the night awake, trying to find out how that ‘partnership’ is structured, what are the conditions for participating, if there are any financial flows to the Department and/or to partner organisations, etc.

I also realised some quirks or errors in the way information is processed and presented by the Food Scanner app, such as the exact same product (in different format) being assigned different ‘red light’ classifications (see the Kellogg’s Corn Flakes example on the side bar). At a guess, it could be that these divergences come from the fact that there is no single source for the relevant information (it would seem that ‘The nutrient data provided in the app is supplied by Brandbank and FoodSwitch’) and that there is not an entity overseeing the process and curating the data as necessary. In fact, DHSC’s terms and conditions for the Food Scanner app (at 6.10) explicitly state that ‘We do not warrant that any such information is true or accurate and we exclude all liability in respect of the accuracy, completeness, fitness for purpose or legality of that information’ . Interesting…

It is also difficult to see how different elements of the red light system (ie sugar vs saturated fat vs salt) are subject to trade-offs as eg, sometimes, a red/green/yellow product is recommended swapping with a yellow/yellow/yellow product. Working out the scoring system behind such recommendations seems difficult, as there will necessarily be a trade off between limiting (very) high levels of one of the elements against recommending products that are ‘not very healthy’ on all counts. There has to be a system behind this — in the end, there has to be an algorithm underpinning the app. But how does it work and what science informs it?

These are all questions I am definitely interested in exploring. However, I called it a night and planned to look for some help to investigate this properly (a small research project is in the making and I have recruited a fantastic research associate — keep an eye on the blog for more details). For now, I can only jot down a few thoughts on things that will be interesting to explore, to which I really have no direct answers.

The Food Scanner is clearly a publicly endorsed (and owned? developed?) recommender system. However, using a moderate research effort, it is very difficult to access useful details on how it works. There is no published algorithmic transparency template (that I could find). The in-app explanations of how the recommender system works raise more questions than they answer.

There is also no commitment by the DHSC to the information provided being ‘true or accurate’, not to mention complete. This displaces the potential liability and all the accountability for the information on display to (a) Brandbank, a commercial entity within the multinational Nielsen conglomerate, and to (b) Foodswitch, a data-technology platform developed by The George Institute for Global Health. The role of these two institutions, in particular concerning the ‘partnership’ between manufacturers and Change4life (now ‘Better Health’ and, effectively, the Office for Health Improvement & Disparities in the DHSC?), is unclear. It is also unclear whether the combination of the datasets operated by both entities is capable of providing a sufficiently comprehensive representation of the products effectively available in England and, in any case, it seems clear to me that there is a high risk (or certainty) that non mass production/consumption ‘healthy products’ are out of the equation. How this relates to broader aspects of competition, but also of public health policy, can only raise questions.

Additionally, all of this raises quite a few issues from the perspective of the trustworthiness that this type of app can command, as well as the broader competition law implications resulting from the operation of the Food Scanner.

And I am sure that more and more questions will come to mind as I spend more time obsessing about it.

Beyond the specificities of the case, it seems to me that the NHS Food Scanner app is a good springboard to explore the regulation of public sector recommender systems more generally — or, rather, some of the risks implicit in the absence of specific regulation and the difficulties in applying standard regulatory mechanisms (and, perhaps, especially competition law) in this context. Hopefully, there will be some interesting research findings to report by the summer. Stay tuned, and keep healthy!

Recent developments in UK procurement regulation -- consolidated overview

I have put together a consolidated review of recent developments in UK procurement regulation, to be included as a country report in a forthcoming issue of the European Procurement & Public Private Partnership Law Review.

It brings together developments discussed in the blog in recent months. including the Post-Brexit rulebook reform, the proposal of special rules for healthcare services commissioning, the procurement chapter in the UK-Australia Free Trade Agreement, and a recent decision in the PPE procurement litigation saga.

In case of interest, it can be downloaded from SSRN: https://ssrn.com/abstract=4016424.

It contains nothing new, though, so assiduous readers may want to skip this one!

Short comments on the proposed regulation on foreign subsidies distorting the internal market, as it relates to procurement

bigstockflag.jpeg

The European Commission is currently consulting on its recent Proposal for a Regulation on foreign subsidies distorting the internal market (COM(2021) 223 final, 5 May 2021). The public consultation will be open until 15 July 2021. I have just submitted my views on chapter four of the proposal, which concerns the rules for the analysis of foreign subsidies distorting tenders for contracts with a value above €250 million. The feedback form only allows for 4,000-character submissions, so here are mine. As always, comments welcome: a.sanchez-graells@bristol.ac.uk.

The proposed Regulation on foreign subsidies distorting the internal market (RFSDIT) is both (1) undesirable and (2) problematic, in particular as it concerns the investigation of foreign subsidies linked to public procurement procedures. The following is limited to chapter 4.

1. Primarily, ch 4 RFSDIT is undesirable because it adds a layer of scrutiny and red tape that will affect high-value tenders submitted by tenderers from jurisdictions that have either signed up to the WTO Government Procurement Agreement, or that have a plurilateral or bilateral trade agreement covering procurement with the EU. Tenderers from other jurisdictions can already be excluded on the basis of the current rules (see Art 25 Dir 2014/24; Art 43 Dir 2014/25), as emphasised in the Commission's 2019 guidance on the participation of third-country bidders and goods in the EU procurement market. First, the (inadvertent) targeting of GPA- or FTA-originated tenders is in itself undesirable on trade policy terms and could erode third countries' bilateral relationships with the EU within the GPA framework, as well as under the relevant FTA (or the UK TCA) even if those already include subsidy-related provisions. Second, it is also undesirable due to the technical shortcomings of the proposal, as below, as there could be a basis for claims of unequal treatment concerning the non-scrutiny of EU-originated tenders that are tainted by illegal State aid. Finally, it is also undesirable because the ex ante nature of ch 4 screening can dissuade economic operators from participating in public tenders even if they think that subsidies they have received could overcome the tests in Arts 3-5 RFSDIT. Recipients of foreign subsidies may rather forgo their chances of being awarded a public contract than trigger an investigation they could avoid under the general motu proprio regime. Such loss of international competition is to the EU public buyers' detriment.

2. Ch 4 RFSDIT is also highly problematic because of its incompatibility with the mechanisms in the EU procurement Directives, as well as the inconsistency of approach with the rest of the chapters in the RFSDIT. First, the proposed rules are incompatible with the trigger for an investigation of the distortive effects of State aid granted to an EU-based tenderer, which derives from the prima facie abnormally low character of its tender (ALT) (see Art 69 Dir 2014/24). EU-generated non-ALT bids are not screened for receipt of (illegal) State aid, even if they can be 'winning tenders' in a given procedure. As above, this can trigger claims of discrimination against non-EU generated tenders. Second, procurement case law pre-empts tenderers from offering commitments related to the tender at hand to the Commission's satisfaction without materially altering their tenders. Such changes would be impermissible under EU procurement law. This is an inescapable limit, which is partly but insufficiently acknowledged in Art 30(1) RFSDIT. This means that any tender where the Commission found an unbalanced distortion of the internal market would lead to the inevitable exclusion of the tender. This is at odds with the appearance of 'correctability' created by Art 30 RFSDIT. This evidences the inadequacy of applying a merger or State aid control logic to the public procurement context. Third, the relative intensity of the foreign subsidy is much lower for procurement than for concentrations under the RFSDIT. Art 18(3) creates a safe harbour of up to 10% of the value of a concentration. Art 27(2) contains no parallel rule. Thus, Art 3(2) offers the only (soft) safe harbour for procurement, which means that subsidies of 2% or less of the tender value would be caught. The reason for this different treatment under RFSDIT opens it to challenge on proportionality grounds. Moreover, it is unclear how a 2% subsidy could create a situation comparable to that of an ALT, which further reinforces concerns of unequal treatment, as above.