Unpacking the logic behind the magic in the use of AI for anticorruption screening (re Pastor Sanz, 2022)

‘Network of public contracts, contracting bodies, and awarded companies in Spain’ in 2020 and 2021; Pastor Sanz (2022: 7).

[Note: please don’t be put off by talk of complex algorithms. The main point is precisely that we need to look past them in this area of digital governance!].

I have read a new working paper on the use of ‘blackbox algorithms’ as anti-corruption screens for public procurement: I Pastor Sanz, ‘A New Approach to Detecting Irregular Behavior in the Network Structure of Public Contracts’. The paper aims to detect corrupt practices by exploiting network relationships among participants in public contracts. The paper implements complex algorithms to support graphical analysis to cluster public contracts with the aim of identifying those at risk of corruption. The approach in the paper would create ‘risk maps’ to eg prioritise the investigation of suspected corrupt awards. Such an approach could be seen to provide a magical* solution to the very complex issue of corruption monitoring in procurement (or more generally). In this post, I unpack what is behind that magic and critically assess whether it follows a sound logic on the workings of corruption (which it really doesn’t).

The paper is technically very complex and I have to admit to not entirely understanding the specific workings of the graphical analysis algorithms. I think most people with an interest in anti-corruption in procurement would also struggle to understand it and, even data scientists (and even the author of the paper) would be unable to fully understand the reasons why any given contract award is flagged as potentially corrupt by the model, or to provide an adequate explanation. In itself, this lack of explainability would be a major obstacle to the deployment of the solution ‘in the real world’ [for discussion, see A Sanchez-Graells, ‘Procurement Corruption and Artificial Intelligence: Between the Potential of Enabling Data Architectures and the Constraints of Due Process Requirements’]. However, perhaps more interestingly, the difficulty in understanding the model creates a significant additional governance risk in itself: intellectual debt.

Intellectual debt as a fast-growing governance risk

Indeed, this type of very complex algorithmic approach creates a significant risk of intellectual debt. As clearly put by Zittrain,

‘Machine learning at its best gives us answers as succinct and impenetrable as those of a Magic 8-Ball – except they appear to be consistently right. When we accept those answers without independently trying to ascertain the theories that might animate them, we accrue intellectual debt’ (J Zittrain, ‘Intellectual Debt. With Great Power Comes Great Ignorance’, 178).

The point here is that, before relying on AI, we need to understand its workings and, more importantly, the underlying theories. In the case of AI for anti-corruption purposes, we should pay particular attention to the way corruption is conceptualised and embedded in the model.

Feeding the machine a corruption logic

In the paper, the model is developed and trained to translate ‘all the public contracts awarded in Spain in the years 2020 and 2021 into a bi-dimensional map with five different groups. These groups summarize the position of a contract in the network and their interactions with their awarded companies and public contracting bodies’ (at 14). Then, the crucial point from the perspective of a corruption logic comes in:

‘To determine the different profiles of the created groups in terms of corruption risk, news about bad practices or corruption scandals in public procurements in the same period (years 2020 and 2021) has been used as a reference. The news collection process has been manual and the 10 most important general information newspapers in Spain in terms of readership have been analyzed. Collected news about irregularities in public procurements identifies suspicions or ongoing investigations about one public contracting body and an awarded company. In these cases, all the contracts granted by the Public Administration to this company have been identified in the sample and flagged as “doubtful” contracts. The rest of the contracts, which means contracts without apparent irregularities or not uncovered yet, have been flagged as “normal” contracts. A total of 765 contracts are categorized as “doubtful”, representing 0.36% of total contracts … contracts belong to only 25 different companies, where only one company collects 508 granted contracts classified as “doubtful”’ (at 14-15, references omitted and emphasis added).

A sound logic?

This reflects a rather cavalier attitude to the absence of reliable corruption data and to difficulties in labelling datasets for that purpose [for discussion, again, see A Sanchez-Graells, ‘Procurement Corruption and Artificial Intelligence: Between the Potential of Enabling Data Architectures and the Constraints of Due Process Requirements’].

Beyond the data issue, this approach also reflects a questionable understanding of the mechanics of corruption. Even without getting into much detail, or trying to be exhaustive, it seems that this is a rather peculiar approach, perhaps rooted in a rather simplistic intuition of how tenderer-led corruption (such as bribery) could work. It seems to me to have some rather obvious shortcomings.

First, it treats companies as either entirely corrupt or not at all corrupt, whereas it seems plausible that corrupt companies will not necessarily always engage in corruption for every contract. Second, it treats the public buyer as a passive agent that ‘suffers’ the corruption and never seeks, or facilitates it. There does not seem to be any consideration to the idea that a public buyer that has been embroiled in a scandal with a given tenderer may also be suspicious of corruption more generally, and worth looking into. Third, in both cases, it treats institutions as monolithic. This is particularly problematic when it comes to treating the ‘public administration’ as a single entity, specially in an institutional context of multi-level territorial governance such as the Spanish one—with eg potentially very different propensities to corruption in different regions and in relation to different (local/not) players. Fourth, the approach is also monolithic in failing to incorporate the fact that there can be corrupt individuals within organisations and that the participation of different decision-makers in different procedures can be relevant. This can be particularly important in big, diversified companies, where a corrupt branch may have no influence on the behaviour of other branches (or even keep its corruption secret from other branches for rather obvious reasons).

If AI had been used to establish this approach to the identification of potentially corrupt procurement awards, the discussion would need to go on to scrutinise how a model was constructed to generate this hypothesis or insight (or the related dataset). However, in the paper, this approach to ‘conceptualising’ or ‘labelling corruption’ is not supported by machine learning at all, but rather depends on the manual analysis and categorisation of news pieces that are unavoidably unreliable in terms of establishing the existence of corruption, as eg the generation of the ‘scandals’ and the related news reporting is itself affected by a myriad of issues. At best, the approach would be suitable to identify the types of contracts or procurement agents most likely to attract corruption allegations and to have those reported in the media. And perhaps not even that. Of course, the labelling of ‘normal’ for contracts not having attracted such media attention is also problematic.

Final thoughts

All of this shows that we need to scrutinise ‘new approaches’ to the algorithmic detection of corruption (or any other function in procurement governance and more generally) rather carefully. This not only relates to the algorithms and the related assumptions of how socio-technical processes work, but also about the broader institutional and information setting in which they are developed (for related discussion, see here). Of course, this is in part a call for more collaboration between ‘technologists’ (such as data scientist or machine learning engineers) and domain experts. But it is also a call for all scholars and policy-makers to engage in critical assessment of logic or assumptions that can be buried in technical analysis or explanations and, as such, difficult to access. Only robust scrutiny of these issues can avoid incurring massive intellectual debt and, perhaps what could be worse, pinning our hopes of improved digital procurement governance on faulty tools.

_____________

* The reference to magic in the title and the introduction relates to Zittrain’s Magic-8 ball metaphor, but also his reference to the earlier observation by Arthur C. Clarke that any sufficiently advanced technology is indistinguishable from magic.

EU law-making in the shadow of the CJEU case law: looking at the “trilogue” black box

This blog first appeared in Elgar Blog on 7 December 2016 as part of our promotion of the recently published book Grith Skovgaard Ølykke and Albert Sanchez-Graells (eds), Reformation or Deformation of the EU Public Procurement Rules (Edward Elgar, 2016). You can now also read Chapter 1: The EU legislative process. An introduction from a political science perspective free on Elgaronline.

After six decades of economic and legal integration, when the European Union (EU) seeks to adopt new rules or to revise existing ones, it hardly ever operates on a clean slate. EU law-making is not only constrained by the political and economic realities of the time, but also by the pre-existing acquis communautaire of rules and general principles as interpreted by the Court of Justice of the European Union (CJEU). By any account, the CJEU has been a great force in the development of EU law, and its case law has pushed the policy-making agenda in rather clear, if controversial ways. EU law-makers thus operate in the shadow of the CJEU case law. This influences law-makers’ starting points and conditions the final solutions to be politically agreed, which will unavoidably be open to scrutiny (and quashing) by the CJEU.

In that space amongst the shadows of the CJEU case law, EU law-makers interact in an increasingly informal manner. They seek ways of flexibilising the legislative process so as to achieve easier and swifter compromises and overcome the criticism of immobilism, sometimes at the cost of renewed criticism of a democratic deficit that the Lisbon Treaty aimed to do away with. Indeed, even if the ordinary legislative procedure is heavily regulated by the EU Treaties and should channel most of the EU’s law-making, informal EU law-making is on the rise. As recently as July 2016, this led the European Ombudsman to call for more transparency of informal negotiations between the European Commission, European Parliament and the Council of the EU, also known as “trilogues” meetings.

It is no exaggeration to say that such “trilogues” are the black box of EU law-making. Under their current operation, it is possible to observe what comes in—legislative proposals are published by the Commission and initial reports by both Council and Parliament are also published—and what comes out of it—in the form of legislation eventually published in the Official Journal of the European Union. But, even after carrying out significant research efforts, it is impossible to crack what happens within the black box and to trace the origin and reasons behind important amendments to proposed legislation, as well as the way in which the final text is drafted.

This creates potential legal uncertainty in terms of the likely interpretation of the texts, which sometimes deviate from previous case law of the CJEU in unexplained ways. It also makes for difficult political assessments of the balance of interests that went into EU law-making and the weight that competing EU, national and group interests carried in shaping the new or revised rules. It can also significantly diminish the technical quality of the final rules, particularly where the trilogues are structured in sequence or address issues in a piece meal fashion, which prevents a final check for internal consistency and eventually leads to difficult systematic interpretation issues.

The case study of the reform of the EU public procurement rules in the period 2011-2014 clearly evidences these issues. The results of our two year research project, now published as Reformation or Deformation of the EU Public Procurement Rules, show both that the entirety of the legislative process was influenced by the CJEU case law, and that some of the most remarkable modifications of the pre-existing EU public procurement rules came about in an unexplained way at trilogue stage. As Dr Grith Skovgaard Ølykke and I stress in our conclusions,

The informal part of the procedure taking place between the 2011 Proposal and the first reading and adoption of the 2014 Directive prevented a repetition of the lengthy process of adopting the 2004 Directive (four years and a full-fledged ordinary legislative procedure, several amended proposals, conciliation and all). However, the early retreat to the trilogue and, hence, informality, significantly reduced transparency compared to that available in the legislative procedure leading to the adoption of the 2004 Directive, where e.g. the amended proposals contain the Commission’s argumentation for accepting proposed amendments or not.

… this still does not clarify the role and influence of the Commission in the post-Lisbon trilogue … Indeed, as stressed by the [European] Parliament itself, ‘given the Commission’s important and active role during Council working party (and even COREPER) discussions, its status as “honest broker” during trilogue negotiations is sometimes questioned in practice’.

A close analysis of the results of our research project helps gain a better understanding of the influence of the CJEU case law in EU law-making, both shaping it and as a benchmark from which policy-makers sometimes try very hard to deviate. However, the results of the research project also stress the limitations of an analysis of the travaux preparatoires and the negotiations throughout the legislative process when important changes and their reasons cannot be observed because they took place in the trilogue black box.

These insights will be interesting in guiding legal interpretive efforts, both in the area of EU public procurement law and more broadly. They will also be high quality and detailed evidence of the need to reform the way trilogues operate, both from a perspective of ensuring high standards of governance through accountability and transparency as stressed by the European Ombudsman, as well as from the perspective of preserving the value of interpretive aids in the context of an ever increasingly complex acquis communautaire.