The challenges of researching digital technology regulation -- some quick thoughts (or a rant)

Keeping up with developments in digital technologies and their regulation is exhausting.

Whenever a technology becomes mainstream (looking at you, ChatGPT, but also looking at blockchain in the rear mirror, and web 2.0 slightly behind… etc) there is a (seemingly) steep learning curve for researchers interested in regulation to climb — sometimes to find little novelty in the regulatory challenges they pose.

It recently seems like those curves are coming closer and closer together, whichever route one takes to exploring tech regulation.

Yet, it is not only that debates and regulatory interventions shift rather quickly, but also that these are issues of such social importance that the (academic) literature around them has exploded. Any automated search will trigger daily alerts to new pieces of scholarship and analysis (of mixed quality and relevance). Not to mention news items, policy reports, etc. Sifting through them beyond a cursory look at the abstracts is a job in itself …

These elements of ‘moving tech targets’ and ‘exponentially available analysis’ make researching these areas rather challenging. And sometimes I wonder if it is even possible to do it (well).

Perhaps I am just a bit overwhelmed.

I am in the process of finalising the explanation of the methodology I have used for my monograph on procurement and digital technologies—as it is clear that I cannot simply get away with stating that it is a ‘technology-centred interdisciplinary legal method’ (which it is, though).

Chatting to colleagues about it (and with the UK’s REF-fuelled obsession for ‘4* rigour’ in the background), the question keeps coming up about how does one make sure to cover the relevant field, how is anyone’s choice of sources not *ahem* capricious or random? In other words, the question keeps coming up: how do you make sure you have not missed anything important? (I’ll try to sleep tonight, cheers!).

I am not sure I have a persuasive, good answer. I am also not sure that ‘comprehensiveness’ is a reasonable expectation of a well done literature review or piece of academic analysis (any more). If it is, barring automated and highly formalised approaches to ‘scoping the field’, I fear we may quickly presume there is no possible method in the madness. But that does not sit right with me. And I also do not think it is a case of throwing the ‘qualitative research’ label as defence, as it means something different (and rigorous).

The challenge of expressing (and implementing) a defensible legal method in the face of such ‘moving tech targets’ and ‘exponentially available analysis’ is not minor.

And, on the other side of the coin, there is a lurking worry that whichever output results from this research will be lost in such ocean of (electronic) academic papers and books —for, if everyone is struggling to sift through the materials and has ever growing (Russian-doll-style) ‘to read’ folders as I do, will eyes ever be set on the research?

Perhaps method does not matter that much after all? (Not comforting, I know!).

Rant over.

An interesting different take on public procurement decision-making (reference to Crowder, 2015)

I have just read an interesting piece of research that sheds different light on public procurement decision-making processes. That short, accessible and interesting piece, [M Crowder, "Public procurement: the role of cognitive heuristics" (2015) 35(2) Public Money & Management 127-34] explores the cognitive heuristics of public procurement processes. As the abstract makes clear

Public procurement processes have been extensively studied, but previous research has not sought to explain public procurement in terms of cognitive heuristics. This paper examines the award of a large public sector contract and outlines how the decisions were made. Heuristics were used throughout the process. Three heuristics—EBA [elimination by aspects], conjunctive, and WADD [weighted additive]—were used in combination to reduce the number of bidders for the contract from a somewhat unmanageable 63 down to four. This paper allows the underlying stages to be viewed from this perspective and therefore it explores procurement in a way that sheds new light on the processes involved.

The paper is easy to follow if one has some experience in public procurement evaluation or, absent that, some knowledge of the rules on exclusion, qualitative selection and short-listing of tenderers [for a summary of the rules under the new Directive 2014/24, see A Sanchez Graells, “Exclusion, Qualitative Selection and Short-listing”, in F Lichère, R Caranta & S Treumer (eds), Modernising Public Procurement. The New Directive, vol. 6 European Procurement Law Series (Copenhagen, DJØF, 2014) 97-129]. As the conclusions stress, the paper shows

that procurement decisions can be explained in terms of cognitive heuristics. The EBA heuristic makes a decision on the basis of a single aspect; the conjunctive heuristic makes a decision on the basis that a number of requirements are all met; and the WADD heuristic makes a decision by weighing up various factors and offsetting the good against the bad. This was reflected in the procurement under study, where the number of bidders under consideration was reduced in precisely this way.

The paper offers a good perspective to complement our understanding of procurement decision-making and provokes some thoughts on how to better regulate these processes in order to avoid weaknesses derived from cognitive biases. 

This is an area that promises to open roads towards interdisciplinary efforts to incorporate the insights of psychology and other sciences into legal research on public procurement. And this seems to me like an area of high research potential, so it may be worth keeping an eye on it!

Pervasive Legal Instrumentalism and Scholarly Herd Behaviour in Law: A Short Reflection on van Gestel & Micklitz (2014)

In their interesting paper "Why Methods Matter in European Legal Scholarship" (2014) European Law Journal 20(3): 292-316, which I read following Steven Vaughan's recommendation on twitter (@lawvaughan), Rob van Gestel and Hans-Wolfgang Micklitz write a compelling criticism against the instrumentalisation of law and legal research (ie the excessively policy-driven approach to legal research that mixes up normative and empirical questions), not least because:

"[it] decreases the attention for methodology, for theory building, and for keeping enough professional distance to one’s object of research. This threatens to result in a creeping process of herd behaviour, in copy pasting the methodology of judicial lawmaking to legal scholarship and in a lack of transparency and methodological justification in scholarly legal publications".

 

Indeed, the part of their paper that I find really interesting (and brave) is the discussion on the risk of herd behaviour in legal research, where they warn about the risks of uncritically focussing legal research on 'hot topics' and the items in the agenda of policymakers/regulators (such as the European Commission) or financing/sponsoring bodies, instead of pursuing an independent ranking of relevant topics with intrinsic research/doctrinal value (pp. 305-307)--and which I remain convinced definitely supports their argument in favour of raising the methodological awareness in European doctrinal legal research.

 

In short, they submit that "the best response to growing heterogeneity of legal sources should be matched with a strengthening of theoretical and methodological components, where possible drawn from the common European heritage in legal theory and spurred by transnational scholarly legal communities" (p. 312). Moreover, they formulate some broad implications of their proposal and launch some open questions, which they intend to focus future debates about (the specifics of legal methods), particularly in view of the Europeanisation of legal education (see some related comments here).

 

I agree with them in that methodological discussions about legal research are becoming of paramount importance and that it is fundamental to base any piece of legal research on a methodology that is not limited to the very narrow confines of (classical) black-letter legal analysis. This is particularly important if one is to embark into any sort of normative recommendation, which requires a benchmark of underlying values and evidence that law cannot provide.

 

Personally, I find the interaction between law and economics particularly important and I have some specific views as to what sort of methodology should be used in the study of European economic law [see A Sánchez Graells, "A Short Note on Methodology: An Eclectic and Heuristic Multi-Disciplinary and Functional Approach to EU Law" (2011)]. Ultimately, I praise and share the words of O Wendell Holmes in "The Path of the Law" (1897) 10 Harvard Law Review 457:

"I look forward to a time when the part played by history in the explanation of dogma shall be very small, and instead of ingenious research we shall spend our energy on a study of the ends sought to be attained and the reasons for desiring them. As a step toward that ideal it seems to me that every lawyer ought to seek an understanding of economics".

This is not to say that economics should drive, control or even dictate the objectives of legal research, nor that efficiency must necessarily be accepted as the ultimate normative value. However, legal research that disregards economic theory and its insights and (willingly or inadvertently) runs against them will have a very limited (if any) value. Moreover, the same is equally applicable to other social sciences and, as van Gestel & Micklitz stress

"this should not imply that we want to turn law students [or law scholars, for that matter] into amateur social or political scientists or economists, but they should at least be able to understand (some of) the language and methods that other (social) sciences apply in order to learn more about the value, validity and reliability of non-doctrinal research methods and techniques" (p. 315).

In any case, beyond the specifics of the methodology employed and the field of (other) science considered more relevant in order to achieve informed and sound outcomes-- which surely needs to be tailored to the specific research question that one attempts to answer--I could not stress enough the importance of having A sound  methodology when one undertakes legal research. In that regard, I fully share and welcome van Gestel & Micklitz's call for further discussion. As they say in their paper, to be continued...