AG proposes extension of Falk Pharma doctrine to framework agreements, for wrong reasons (C-9/17)

In his Opinion of 13 December 2017 in Tirkkonen, C-9/17, EU:C:2017:962 (not available in English), Advocate General Campos Sanchez-Bordona has proposed the application to a framework agreement for the provision to farmers of advisory services funded by the European Agricultural Fund for Rural Development (FEADER) of the Falk Pharma doctrine (ie that the absence of a choice in concreto of the awardee of a contract by the contracting/funding authority excludes the applicability of the EU public procurement rules; see Judgment of 2 June 2016 in Falk Pharma, C-410/14, EU:C:2016:399, and here).

In his view, the fact that individual farmers—and not the competent authority administering the FEADER funds—could choose the specific rural advisor that would provide them the services carved the framework agreement out of the scope of application of the EU (and domestic) public procurement rules—which were therefore not applicable to the tender of the framework agreement in the first place.

In my view, the Tirkkonen Opinion engages in an unjustifiably expansive interpretation of the Falk Pharma Judgment that both ignores some of the basic elements in the functioning of framework agreements, and takes that Judgment’s functionally-erroneous interpretation of the concept of procurement one step too far. If the Tirkkonen Opinion was followed, in combination with Falk Pharma, it would create a significant risk of ineffectiveness of the EU public procurement rules for aggregate and dynamic contracting mechanisms. Therefore, in this post, I present my reasons for a plea to the Court of Justice of the European Union (ECJ) not to follow AG Campos in this occasion, as I think his approach is problematic, both from a positive and a normative perspective.

Tirkkonen – a bad case raising the wrong issues

Why ignore explicit requirements in secondary EU law?

The way the preliminary reference in Tirkkonen reached the ECJ evidences that this is a bad set of circumstances on which to develop the case law on the scope of application of the EU public procurement rules. In the case at hand, the Finnish Agency for Rural Space (Maaseutuvirasto) tendered a framework contract for the provision of advisory services to farmers. Given the (expected) high volume of demand for advisory services, the framework was intended to include as many qualified rural advisors as possible, subject to their passing of an exam to ensure their knowledge and competence (AGO, C-9/17, para 19). Rural advisors admitted to the framework agreement could then be chosen by individual farmers (who should in principle chose the closest advisor, although some exceptions applied), and their services would be remunerated on the basis of hourly rates paid by Maaseutuvirasto, with the beneficiary farmer covering applicable VAT charges (AGO, C-9/17, para 18). It is not explicitly stated in AG Campos' Opinion, but it is worth stressing that the Maaseutuvirasto had set the hourly rate payable to rural advisors, and that the award (ie admission to the framework contract) was to be decided solely on quality (ie competency to provide the service) (see here for details (in Finnish), and thanks to K-M Halonen for help with the translation). The suppression of price competition will be relevant for the assessment below.

The advisory services organised by Maaseutuvirasto were ultimately funded by FEADER for the period 2014-2020 and, under the relevant rules (Reg 1305/2013/EU, Art 15(3), and Impl Reg 808/2014/EU, Art 7), the selection by the Finnish (and all other national) competent authority of the providers of those advisory services was explicitly subjected to European and domestic public procurement rules, which required for the selection to be made: ‘through calls for tenders. The selection procedure shall be governed by public procurement law and shall be open to both public and private bodies’ (Art 15(3) Reg 1305/2013/EU). It was reiterated that the 'calls for tenders referred to in Article 15(3) of Regulation (EU) No 1305/2013 shall follow the applicable Union and national public procurement rules' (Art 7 Impl Reg 808/2014/EU). The Finnish government had no doubt that EU and domestic procurement rules applied, and thus tendered the contract as described above.

Therefore, against this background, a preliminary reference enquiring about the potential non-applicability of the procurement rules to the tender of the framework agreement despite the explicit requirements in special (in the sense of lex specialis) secondary EU legislation is beyond bizarre (see below). However, AG Campos does not see a problem here, and considers that

… that reference to procurement law must be interpreted in the sense that the procedure for the selection of rural advisors must comply with the principles (of non-discrimination, equal treatment and transparency) that govern that sector of the legal order. It does not portray, in my view, a requirement that implies subjection to each and all of the provisions of the EU Directives on public sector procurement (AGO, C-9/17, para 34, own translation from Spanish).

I disagree with this assessment, which is not based on any specific reasons, and which violates the natural reading of Reg 1305/2013/EU and Impl Reg 808/2014/EU. Moreover, it comes to reduce the value of the explicit reference to procurement law in those provisions, and to collapse it into the general principles that are common with general internal market law and, more importantly, the eponymous general principles of EU law—which would be applicable anyway to all activities implementing the relevant instruments of secondary EU law. Therefore, AG Campos’ position not solely deviates from the natural reading of the provisions, but also runs contrary to the functional reasons for the inclusion of the explicit reference to procurement rules (ie to go beyond the general requirements of the always applicable primary EU law). Thus, already on the weakness of the reasons for a deviation from the literal and functional interpretation of those provisions of secondary EU law, I think that the ECJ should largely ignore AG Campos’ Opinion and simply answer the question by confirming the applicability of the EU (and domestic) procurement rules on the basis of the explicit requirements in Reg 1305/2013/EU and Impl Reg 808/2014/EU.

Why not simply state that Finnish procurement law was wrong?

Beyond that first clear-cut solution, which I think highly unlikely the ECJ will adopt, the Court will have to explore the general (as in lex generalis) reasons that still justify the applicability of the EU and (domestic) procurement rules to the case—also contrary to AG Campos’ Opinion. To that end, it is still necessary to understand why the preliminary question was sent to the ECJ—which is explained by a misconstruction of the EU public procurement rules and, in particular, by the harsh consequences of an exceedingly restrictive approach to documentary clarification in the domestic Finnish procurement rules that violates the Manova-Slovensko line of case law (see here, here and here).

In that regard, it is worth noting that the preliminary reference derived from the fact that, in the context of the tender for the framework agreement, Ms Tirkkonen failed to properly complete all required documentation—ie she had failed to indicate whether she accepted or rejected the tender conditions attached to the draft framework agreement (AGO, C-9/17, para 20). She was thus excluded from the framework agreement. Her complaint is fundamentally grounded on the fact that she should have been given the opportunity to clarify whether she accepted the conditions or not prior to her exclusion from the framework agreement.

It is a settled legal fact of the case that, under Finnish law, the omission of that indication of acceptance of the general conditions would only be susceptible if the clarification or correction of the tender was not controlled by public procurement law (which excluded such possibility of clarification), and was rather subjected to general administrative law governing the relationships between citizens and the public administration (AGO, C-9/17, para 3).

Therefore, the harshness of the Finnish procurement rules is behind the interest of the claimant in excluding the tender from the scope of application of domestic procurement rules—which can only be done by seeking a carve-out from the concept of procurement under the EU rules. And, more importantly, the Finnish approach is in contravention of EU law—oddly, as confirmed by AG Campos himself: ‘if Directive 2004/18 was applicable, it would result that the contracting authority would be able to accept, in the context of public procurement, the correction of formal shortcomings that do not imply the submission of a new offer, or substantially altered the terms of the initial offer. On this point, I refer to my Opinion in case MA.T.I. SUD y DUEMMESGR (C-523/16 y C-536/16, EU:C:2017:868)’ (AGO, C-9/17, para 23, fn 7, own translation from Spanish; for discussion of MA.T.I. Sud, see here).

Consequently, the second clear-cut solution for the ECJ is to (i) pick up on the incorrect interpretation of EU public procurement law that underpins the preliminary reference, (ii) reformulate the question and consider that it asked whether the exclusion from the framework agreement due to the formal shortcoming in the documentation and without the possibility to correct it was required or allowed by EU procurement law, (iii) reiterate the Manova-Slovensko case law, and (iv) leave it for the national court to decide on the legality of the exclusion (with a clear hint that exclusion in this case was not justified, due to the logical assumption that would-be rural advisors understood that accepting the general conditions of the draft contract was a requirement for entering into specific contracts, and that confirming such acceptance does not constitute a new offer or substantial modification of the initial offer).

For some reason, however, I am also not optimistic that the ECJ will adopt this second solution and pass on the opportunity to clarify its Falk Pharma case law. Should the ECJ engage with the question and the issues raised by AG Campos, and for the reasons below, I think that the ECJ should provide clarification of Falk Pharma in the opposite direction to that adopted by the Tirkkonen Opinion.

Tirkkonen Opinion ignores how framework agreements work

Once the argument concentrates on the definition of procurement under Article 1(2)(a) of Directive 2004/18/EC, AG Campos summarises the Falk Pharma doctrine as establishing that

… the choice of a tender and, thus, of a successful tenderer, is intrinsically linked to the regulation of public contracts by that directive and, consequently, to the concept of ‘public contract’ within the meaning of Article 1(2) of that directive (AGO, C-9/17, para 37, own translation from Spanish, with reference to Falk Pharma, para 38).

And that

… in the public contracts subjected to Directive 2004/18 a final awardee must exist, which is preferred to the rest of its competitors on the basis of the characteristics of its offer. And this key element is applicable ‘for every contract, framework agreement, and every establishment of a dynamic purchasing system’, for which ‘the contracting authorities are to draw up a written report which is to include the name of the successful tenderer and the reasons why his tender was selected (AGO, C-9/17, para 38, own translation from Spanish, with reference to Falk Pharma, para 39).

This leads AG Campos to argue that, in the framework tendered in Tirkkonen, ‘it is not possible to identify the existence of award criteria of the advisory services contracts, but solely of criteria for the selection of economic operators with capability to offer those services (sic)’ (AGO, C-9/17, para 39, own translation from Spanish and emphasis added). AG Campos continues with a discussion of the distinction between selection and award criteria as per Ambisig (C-601/13, EU:C:2015:204, paras 40 and ff, see here), which I consider irrelevant—for the crucial point is that, in multi-supplier framework agreements (as well as in dynamic purchasing systems, as discussed here), the inclusion in the framework does not (ever) imply the choice of the ‘winner’ of the (call-off) contracts but, conversely, exclusion from the framework does prevent the excluded economic operators from providing the service.

In my view, this is the relevant aspect, for the inclusion in the framework is not simply an identification of the capable or qualified economic operators, but the limitation to those included in the framework of the possibility of entering into specific contracts in the terms set in the framework. AG Campos’ maximalistic position would lead to the inescapable logical conclusion that framework agreements are not public contracts for the purposes of EU public procurement law, despite being explicitly regulated, quod non.

The flawed logic of the premise established by AG Campos in para 39 of his Opinion makes the rest of his reasoning crumble. In my view, this defect affects his reasoning that

… what is determinative, in relation to the contracts subject to Directive 2004/18, is not the checking of the economic operators’ capability to provide the advisory service (qualitative selection criterion), but the comparison of the offers of the competing tenderers, once considered capable, with a view to finally chose that or those which will be entrusted with such provision (award criterion) (AGO, C-9/17, para 44, own translation from Spanish).

And that

… the selection that matters, for the purposes of the concept of public contract in Directive 2004/18, is that which results from the comparison between the capabilities and merits of the offers of the different candidates. That is, what is decisive is the final award, comparatively or by contrast, to the best offer, not the initial selection by reference to a threshold meeting which does not imply competition between the candidates (AGO, C-9/17, para 45, own translation from Spanish).

Ultimately, following this same reasoning, AG Campos takes issue with the fact that there was no competition between the candidates that expressed interest in being included in the framework agreement because the contracting authority ‘did not restrict ab initio the number of potential providers of the services, nor did it carry out a comparison of the offers between them, or chose in a definitive manner one or several of them, on the basis of a comparative evaluation of their respective contents, to the exclusion of the rest’ (AGO, C-9/17, para 48, own translation from Spanish).

However, this triggers two issues. First, under Dir 2004/18/EC, there was no obligation to establish a maximum number of economic operators to be admitted to a framework agreement. Art 32(4) Dir 2004/18/EC solely established a minimum of three for multi-supplier framework agreements, but did not require a maximum number. Second, it is in the nature of framework agreements—particularly those involving mini-competitions, as per Art 32(4)II Dir 2004/18/EC—that the contracting authority, at the point of deciding which economic operators are included in the framework, does not ‘chose in a definitive manner one or several [offers], on the basis of a comparative evaluation of their respective contents, to the exclusion of the rest’ for the purposes of the award of the relevant call-off contracts—which is the situation comparable to Tirkkonen. In particular, it is possible that an economic operator included in a framework agreement is never awarded a call-off, especially if there are mini-competitions, which in my view deactivates the functional reasoning of AG Campos.

In my view, AG Campos also misinterprets the implications of the fact that the framework agreement in Tirkkonen was closed to the economic operators not initially admitted to it, in relation to the ECJ’s Judgment in Falk Pharma. In that regard, it is relevant that the argument was made that the closed nature of the framework agreement distinguishes it from the open-ended mechanism discussed in Falk Pharma, which AG Campos rejects in the following terms:

It is true that, strictly, by limiting the contracting system, during its term, to the economic operators initially admitted by the Agency [Maaseutuvirasto] (which prevents access by new advisors) a certain quantitative restriction is being imposed. However, this is but a consequence of the pure and rigorous temporary limitation of the system of funding for advisory services, which is parallel to the program of rural development for continental Finland 2014-2020 (sic).

For the rest, the reference by the Court of Justice in Falk Pharma to the permanent openness of the contracting system to new tenderers was not, in my view, the ratio decidendi of that case, but rather a statement made ad abundantia. What was determinative in that occasion was that the contracting authority had not awarded, in exclusive, the contract to one of the tenderers [Falk Pharma, para 38].

In this case, just like in the Falk Pharma case, there has not been any element of true competition between the candidates, to evaluate which of their offers is the best and displaces, simultaneously, the remaining other (AGO, C-9/17, paras 51-53, own translation from Spanish)

The reasoning in these paragraphs is strongly skewed towards a very narrow understanding of procurement as implying the award of contracts solely to a winning tenderer, which is not the way framework agreements (and dynamic purchasing systems) operate. I cannot share the analysis in any of these steps of the reasoning.

Firstly, I think that a temporary justification for the irrelevance of the selective nature of a framework agreement is a logical non sequitur. The fact that the funding is limited to the period 2014-2020 can be used to justify the creation of a framework of six years’ duration, but it can have no bearing on the fact that a restriction of the potential suppliers derives from the framework agreement itself. The Maaseutuvirasto could have chosen a fully open licensing system, which would then have avoided the situation of excluding would-be rural advisors as a result of the one-off chance of being accepted into the system (which is a structural result of the framework agreement).

Secondly, in Falk Pharma, the ECJ did not simply consider the lack of choice of a specific supplier and consider the open-ended nature of the ‘authorisation procedure’ ad abundantia, but rather made this a crucial aspect of the analysis, by establishing it as a defining characteristic of the mechanism (see C-410/14, para 14). This is particularly clear on the explicit distinction the ECJ made with framework agreements when it stressed that

it should be noted that the special feature of a contractual scheme, such as that at issue in [Falk Pharma], namely its permanent availability for the duration of its validity to interested operators and, therefore, its not being limited to a preliminary period in the course of which undertakings are invited to express their interest to the public entity concerned, suffices to distinguish that scheme from a framework agreement (C-410/14, para 41, emphasis added).

Finally, the third point on absence of competition is also problematic. Taken to its logical extreme, this would mean that contracting authorities could avoid compliance with procurement rules where they set ‘take it or leave it’ conditions for the provision of services or supplies. This makes no sense because, particularly where there is scarcity in the number of awards (in this case, a limit of total available funding, as well as the restriction in the number of potential awardees that results from the closing of the framework agreement at the initial stage of the 2014-2020 period), there is always implicitly an element of competition (ie to tender or not, and tendering results in a constraint on the overall number/value of awards available to the other competitors) and the fact that the contracting authority limits the dimensions in which the tenderers compete (in Tirkkonen, and implicitly, their geographical coverage) should not exclude this from compliance with procurement rules.

For all the reasons above, I think the Tirkkonen Opinion misconstrues the relevance of the openness of the system in Falk Pharma, and the explicit distinction made by the ECJ between that system and framework agreements. Moreover, the Opinion gives excessive weight to the need to compare tenders or offers (and the choice of one, and almost only one, to the exclusion of all others) for (covered) procurement to take place. In particular, it misrepresents some of the particular features of framework agreements and opens the door to their de-regulation where contracting authorities set ‘take it or leave it’ conditions (eg, in this case, provision of services at rates established by the contracting/funding authority) and then delegate or decentralise decisions on call-offs, even if they provide general guidelines on the way they should take place. For the reasons set out below, I think the Opinion is not only inaccurate from a positive legal analysis perspective (as discussed so far), but also from a normative perspective.

The undesirable combined effect of Falk Pharma and Tirkkonen

Should the ECJ follow the Tirkkonen Opinion, and as a result of the cumulative effect of the resulting expanded Falk Pharma doctrine, Member States willing to avoid compliance with EU public procurement rules could easily do so by creating systems of ‘user/beneficiary choice’. This could be quite problematic particularly in the context of services and supply contracts, where the existence of end users detached from the contracting authority can always enable this type of mechanisms.

In the extreme, if central purchasing bodies created this type of mechanisms for use by individual decision-makers (eg civil servants or public employees), the atomisation of procurement that would ensue could well result in a de-regulation of the procurement function. Procurement rules would not apply to the CPB because it would not ‘choose definitely’ the specific supplier or provider, and they may not apply to the decision to call-off that does exercise that choice if the value of the call-offs is small enough—which would then trigger litigation around the legality or less of the atomisation of the procurement decision on the last stage, for which analysis the concept of ‘separate operational units’ in Art 5(2) of Directive 2014/24/EU (see also recital (20)) would become highly relevant; see K-M Halonen, 'Characteristics of Separate Operational Units – A Study on Aggregation Rules under Public Procurement Law' (2017) report for the Competition Authority; see here. There is thus a functional need to keep proper checks and balances at the level of creation of the mechanism.

On the whole, I was already concerned that Falk Pharma was eroding the scope and effectiveness of the EU public procurement rules, but Tirkkonen could magnify such undesirable effect. Moreover, this would simply displace the problem towards general EU free movement law, which is not a sensible approach in view of the more developed criteria and rules in the EU public procurement framework. Thus, also from a normative perspective, I would plea to the ECJ not to adopt the same approach of AG Campos on this occasion.

Creating reliable econometric models of the CJEU case law: a response to criticisms (by Arrebola, Mauricio & Jimenez)

One of the most satisfactory activities in academia is to engage in debate and discussion. Only by subjecting ideas to tough scrutiny can we advance in our knowledge. Thus, I am extremely pleased that Carlos Arrebola, Julia Mauricio and Hector Jimenez have reacted so quickly to my criticism of their recent paper (here) and come back with a thoughtful and forceful rebuttal. I am posting it below. You will see that there are important points of disagreement that will probably require two (or more) follow-up studies in the future. Seems like I need to brush up my econometrics...

Creating reliable econometric models of the CJEU case law:
a response to Sanchez-Graells’ criticisms

by Carlos Arrebola, Julia Mauricio and Hector Jimenez

In a recent study, we used econometric methodology to quantify the degree of influence of the Advocate General on the Court of Justice. Based on data collected from 20 years of actions for annulment, we concluded that the Court is 67% more likely to annul an act if the Advocate General suggests so in her opinion. In a post last Tuesday, Sanchez-Graells examined our paper. As he said, our conclusion is ‘bold [...] and controversial [for its] implications’, and as such it should be subject to ‘tough scrutiny’. We most definitely agree on both the importance of our claim and the need to test it rigorously. As we stated in our paper, if the conclusions are true, the role of the Advocate General within the Court might need to be reconsidered in order to secure judicial independence.

However, Sanchez-Graells voiced several criticisms regarding our econometric model that prevent him from accepting the validity of our results. We greatly welcome the debate, and appreciate the comments in his post, although we ultimately disagree. While we acknowledge that quantitative methodology is not perfect, we argue that our results are a reliable estimation of the influence of the Advocate General (hereinafter, “AG”) on the Court. If not in the specific number of 67% increased probability of a judicial outcome, our results are at least an indication that the influence relationship is positive, as it is shown by the six different econometric models estimated in our study. In the spirit of discussion and debate of this blog, we address Sanchez-Graells’ criticisms along with several other factors that, in our opinion, should have been taken into account when assessing our paper’s reliability.

1. The impossibility of using Randomised Controlled Trials

In his post, Sanchez-Graells suggests that we were too quick to discard the possibility of testing the hypothesis of the influence of the AG on the Court using Randomised Controlled Trials (“RCTs”). For a layperson, RCTs are the type of scientific methodology used in many areas of science to study causality. One of the main examples where RCTs are used is medicine. In order to prove the validity of a new drug, several groups of patients with similar features are randomly selected. Normally, one of those groups would be the control group. The control group would receive a placebo, instead of the actual drug. In this way, the researchers can easily infer whether the health outcome is caused only by the drug. If both the group taking the placebo and the group taking the drug had the same reaction, it would be clear that some external factor other than the drug had caused it. If, on the other hand, the group taking the drug and the placebo group reacted differently (for example, in the case of an illness, if the group taking the drug was the only one to recover), it could be said with certain confidence that the drug caused the recovery.

In our paper, we suggested that RCTs are not a possibility because it would require using the Court of Justice as a laboratory, experimenting with cases, judges and AGs. Nevertheless, Sanchez-Graells argued that we should have considered those cases in which the AG does not participate as our “control group”. This is a misconception about how RCTs are designed. A vital feature in the design of RCTs is making sure that the observations that included in the sample are randomly drawn. This is because, ideally, you would like every observation to be identical, so that the only factor that affects it is the treatment that you are examining in the experiment. In the case of medicine-related RCTs, you want patients with the same characteristics, symptoms, etc., so that whatever happens after taking the drug can only be traced back to the drug. In our study, we would need the same case to be repeated several times, with the same legal problem to be solved by the same judges, having access to the same amount of precedent, lawyers with the same ability to plead cases, etc. Only having that could we then observe what would happen if we took the element of the Advocate General out of the equation. However, cases are never the same. Unlike illnesses, where patients tend to have the same symptoms, cases are much more complex. Legal problems rarely have the same surrounding circumstances.

So, if we followed Sanchez-Graells’ suggestion, we would be ignoring a set of external factors that actually affect the outcome of a case. We would be wrongly attributing it to the Advocate General’s intervention, when actually it could be something else. That is, if we had two cases, one with an AG’s opinion, and one without, in which the Court reached different results, we could not say that the Advocate General caused that different result. It could be that the case had different facts, and that is why the Court decided differently. Or, it could well be that the judges were presented with different arguments by the parties, and it was the lawyers, and not the AGs, who persuaded the Court. Furthermore, Sanchez-Graells’ suggestion is unfeasible because there is a clear bias. As he explained, the cases in which the CJEU considers that there are not going to be problematic legal issues, they decide not to have an AG opinion. It means that from the very beginning of the case they are sensing that it might have an easy or clear legal solution. In other words, Sanchez-Graells is suggesting that we compare in our analysis a simple cold, with a more complicated condition, such as cancer, and that we can thus establish whether radiotherapy has any impact on health. The outcome to such a query would have a misleading result, because the colds would have a rate of recovery close to 100%, whether the cancer would be lower. However, that would not tell us anything about the effectiveness of radiotherapy. In the same way, if a case deals with unproblematic legal issues, the opinion of the AG will probably not do much to affect the Court, because the Court would have come to that conclusion by itself without any external influence. We cannot simply compare those two scenarios without losing information. After all, there would not be any “random” selection of groups, clearly not fulfilling the requirements to conduct a RCT.

For that reason, the only way to approximately estimate causality is to use regressions, in which you can account for as many variables as possible that may influence the Court, including the Advocate General, and including variables that will account for how easy it is to solve a case or clear a case is. That way we will know the exact magnitude of the variable AG on the Court.

2. Designing a reliable regression

Once we establish that the most accurate measure is a regression model accounting for variables that affect the outcome of the Court, the difficulty arises in deciding which variables to include and how to code them. It is in this respect that we think Sanchez-Graells raises his most valid criticism of our study. We acknowledge that our variables are not perfect. We will never be able to establish causality without a shadow of a doubt. This is simply because, as we said, we will always miss variables that affect the case that we will not be able to track, codify and insert in our database. Taking this to an extreme and absurd example, we will never be able to verify whether the judge in the deliberating room had a headache and wanted to go home soon, rushing her decision. However, the fact that we will always miss variables does not mean that our model cannot be reliable. We still include a number of important variables that can explain a substantial amount of what goes on in the courtroom. There are different ways in econometrics to determine the extent to which a model, albeit missing variables, is an accurate depiction of reality. For our study, these measures suggest that the model is indeed reliable. We will come back to this in a moment.

Another aspect of coding variables is, as Sanchez-Graells comments, the oversimplification. In our study, we used actions for annulment, where the outcomes of a case can be (i) annulment, (ii) partial annulment, (iii) dismissal of the case, or (iv) inadmissibility of the case. We decided to simplify this variable by looking only at whether the Court decided to annul (in any of its forms) or not. But, the oversimplification is necessary to make it more reliable, because in order to have a dataset capable of yielding significant results, we need to have a representative sample. In our case, we only had data for a very small number of partial annulments. Including them as a separate variable from total annulment would have only created “noise” in our model, making the results less significant, statistically speaking.

Sanchez-Graells especially criticises our grouping of dismissal and inadmissibility cases together, because he says that dismissing a case and declaring it inadmissible are very different things. However, that discussion in his post is unnecessary, because as he himself notes later on, our results ‘cannot be interpreted regarding inverse AG recommendations (ie recommendations to inadmit/dismiss)’. Our results are only relevant for decisions to annul or partially annul; we do not make any claim about other type of cases, which Sanchez-Graells also criticises.

However, the fact that we decided to look at the question in terms of what happens if the AG suggests to annul the act, rather than if she suggests to dismiss it or declare it inadmissible, does not affect the reliability of our results. In fact, the only thing that Sanchez-Graells is postulating is a new hypothesis. He is saying that, in his opinion, we would have got other results if we had constructed the model differently. That is a point that we cannot falsify without fiddling for a few more weeks with our data in the econometrics software. But, we invite people, and we ourselves may do it in the future, to carry out other studies, with the same or different data to check that the results are not affected if we look at things in a different way; by, for example, looking at what happens if the AG suggests dismissal, or what happens if we gather data from other periods of time. Nonetheless, the reliability of the results that we presented is a separate issue.

So, if we have acknowledged that we are not going to be able to include every variable, and that our data is only a sample, why are we confident in our results? In the paper we explain it more technically, but, basically, there are econometric measures that indicate that the model that we have created is accurate when the estimation that we get from the model is compared with actual data from reality. That is the reason why we know it is a fairly reliable model.

3. Final caveat

Whilst reading Sanchez-Graells’ words, we could not avoid feeling something we felt many times before. Lawyers are more comfortable sticking to arguing with words.  We feel somehow threatened by this terra incognita called econometrics. There seems to be a certain reticence to attempting to use mathematics to help us in our enquiries. It is worth saying that we are not accusing Sanchez-Graells of not wanting to engage with quantitative methodology. In fact, we know that he has used some statistics previously, and we would not expect a “more economic approach” type of person to disregard this evidence-based methodology.

We want to end this post with a final note about quantitative methodology. We want to say that although judicial proceedings and legal arguments cannot always be equated to numbers, and other methodologies are extremely valuable to legal research questions, quantitative analysis can help elucidate complex legal questions. As many other subjects in social sciences did before us, statistics can become a tool at the service of legal researchers. In this sense, it is worth reminding the readers that, a few centuries ago, economics was equally a merely discursive subject, and anyone who has read the Wealth of Nations can be a witness to that.  But, now, economics and mathematics cannot be separated. Therefore, we would encourage researchers to embrace statistics and econometrics, and see how they can help with their enquiries. Quantitative analysis tries to be evidence-based and objective. Therefore, anyone who believes in the benefits of science will prefer a claim based on quantitative methodology to a hypothesis made, to follow the words of Sanchez-Graells, on the basis of ‘anecdotal impression’.

The difficulties in an econometric analysis of CJEU case law -- a propos Arrebola, Mauricio & Jiménez Portilla (2016)

Carlos Arrebola and Ana Julia Mauricio (PhD students at the University of Cambridge), together with  Héctor Jiménez Portilla (of the Overseas Development Institute (ODI)) have published an interesting and thought-provoking  paper (*) where they try to measure the influence of the Advocate General (AG) on the Court of Justice of the European Union (CJEU) [for a short summary of their paper, see here]. This is an area where EU law scholars have been struggling to find an objective way to measure/prove/dimiss any claim of AG influence over the CJEU--as Arrebola et al clearly stress in their excellent literature review.

In a nutshell, Arrebola et al claim that their 'findings suggest that the CJEU is approximately 67 percent more likely to annul an act (or part of it) if the AG advises the Court to annul than if it advises the Court to dismiss the case or declare it inadmissible. In their view, these results raise several questions as regards judicial independence and the relevance of the figure of the Advocate General, providing a grounded basis for future discussions and judicial reform.'

Their claim is as intuitively appealing as it is bold (and controversial, in terms of the implications they derive) and, in my view, it deserves a tough scrutiny of the way they reached this conclusion. The following are some of the doubts that I have had while reading the paper, which I am limiting to the three main doubts I am struggling with. Overall, these doubts leave me with the impression that, unfortunately, the paper does not actually deliver on its main goal of contributing 'to a more comprehensive understanding of the role of the Advocate General in the makeup of the Court of Justice of the European Union'.

Their model in a nutshell

Let me frame my doubts in an stylised summary of their econometric model. In short, they have looked at 'data from 20 years of actions for annulment procedures before the Court of Justice. Every case from January 1994 to January 2014 has been included, with the exception of appeals from the General Court and those cases that do not have an AG opinion. We collected a total of 285 observations. For these cases, we have examined the behaviour of the Court and the Advocate General as regards to their decision to annul or not to annul the legal act in question' (p. 15). 

They have coded these cases to examine the relationship between two main variables: the recommendation of the AG and the final decision of the CJEU. There are other variables they take into account, but those do not affect my analysis, so I am sticking to the two main variables for simplicity of argument. They explain why they have chosen annulment cases in the following terms: 'we have created two dichotomous (also called dummy or binary) variables: ECJannulment and AGannulment. ECJannulment is the one that we have considered as the dependent variable. It takes the value of 1 if the Court decided to annul or partially annul an act, and 0 if it dismissed the case or deemed it inadmissible. AGannulment is the variable that we have considered independent. It takes the value of 1 if the Advocate General issued an opinion recommending the Court to annul or partially annul an act, and 0 if it recommended dismissing the case or declaring it inadmissible' (p. 15). 

With this information, they have run a 'probit model [which] is a regression that explains the predicted probability of the dependent variable adopting the value 1. In our case, it outputs the predicted probability of the Court annulling an act, subject to the value given to the other variables included. Therefore, the probit model provides a simple way to interpret the results in terms of predicted probability from 0 to 1. Instead, if we had chosen a linear regression model, the result would not be enclosed between 0 and 1, making the interpretation impossible, as it could yield some predicted probabilities to be negative or above the unit' (p. 25). This is what allows them to reach their main finding that 'when the Advocate General recommends annulment, the Court is 67 per cent more likely to annul' (p. 30).

My main doubts

Firstly, I am not sure that the model the authors use is the best suited to the analysis of such a complex issue as the influence of the AG on the CJEU. One of the reasons (probably the main reason) why the authors decide to use a probit model is that they consider that it is not possible to establish a group of annulment cases that can work as a control (ie what they call the impossibility of conducting a randomised controlled trial). They consider that this would be the best way to avoid selection bias, but that in their study 'it is not possible to create a randomised controlled trial to define the causal effect of the AG opinion on the Court of Justice. This would require having the ability to design empirical experiments using the Court of Justice as a laboratory, which is unfeasible in practice' (p. 13, with more details in fn 54).

I disagree with their view about the impossibility to use a randomised controlled trial. There is a group of annulment procedures where no AG Opinion was submitted, and this could be used as a control group. It is important to note that, according to the Statute of the CJEU, '[w]here it considers that the case raises no new point of law, the Court may decide, after hearing the Advocate General, that the case shall be determined without a submission from the Advocate General' [Art 20(5)]. 

This is organised according to the Rules of Procedure of the CJEU, according to which 'The preliminary report shall contain proposals as to whether particular measures of organisation of procedure, measures of inquiry or, if appropriate, requests to the referring court or tribunal for clarification should be undertaken, and as to the formation to which the case should be assigned. It shall also contain the Judge-Rapporteur’s proposals, if any, as to whether to dispense with a hearing and as to whether to dispense with an Opinion of the Advocate General pursuant to the fifth paragraph of Article 20 of the Statute. The Court shall decide, after hearing the Advocate General, what action to take on the proposals of the Judge-Rapporteur' [Art 59(2) and (3)].

Therefore, the annulment cases where there is no AG Opinion are an important instrument for potential control tests. These cases only come to be decided without an AG Opinion because both the CJEU (rectius, the Judge-Rapporteur) and the AG agreed that the case raised no new point of law. Thus, there is no indication that the AG can influence the CJEU on any other point than the existence or not of new issues to be considered. Admittedly, there could already be scope for some indication of the AG (and the CJEU's) position on the substance of the case in this first judgement. However, I would think that running controls on the basis of these cases could be useful.

In these cases, the CJEU (at least formally), decided whether to admit or dismiss, annul (totally or partially) the case without submission of the AG. If there was a significant divergence of the probability of annulment between these two groups of cases, the argument that the author's raise in the paper would be strengthened. On the contrary, if the CJEU showed the same likelihood of annulling/dismissing regardless of the existence or not of an AG Opinion, the claim would be significantly weakened. I do not imagine this to the ultimate test for the arguments raised in the paper, but I would see it as an important one.

Secondly, I am skeptical of the way in which the authors simplify the setting for annulment procedures. They construct them as binary: that is, the only options available to the AG and the CJEU are to either declare the case inadmissible/dismiss it (0) or annul the provision in question totally/partially (1). I understand the need to simplify decisions to annul grouping together full and partial annulments (which they explain in p. 17). I remain unconvinced by their arguments regarding declarations of inadmissibility and dismissals. They simply indicate that 'inadmissibility and dismissal are sometimes used as interchangeable terms, although technically the substance of the case is not analysed in cases of inadmissibility, whilst it is in cases that are dismissed' (p. 17). However, they do not consider this a major issue and proceed with the grouping of both types of results as a single outcome of the case.

The difficulty I have with this strategy is that the rules on admissibility/inadmissibility are procedural in nature and they set up a first filter for cases to come to a full analysis. It can also be argued that they are much simpler than the rules applicable to the potential annulment of the challenged provisions, which depend on much more complex assessments of both procedural and substantive EU law. Thus, grouping decisions on (procedural) inadmissibility with those on dismissal of the annulment claim after a full analysis seems to create a significant conceptual problem. At this point, it may be worth stressing that the authors had mentioned that 

we have decided to estimate regressions including other variables that could potentially be biasing the results if we only looked at what the Advocate General said and whether the Court followed the Advocate General’s position. In particular, one of the bias factors is the clarity of the law in a given case. For example, the Court and the Advocate General could reach the same result in a case not because the Court decided to follow the AG opinion, but because the law was clear on what the outcome should be, and there was no room for different interpretations. Therefore, not accounting for the clarity of the case could overestimate our measure of the influence of the Advocate General (pp. 14-15, emphasis added). 

My problem is that the authors seems to have forgotten to include this very bias-check in the way they have constructed their variables. By grouping (relatively simpler) procedural checks with (relatively more complex) full assessments, they have created a variable that is very hard to reconcile with reality outside of their model.

Thirdly, even within the context of their model, I am not sure what to make of their results. Their findings indicate that, when the AG recommends the annulment of an act, the CJEU is almost 67 per cent more likely to annul the act than if the AG had not proposed its annulment (ie, had she advocated for either inadmissibility or dismissal). I have trouble interpreting this number due to the conceptual issue mentioned above (ie, conflation of inadmission and dismissal), which makes the recommendation of the AG (as coded) ambiguous. This makes me wary of the claim that 'even if the number of 67 per cent of increased probability is called into question, it is difficult to deny that there is some level of influence' of the AG on the CJEU (p. 34, emphasis added), and that 'our analysis shows that there is some component in the making of a decision that is simply attributed to what the Advocate General recommended' (p. 35, emphasis added).

From the numbers in the paper, I have been unable to work out the effect that an AG recommendation to inadmit/dismiss has on the CJEU's willingness to do so. Intuitively, I would expect that, if by itself the Opinion of the AG is such a relevant factor as the paper claims, then the CJEU should also be more inclined to inadmit/dismiss when the AG submitted such a recommendation. However, in that case, I would not necessarily find the causal explanation between the AG recommendation and the CJEU's decision persuasive. An alternative interpretation not linked to the influence of the AG over the CJEU would need to be dispelled: ie the zeal with which the CJEU keeps control of its docket. The intuition would be that the CJEU may be engaged in an interpretation of inadmissibility rules that prevents a floodgate of claims, which could well override whatever position the AG decides to take. In my personal opinion, and based on anecdotal impression, this is what has been happening regarding annulment procedures promoted by unprivileged applicants (with all the issues that the Plaumann, UPA, Inuit, saga have created; see here).

In the end, the difficulty I have is that their results do not necessarily make a lot of intuitive sense because they cannot (or at least not immediately) be interpreted regarding inverse AG recommendations (ie recommendations to inadmit/dismiss) and their effect on the CJEU. Somehow, there seems to be an implicit assumption that 'influence' of the AG is stronger if it prompts the CJEU to annul than if it prompts the CJEU to inadmit/dismiss. If all of this is incorrect, then my only residual criticism is that the paper could have been made more accessible for non-statisticians.

Conclusion

Overall, I remain unconvinced that the results of Arrebola et al significantly contribute 'to a more comprehensive understanding of the role of the Advocate General in the makeup of the Court of Justice of the European Union'. Thus, I am not prepared to engage with the implications in terms of judicial independence and potential (further) reform of the CJEU that they draw (pp. 34-38). Given the disagreement with their methodology and the diversity of views as to how to interpret their results, I have contacted Carlos Arrebola and offered him to reply to my criticisms in a guest post. He has kindly accepted. Keep an eye out for it in the coming days.

(*) The full reference for the paper is: C Arrebola, AJ Mauricio and H Jiménez Portilla, 'An Econometric Analysis of the Influence of the Advocate General on the Court of Justice of the European Union' (January 12, 2016). Cambridge Journal of Comparative and International Law, Vol. 5, No. 1, Forthcoming; University of Cambridge Faculty of Law Research Paper No. 3/2016. Available at SSRN: http://ssrn.com/abstract=2714259.

AG (dangerously) stresses possibility to indirectly challenge State Aid decisions via Art 267 TFEU

In his Opinion of 27 June 2013 in case C-284/12 Deutsche Lufthansa, Advocate General Mengozzi stressed that (provisional) Decisions of the European Commission in State Aid cases are open to (indirect) challenges via a reference for a preliminary ruling on their validity under Article 267 of the Treaty on the Functioning of the European Union. 

In the case at hand, the complainant before the Commission seeked interim measures against the beneficiary of a measure that the Commission qualified as State aid in its decision to open a formal investigation. The domestic court competent in the matter remained unconvinced by the Commission's preliminary assessment and seeked ways not to adopt interim measures on the basis of such an assessment. It referred the following question to the CJEU:

Does the uncontested decision of the Commission to initiate the formal investigation procedure under Article 108 paragraph 3, second sentence, result in the national court seised of a procedure which aims to recover payments already made and the prohibition of future payments being bound by the legal assessment expressed by the Commission in that decision on the state aid character of the measure in question?

In paragraph 42 of his Opinion, AG Mengozzi indicates that:

under the combined effect of Article 108 paragraph 3, last sentence, and the qualification as a new aid of the controverted measure [in the provisional decision of the Commission], the opening of the formal investigation procedure generates the obligation of the Member State concerned to suspend its execution from the date of adoption of the decision to open the investigation and until a final decision is reached, regardless of the objective nature of the controverted measure [...]. National courts will therefore be obliged to take all necessary measures to ensure compliance with this requirement and to eliminate the consequences of any breach thereof, regardless of any previous assessment of the measure under Article 107, paragraph 1. In case national courts harbour doubts about whether the requirements to qualify the measure as aid are met in the given case, which justify the initiation of the formal investigation procedure, national courts may refer a question of validity under Article 267 TFEU, first paragraph, letter b) (Opinion in C-284/12 at para 42, own translation from Spanish).

This comes to stress the (procedural) difficulties derived from the joint competence of domestic courts and the Commission to interpret and apply the notion of aid under Article 107(1) TFEU--as stressed in paragraph 10 of the Commission Notice on the enforcement of State Aid law by national courts, which also metions the possibility for a preliminary reference in paragraph 90, but (impliedly) in a context where no concurrent Commission investigation is in place--and can create significant complications by way of parallel procedures (before the Commission, the national courts and the CJEU) in one and the same case. Such duplication of procedures can only result in a waste of resources and, most likely, in legal uncertainty and potentially contradictory outcomes.

In my view, leaving the door open for a reference for a preliminary ruling (of validity) against a provisional assessment of the European Commission is excessively deferential towards domestic courts and can have significant undesirable effects. This is not satisfactory and would justify the adoption of a more streamlined procedural system whereby national courts would have to suspend their powers of interpretation of the concept of aid and limit their role to the adoption of effective interim measures when the Commission is still completing its investigation on a given measure. 

In my view, this could be easily achieved by simply applying Article 4(3) of the Treaty on European Union, since the need for sincere cooperation in this type of matters seems out of the question. It will be interesting to see how far the CJEU is willing to go in the balance between the sphere of jurisdiction/competence of domestic courts and ensuring a mangeable procedural system in State aid law.

This is not (well, yes) binding, but (maybe) you can disregard it. AG Kokott on soft law and EU competition policy

On 6 September 2012, AG Kokott issued her Opinion in case C‑226/11 Expedia Inc. The case is about the effects of soft law instruments adopted by the European Commission on other competition authorities and courts entrusted with the enforcement of EU Competition Law. Or, as the AG shortly puts it, the CJEU must decide if  the notices of the European Commission in the field of competition law are binding on the national competition authorities and the national courts.

The question has arisen in relation to the ‘de minimis notice’, in which the Commission sets out the circumstances under which it presumes that there is an appreciable restriction of competition within the meaning of Article 101 TFEU. Given the narrow scope of the question (ie whether the de minimis notice is binding), it is odd that the case has actually gone through, since the notice itself clearly indicates that "[a]lthough not binding on them, this notice also intends to give guidance to the courts and authorities of the Member States in their application of Article [101 TFEU]" (para. 4, emhasis added).

The answer should almost be automatic: "No, it is not binding". However, since most domestic EU rules (directly or indirectly, expressly or implicitly, willingly or reluctantly) incorporate the corpus of Commission's competition notices, the question becomes trickier than a mere literalist approach to the (originally) soft law instruments would suggest.

Therefore, as clearly indicated by the AG, given that the case affects the 'enforcement thresholds' for EU Competition rules, it can be seen as a matter involving the delineation of the scope of application of EU Competition Law: "The Court’s reply to the question referred will to a large extent determine the scope which the national competition authorities and courts will have in the future when applying Article 101 TFEU" (AG Kokott in C-226/11, at para 5).

However, the general framework in which the reference for a preliminary ruling by the French Cour de Cassation has been assessed by AG Kokott in Expedia relates to the broader topic of the legal nature and effects of soft law instruments in the area of EU Competition Law (and, more generally, on the topic of 'soft EU Law', although in most other policy areas it is much less used)--which oddly enough (and probably not unexpectedly), are in a fast and steep 'hardening' process that might as well end up equating them to full EU legislative instruments, at least in terms of their legal effects.

This topic is a personal favourite of mine, and one which is due to raise a myriad of cases in the decentralised system of (private) EU Competition Law enforcement (as discussed in Sánchez Graells, A., 'Soft Law and the Private Enforcement of the EU Competition Rules' in JL Velasco San Pedro (ed), Private Enforcement of Competition Law, Valladolid, Lex nova, 2011, p. 507-520,  http://ssrn.com/abstract=1639851).

Even if the wording of her opinion seems to adjust to the natural, automatic answer already hinted at:  "it must be concluded that the de minimis notice is not, of itself, intended to produce binding legal effects" (see AG Kokott in C-226/11, at paras. 26 to 34), actually, her position is almost the opposite. In my reading of her opinion, AG Kokott basically submits to the CJEU that Commission's notices are (somehow) binding on the national competition authorities and the national courts in that they must take them in due account when conducting competition analysis, but that authorities and courts can depart from the content of the notices as long as they prove in some way that the content of the notice is inadequate (in the case at hand)--which brings an onus of proof to the picture that seems to tilt the standard position to the Commission's soft law instruments indeed having (initial) binding effects.

This is a rather creative and flexible solution (which circumvents the standard position that soft law instruments cannot generate binding legal effects), for sure, but one that leaves many questions unanswered and that merits some further thought. In that regard, it is interesting to see how the AG reached her conclusion.

According to AG Kokott,

35. Although the de minimis notice has no binding legal effects, as I have just shown, it would be a mistake to regard it as of no importance at all in law for proceedings concerning cartels. Publications like the de minimis notice are in the nature of ‘soft law’ the relative importance (sic) of which in cartel proceedings, at the European and the national levels, should not be underestimated [...]

38. The Commission’s leading role, firmly anchored in the system of Regulation No 1/2003, in framing European competition policy would be undermined if the authorities and courts of the Member States simply ignored a competition policy notice issued by the Commission. It therefore follows from the duty of sincere cooperation which applies to all the Member States (Article 10 EC, now Article 4(3) TEU) that the national authorities and courts must take due account of the Commission’s competition policy notices, such as the de minimis notice, when exercising their powers under Regulation No 1/2003 [...]

39. [...] even though no binding requirements concerning the competition-law assessment of agreements between undertakings arise for national competition authorities and courts from the Commission’s de minimis notice, those authorities and courts must nevertheless consider the Commission’s assessment, as set out in the notice, of what constitutes an appreciable restriction of competition and must give reasons which can be judicially reviewed for any divergences (AG Kokott in C-226/11, at paras. 35-39, references omitted and emphasis added).

In my view, this is quite an amazing exercise of saying one thing and the opposite in the space of less than 10 paragraphs. Reading the conclusion suggested by AG Kokott, one is left scratching his head and thinking "so, no legal effects, huh?":

Consequently the national competition authorities and courts are free to proceed against agreements between undertakings below the thresholds of the de minimis notice, provided that they have taken due account of the Commission’s guidance in the notice and that, in the particular case, there is evidence, other than the market shares of the undertakings concerned, which suggests that the effect on competition is appreciable (AG Kokott in C-226/11, at para. 43, emphasis added).

I think that this is a very dangerous step that follows the rocky path of attaching (soft!) legal effects to soft law instruments, and I sincerely hope that the CJEU will only follow the position advanced by AG Kokott in paragraphs 26 to 34 of her Opinion in Expedia. More specifically, the preferable answer to the reference for a preliminary ruling by the French Cour de Cassation would be very short: "[a]s the Court has found in another connection, Commission notices in the area of EU competition law do not have binding legal effect for national authorities and courts. That is so also in the present case with regard to the de minimis notice" (AG Kokott in C-226/11, at para. 26). Any other answer will open a Pandora's box.