Contents
Abstract
	1. Introduction
	2. On measuring justice
	3. On the sea of indicators
	4. The methodologies and methods of measuring justice
	5. A virtuous hegemony of good governance?
	6. The dilemmas of measuring
	7. Alternatives?
	8. Conclusion
paliwala&kamchedzerapdf  paliwala&kamchedzera

 

JUSTICE INDICATOROLOGY: A NEW THEATRE FOR JUSTICE?**

Abdul Paliwala^* and Garton Kamchedzera^#

---

Abstract

This article interrogates the images of justice produced by justice indicators, which tend formally to become truths about the nature of justice and injustice in jurisdictions worldwide. However, what are these indicators really constructing? Underlying the construction of indicators and indicatorology may be forms of discipline and power implicated in what Mitchell terms ‘rule by experts’.¹

This article analyses the ‘sea of indicators’ of justice, and consider ways in which they construct these new forms of discipline and power. The article then suggests that more qualitative approaches to measuring justice, which have been recently favoured by the World Bank and the UNDP, among others, may not necessarily provide a solution as they ignore the wider realities of global injustice. The article then argues that it is necessary to interrogate the internal dynamics of indicatorology with the wider alternative frameworks of global (in)justice as indicated in the work of Pogge, Baxi, Santos and Sen.

---

KEYWORDS

Justice Indicators, Malawi, Rule of Law, Good Governance, Counter-hegemony, Subalternism

---

1. INTRODUCTION

In 2008, a workshop of experts, academics, and practitioners on development work on justice noted that “the world today is swimming in indicators of justice, safety, and the rule of law.” The same workshop, however, noted, in disappointment, that the available indicators focused “so much on rules and activities and not on people and experiences.”² In 2012, two expert institutions in measuring the rule of law, including justice, The Hague Institute for the Internationalisation of Law and The World Justice Project respectively claimed a “measuring revolution” on measuring the rule of law, underlining the quality, amount, and usability of indicators.³

This article interrogates the images of justice produced by justice indicators, as advanced by such experts. These images then become truths about the nature of justice and injustice in jurisdictions worldwide. However, what are these indicators constructing? The article is part of a continuing research project considering justice indicators in Malawi. Our starting concern was whether Malawian society was becoming more just and better governed with greater emphasis on the rule of law or, was the situation getting worse? At its inception, we felt that we were swimming in a sea of indicators and thus our initial concern became that of deconstructing this sea with particular reference to its contribution to dealing with palpable social injustices.

This article is a preliminary analysis of the links between measurement of justice and injustice as part of contemporary technologies of discipline and power. It involves a critical examination of the objectives and conceptual framework of those engaged in developing indicators; the nature of the research methodology, and the linkage of the performance of public duties to injustice. In particular, it suggests that the indicators which attempt to measure aspects of ‘justice, good governance and rule of law’ are devices in the construction of images of justice. It is suggested that this principle continues in the more recent transition in which the dominant quantitative approaches are supplemented by more qualitative approaches to measuring justice which have been recently introduced by the World Bank and the UNDP, among others. This is because these indicators ignore the wider realities of global injustice and the problematic relationship between justice and (in)justice. It is thus necessary to consider and interrogate the internal dynamics of what we have called ‘ indicatorology’ with the wider alternative frameworks of global (in)justice as indicated in the works of Pogge, Baxi, and Santos, in contrast with that of Sen.

2. ON MEASURING JUSTICE

Both measuring in general and the measuring of justice may be Aristotle’s long legacy. For Aristotle, measuring was an ethical phenomenon intrinsically connected with perception and thinking. Aristotle’s two major contributions to the idea of justice are his principle of equal distribution for male citizens (i.e. not women or slaves), a principle which finds its echoes in Rawls; and the principle of ‘ reciprocity’⁴. Thus reciprocity is the basis on which the exchange of goods and services and in fact all things can be assigned values and be measured. Money, for example, becomes a key measure of all things.⁵ What we have here is the basic highly contradictory relationship between measuring, justice, and money (as the symbol for market) which continues to this day. Shakespeare’s tragic figure of Shylock, as both the creator and victim in The Merchant of Venice, articulates the precise nature of the contradiction by which the abstract measure of justice in itself produces injustice.⁶ Thus, if legal ‘justice’ itself produces ‘injustice’, the concepts are not necessarily complementary in the sense that ‘injustice’ is an absence of ‘justice’ but different.⁷

Thus the recent mushrooming of justice (and we will add good governance and rule of law) indicators has at its core fundamental contradictions between the nature of justice and injustice. The “who”, “what” and “why” of the measuring of justice are critical issues in the construction of images of justice. In this regard, Morse suggests that we may have a new science of indicatorology:

Indicators can be powerful and useful tools. They summarise complexity, not by accident, but by design, and speak with a quantitative and apparently objective authority which commands respect. But such power works both ways and can be used to support recommended action from all sorts of perspectives…[M]uch depends on who selects…, the ways in which they are ‘measured’ and presented. The power held by those wielding indicators is rarely acknowledged, and instead the processes of creation and use are presented in benign, technical and, of course, objective language.⁸

Thus underlying the construction of indicators and indicatorology may be forms of discipline and power implicated in what Mitchell considers ‘rule by experts’ (Mitchell, 2002).⁹ In his fascinating study of two Egyptian colonisations, Mitchell suggests that a key aspect of modern social science has been the character of ‘calculability’ or a new ‘politics of calculation’ which is based on ensuring, among other things, that quantities and performances are ‘measurable’. It is such intricate measurability which is then used to construct the forms of discipline and power which became part of the new truth of modern colonial (and now post-colonial) scientific dominance or ‘rule of experts’.

3. ON THE SEA OF INDICATORS

The 2008 workshop on indicators was scathing in its criticisms about the pertinence of the array of indicators on access to justice and safety. Not “everyone is happy with or helped by the growing global supply of indicators”, was one of the conclusions.¹⁰ Some at the workshop stated that they were distracted by the sheer array of indicators and the burden associated with their collection. Few of the indicators seemed relevant to the aspirations of justice and concerns of safety in the associated countries, the conference observed. There was too much focus on the rules and activities of individual government agencies. The workshop further noted that “ few of the indicators in play today are sensitive enough to register and reward incremental change in a realistic time frame.”¹¹ The agreement at the workshop was that “indicators should capture ‘the human dimension’ of justice.”¹² Thus, although there was a sea of indicators on justice and safety, there was also much frustration from the perspective of actual experiences.

In 2012, amidst“mounting skepticism regarding the success of rule of law promotion by the UN and other international organizations and donors at the national and international levels during the past two decades”, two expert institutions in measuring the rule of law, including justice, The Hague Institute for the Internationalisation of Law and The World Justice Project respectively claimed some revolution in measuring justice:

A measurement revolution has taken place in the fields of governance, justice, and the rule of law. Not only have the quality and amount of available data exponentially increased in the past two decades, but more importantly, the knowledge about precisely how to effectively use these data to advance reform in the field has greatly improved.¹³

However,the Hague Institute for the Internationalisation of Law and The World Justice Projectclaimed that such knowledge remained “buried in the hands of a handful of experts scattered around the world; it has not been fully internalized by the rule of law community, and it remains largely ignored by government reformers in all corners of the world today.”¹⁴

So, what do we make of such sea of indicators and “innovations”? Our initial assumption was that there was an absence of coherence. However, we now discern chaotic patterns which involve construction of dominant images of justice. It is apparent that indicators are generally formulated by, or on behalf of, agencies involved in promotion of good governance, rule of law and justice at a global level. Thus the agencies involved include agencies of the UN such as the UNDP, UNHCHR; international financial institutions such as the World Bank; organizations of the rich nations such as the OECD; and developing country aid donors such as USAID, DFID (UK), sida (Sweden), SPFG (Spain). These represent aid giving organizations who have been promoting good governance, rule of law and justice and the use of indicators becomes an obvious instrument of assessment and control of aid. The significance of this is that for the aid donors, the perspective will inevitably be achievement of their own top-down objectives, that is, “How are we doing?” rather than objectives framed by people within the countries themselves. It has observed as follows:

A fair and efficient justice sector is critical to the protection of human rights and the resolution of disputes between citizens. It is also essential for ensuring the proper functioning of public institutions and encouraging national and foreign investment. Measured against The World Bank’s rule of law (ROL) governance indicator, all 48 countries in sub-Saharan Africa scored poorly, with 41 countries ranked below the 50th percentile. These results are a clear reflection of the significant hurdles faced by countries and donors alike in their efforts to improve rule of law in the region.¹⁵ (Emphasis ours)

A second tier of organizations within indicatorology are global civil society institutions who also aim to promote good governance, the rule of law and justice. These include agencies such as the Open Society Institute, Bertelsmann Transformation Index, the American Bar Association-sponsored World Justice Project, Transparency International and the Mo Ibrahim Foundation.¹⁶ While the Mo Ibrahim Index is formulated on behalf of an African foundation, it was initially organized from Harvard’s Kennedy School of Government. The Vera Institute for Justice is a foundation based in New York which aims to “help leaders in government and civil society improve the systems people rely on for justice and safety.¹⁷ Like Harvard’s Kennedy School and the Brookings Institution currently involved with the Mo Ibrahim Index, the Vera Institute acts as an agency to develop an index for others. More innovatively, the Vera Institute has worked on the World Justice Project in its ALTUS partnership with institutions in five jurisdictions of the global South. In principle, the organizations at the forefront of the development of indices are Northern-led. The culture is one promoting global rather than local agendas of good governance and rule of law.

In the globalised post-colonial environment, the necessity for measurement and calculable criteria becomes even greater for experts and consultants who are not steeped in local cultures but see salvation in the techno-science of comparative calculation¹⁸. The experts both construct and are constructed by ‘institutional imperatives’ of bureaucratic organizations such as the World Bank. It has been observed as follows:

For fiduciary and quality control mechanisms to work at scale in such an organization, project documents need to be written and assessable by non-specialists providing results indicators that can be easily measured; the privileging of templates, bullet points, matrices, or the simple triangle to deftly link together fundamental aspects of social life leaves little space for the inherent complexities that characterize a given context. To get approval to support a particular initiative or policy reform, staff must present a coherent, persuasive story about how certain actions will lead to expected change in a given context in a given time frame(usually three to five years).¹⁹

Yet, such constructed calculations become claims to truth with dramatic results. A calculation that a country has slipped in an index of the rule of law can have an immediate and dramatic effect in negotiations between governments and donor agencies, but may also be evidence to the local population of the poor state of governance of their country. This interesting tender for an indicators agreement for Malawi provides an insight into indicators:

TA on Social Sector Indicators for the Performance Assessment Framework²⁰ :

The objective of the proposed Technical Assistance contract is to provide a team of Experts to support the Malawi Government in developing M&E system that will deliver reliable data to allow for a regular monitoring on the MGDS, the PAF and the PRBS2 indicators.

• Follow up with relevant Ministries and other relevant arms of government on PAF information;
• Liaise with the Director of M&E in MEP&D on how best to establish an M&E system for efficient data collection in relation to the PAF;
• Review the existing data collection mechanisms and provide options for betterment;
• Ensure that local indicators conform and be adapted to internationally defined measures (MDGs);
• Prepare documentation required for accessing financial support to undertake the improvements proposed to the M&E system in terms of human resources, equipment needs etc;
• Assess capacity needs gaps in the beneficiary institutions affecting PAF indicators and devise / propose interventions;
• Assist in the alignment of PAF indicators with GoM M&E Road Map and MDGS indicators for consistency.

Beneficiary: MW - Malawi Ministries of Health, Education and Gender, Malawi: Duration: 2007, FED

While the ostensible objective of the programme is to assist the main beneficiaries, the relevant Government of Malawi ministries, the framing of the programme is such that the ‘team of experts’will assist in the production of monitoring data for assessment by other experts in conformity with international standards.

The approaches to indicators vary between measurements of social justice, such as the quality of life, to those of legal justice such as the rule of law and access to justice. Firstly, there are the broad brush approaches which attempt to measure the general quality of life of which justice is a key component. The most well-known of such approaches is the Human Development Index which is used by the UNDP in its Human Development Report released annually.²¹ Within this overall framework, the idea of security and safety become key components to measure among the indices used by the OECD and the Vera Institute of Justice.²² When related to the performance of the justice system, this approach could be useful to show the appropriateness and effectiveness of the justice system’s contribution to the overall quality of life.

While the idea of measuring the quality of democratic governance started in the late 1970s with the USAID’s Democracy and Governance Index,²³ the most significant shift in the nature of the measurements took place in the 1990s with the failure of the IFI approaches to structural adjustment programmes based entirely on economic restructuring. The IFIs diagnosed the failure not so much as that of economic policies but of the lack of good governance, and thus good governance became the focus of measurement.²⁴ The World Bank’s Governance Index²⁵ serves as a key engine of measurement. The Index gives significance to “voice and accountability”. This is in relation to selection, accountability, and replacement of authorities and interactions among civil society, business players and politics. Next on the Index is regulatory framework and governmental effectiveness, focusing on stability and lack of violence. Control of corruption is also part of the Index as is “the rule of law.” The Index views the rule of law in relation to the respect for institutions and laws.

The IFI’s norm-setting influence is soon realized with other Western donor agencies adopting good governance as a measure for organizing their development cooperation strategies. Of equal significance is the fact that even independent organizations such as the Mo Ibrahim Foundation and Transparency International adopt good governance as the icon of measurement. Thus the Mo Ibrahim Foundation now does category scores for African leadership on safety and the rule of law, participation and human rights, sustainable economic opportunity, and human development,²⁶ while Transparency International focuses on the single governance issue of corruption and produces an annual corruption perception index.²⁷ There are differences in measuring and independent measurers are to be preferred, but as we explore below there is a hegemony of good governance.

The “rule of law” takes its place in the pantheon of indicators as a key measure of good governance. Carothers and other scholars have noted²⁸ a lack of clarity on what the rule of law entails for purposes of measurement. Despite differences in descriptions on what the rule of law means, various characterizations applied to developing countries point to legal and constitutional reform to ensure better compliance with the rule of law. Equally significantly, the rule of law becomes a vehicle for the formalization of a liberal conception of rights and not one linked to social justice. Thus, the World Justice Project, initiated by the American Bar Association but now describing itself as “an independent, non-profit organization, develops communities of opportunity and equity by advancing the rule of law worldwide,”²⁹ defines the rule of law as:

[A] system in which the following four universal principles are upheld:

1. The government and its officials and agents as well as individuals and private entities are accountable under the law.
2. The laws are clear, publicized, stable and just, are applied evenly, and protect fundamental rights, including the security of persons and property.
3. The process by which the laws are enacted, administered and enforced is accessible, fair and efficient.
4. Justice is delivered timely by competent, ethical, and independent representatives and neutrals who are of sufficient number, have adequate resources, and reflect the makeup of the communities they serve.”³⁰

The Project describes these as “four universal principles,” from which it further develops a “Rule of Law Index.” The Project describes the Index as “an innovative quantitative assessment tool.” The focus though is not on justice, but the rule of law. The task is to provide “original data regarding a variety of dimensions of the rule of law, enabling the assessment of a nation’s adherence to the rule of law in practice, identify a nation’s strengths and weaknesses in comparison to similarly situated countries, and track changes over time.³¹ Thus although the name of the project may entail that its focus is on justice, the dominant determining factor is the “rule of law.”

In 2013, the Project launched its report on Malawi in Malawi, witnessed by, among others, the country’s State President. The following table shows Malawi’s scores and rankings on the rule of law, as the determining indicator of justice.

Table 1: Malawi’s Rule of Law Scores and Rankings According to the World Justice Project

Factors	Scores	Global Rankings	Regional Rankings	Income Group Rankings
Limited Government Powers	0.49	65/97	9/18	6/15
Absence of Corruption	0.44	57/97	7/18	3/15
Order and Security	0.69	57/97	4/18	5/15
Fundamental Rights	0.47	81/97	12/18	10/15
Open Government	0.43	68/97	6/18	5/15
Regulatory Enforcement	0.45	65/97	8/18	4/15
Civil Justice	0.59	35/97	3/18	2/15
Criminal Justice	0.45	58/97	7/18	5/15

Source: World Justice Project, http, accessed on 24^thSeptember, 2013

Since the publication of its 2012 results, the World Justice Project has added “informal justice” as another factor, although there were no indicators yet for Malawi by 2013.

Informal justice, alongside criminal justice and civil justice, are of direct relevance to justice on an index that is really about the rule of law. Even in the case of these three types of justice, the premise is the essence of the rule of law as understood by the World Justice Project. Thus the Project deems civil justice important because in “ a rule of law society, ordinary people should be able to resolve their grievances and obtain remedies in conformity with fundamental rights through formal institutions of justice in a peaceful and effective manner, rather than resorting to violence or self-help.”³² Similarly, criminal justice is important from such a rule of law perspective because an “effective criminal justice system is a key aspect of the rule of law, as it constitutes the natural mechanism to redress grievances and bring action against individuals for offenses against society.”³³ The World Justice Project recognizes that informal “systems often play a large role in cultures where formal legal institutions fail to provide effective remedies for large segments of the population or when formal institutions are perceived as foreign, corrupt, and ineffective.” However, while “recognizing the importance of these informal systems, a necessary element of the rule of law is that informal systems are effective, impartial, and protect fundamental rights, and are held to the same standards of fairness in resolving disputes as formal systems.”³⁴

The World Justice Project has not been alone in attempting to proffer a “global approach” in measuring justice but focusing only on the rule of law. A similar assumption, the image of justice as reflected in perceptions, is further exemplified by the Rule of Law Index (Rolx), an initiative that commenced in 2008. The result has been “UN Rule of Law indicators.”³⁵ Launched in June 2008, the project is a joint initiative of the Vera Institute, certain UN agencies, and the World Bank. The UN agencies involved are the United Nations Department of Peacekeeping Operations and the Office of the United Nations High Commissioner for Human Rights, in cooperation with the Department of Political Affairs, the Office of Legal Affairs, the United Nations Children’s Fund, the United Nations Development Program, the United Nations Development Fund for Women, the United Nations High Commissioner for Refugees, and the United Nations Office on Drugs and Crime.The Rolx was “tested” and was, by 2013, in implementation in three economically poor countries; Haiti, Liberia and South Sudan.³⁶ The UN Rule of Law Indicators now come with an implementation guide and tools.³⁷

Despite similarities in attempting to measure the rule of law, differences in approaches are apparent when attempts are made to measure the quality and incidents of the justice system as a key component of the rule of law. The differences are between technicist and institution-based as opposed to people-centred approaches; and between quantitative and qualitative methodologies. The work of the European Commission on the Efficiency of Justice, CEPEJ, exemplifies a technicist and institution-based approach focusing on the performance of judicial and similar institutions.³⁸ CEPEJ’s ‘ diagnostic’ focus areas are the judicial budget, legal aid, courts, judicial staff, lawyers, the users of courts, information, technology, equipment and the courts, treatment of criminal and civil cases, length of proceedings, and enforcement of court’s decisions.³⁹ In contrast, on the same subject, the Swedish Ministry of Foreign Affairs, while essentially institutional, is more value-based. It lists judicial independence, accountability, control, efficacy/efficiency, access to justice, and legal certainty⁴⁰ as areas for focus.

Many organizations, especially development agencies, have attempted to determine the attributes of a justice system or legal system, define respective variables, and agree on indicators. DFID⁴¹, OECD,⁴² SMFA⁴³, sida,⁴⁴ UNDP⁴⁵ and the World Bank⁴⁶ have all issued guidelines on justice-related “diagnostics.” However, there is no common list of attributes, let alone consensus on sub-components, mainly because focus areas tend to correlate with the donors’ mandates. With regard to justice, the commonest attribute among donors and other development agencies seem to be access to justice, an apparently people-centred criterion.

However, people-centredness can be measured in as an official top-down a fashion as other institution-based criteria on other aspects of the rule of law. It is not hard to add to Table 2 more sub-components. For example, how justice duty bearers understand their roles and duties and the existence and implementation of a justice sector policy could fit in any of the two columns against each institution.

Table 2: Sub-Components on Access /Accessibility of Justice for DFID, SMFA and UNDP

Donor	Entry Point Subcomponent or Elements of Focus Area	Indicators
DFID	Political independence of courts Just laws Procedural rules Robust institutions Appropriate and accessible legal service Legally accountable government Mediating civil society organisations legally literate citizens	[Guidelines deal with examples and the setting of performance indicators]
Spanish Ministry of Foreign Affairs	Population’s level of information as to rights Population’s level of trust in justice Existence of organisations giving instruction in rights related matters Existence of organisations that offer free legal services	Percentage of population, broken down by social category, who replies that they know what their rights are and how to claim them. Percentage of indicted individuals that do not have legal representation in criminal trials. Unit of measurement: interval Percentage of illiterate population.
UNDP	Legal protection Legal awareness Legal aid and counsel Adjudication Enforcement Civil society and parliamentary oversight	[Guidelines deal with examples and the setting of performance indicators]

Table 2 underlines that in measuring justice among donors or development partners, the notion of justice is assumed to be formal legal justice and the notion of rights as formal legal human rights. Thus even under the slightly more people-centred approaches, the consultation of people is based on such criteria. The peoples’ own approach to justice, the informal justice system, has for long been routinely ignored, except as subjects for reform. In addition, there is little consideration of the factors affecting the right holder’s capacity to demand justice, including an assessment of the right holder’s skills and access to organisational resources.

4. THE METHODOLOGIES AND METHODS OF MEASURING JUSTICE

Much as there is variety on what to measure with regard to justice, there are no set methodologies, methods, and tools on how to do it. Indeed most of those who have issued guidelines do not indicate the methodologies and methods to use in the collection and analysis of data. The World Bank, for example, states:

The choice of methodologies will largely be shaped by the design of the assessment, particularly where the assessment’s length, scope, and cost are predetermined. Most assessments will rely on a variety of data sources (1) because no single one will provide all the information needed, and (2) because in combination, they can be used to corroborate findings and check against the biases inherent to any single method.⁴⁷

In practice, institutions specialising in the assessment of justice-related matters have adopted both qualitative and quantitative methodologies. The key informant interview method has been dominant in the qualitative research paradigm. Certain agencies have used in-house experts, academic advisers, and local correspondents.⁴⁸ Occasionally, focus group discussions are adopted. Documentary reviews are common both to the qualitative and quantitative paradigms.⁴⁹ Within quantitative methodology, the use of surveys has been dominant, particularly by those agencies that focus on people’s perceptions.⁵⁰ In Malawi, a perception survey on human rights and justice-related matters was completed in 2006.⁵¹ In 2011, this was updated, to support the formulation and implementation of a particular project, the Democracy Consolidation Programme⁵². With UNDP and other donors and development agencies pushing for the establishment of sectors on governance in Malawi, another survey on democratic governance was completed in the same year,⁵³ with the two processes proceeding independently.

Checklists and lists of questions, as Carothers has noted,⁵⁴ have been the usual tools for data collection and analysis. For example, AfriMAP’s checklist on the “rule of law and the justice sector”⁵⁵ is an attempt to collect information on a wide range of issues connected with justice. In this regard, Kanyongolo completed one report for AfriMAP and the Open Society in 2006.⁵⁶ On the other hand, CEELI has developed indexes specific to judicial reform, legal education and reform, legal profession reform, and prosecutorial reform.⁵⁷ The UN Rule of Law Indicators implementation guide has lengthy questions mainly on the attributes of the judiciary, the police, the prisons, and other aspects of what is increasingly known in development programming circles as a sector about justice.

Once an indicator is established, an underlying methodological problem is what type of evidence is considered appropriate to determine the application of the indicator in a specific context. The Vera Institute study has this interesting guidance about the quality of evidence to determine equal access to justice; preferring Box 2 to Box 1:

Box 1: A Basket of Indicators Used to Measure a Single Concept

Aim: Equal access to justice across economic divides Indicator 1: Number of new courts opened in rural and urban areas with concentrations of marginalized populations Indicator 2: Number of courts per 100,000 residents Indicator 3: Percentage of citizens who say that they have access to court systems to resolve disputes Indicator 4: Percentage of accused not represented at trial

Box 2: A Better Balanced Basket

Aim: Equal access to justice across economic divides Indicator 1: Percentage of citizens who say that they have access to court systems to resolve disputes, disaggregated by gender, ethnicity, region, and level of urbanization Indicator 2: Percentage of accused persons legally represented at one or more court appearances in their cases, disaggregated as above Indicator 3: Percentage of citizens who say that the police will respond to them without requiring a bribe if called to resolve a dispute, disaggregated as above Indicator 4: Ratio of prosecution caseloads in courts serving wealthier communities to those in courts serving marginalized communities

While it is obvious that Box 2 provides a better quality of evidence, it still raises questions as to the methods used to determine the evidence and, more significantly, the sampling methods and criteria. For example, the following questions would appear relevant: What are the class and geographical (town village) divides? Should one include corporations and citizens? Should one include foreign corporations on major contracts who often opt out of the system all together by insisting on international arbitration in their agreements? Alternatively, would more qualitative approaches involving deeper case studies provide better pictures of justice or injustice?

A key methodological problem is that the very notion of ‘indicators’ promotes a link to quantitative methods. However, such counting ensures that what people themselves consider most significant may not be countable. More significantly, counting is not holistic enough to enable a proper assessment of issues from peoples’ own world views.

However, as the weaknesses of traditional methodologies became apparent, there has been, more recently, a greater recourse to ethnographic qualitative methods. This has come in a period of increasing dissatisfaction with existing methods. These criticisms may be having some effect. Therefore the Justice for the Poor approach of the World Bank⁵⁸ and UNDPs justice for all⁵⁹ are both moving from emphasis on formal to informal systems with a greater emphasis on ethnographic studies. In a very interesting discussion paper, Sage and Woolcock of the World Bank would go even further in promoting grounded approaches with anthropological sensibility which depart from one size fits all indicatorology. After an incisive critique of traditional methodologies, they suggest:

The alternative we propose is one self-consciously (and, we hope, confidently) grounded in a social theory of local level transformations and the modernization of social relations, combined with an anthropological sensibility with respect to the social construction of rules systems, both formal and informal.⁶⁰

For example, the interesting innovative approach of the UNDP’s Justice for All does involve a deep qualitative study.⁶¹ However, it still starts by establishing, first, a top-down ‘Normative Framework’ which is then evaluated in terms of the legal awareness of this framework among the people; whether there is access to an appropriate forum, whether the grievance is effectively handled, and whether a satisfactory remedy is obtained. In contrast, Bedner and Vel’s ROLAX framework is a deeper anthropological approach which commences by getting the users to define the key issues in access to justice, considering the forum which the user wishes to select, and the effective handling of the grievance.⁶² Furthermore, it takes the issue of the remedy further by considering not only whether the remedy is effective, but the extent to which it is effectively implemented and leads to social consequences for the community. It is thus deeply embedded in user perspectives in defining the issues, the forums and the result. Thus, behind the apparently similarly progressive methodologies, there are two different images of justice being constructed. The former imagines and defines justice in terms of the institutional needs of the global institution and the latter attempts to explore it in terms of the peoples’ own imagination.

5. A VIRTUOUS HEGEMONY OF GOOD GOVERNANCE?

A number of issues emerge from the examination of the various approaches. The first is the variety of the focus of the indicators. To what extent does the variety indicate a conflict of approaches? Is there one correct approach? Participants at the Harvard Workshop⁶³ considered that there was “common ground” in these indicators, and, in fact, a great deal of complementarily. The commonality may be the hegemonic umbrella of ‘good governance’. According to the World Bank, this encompasses ideas of transparency, accountability, opportunity, empowerment security and efficiency. In such an emphasis, good governance is so pervading that it is not far-fetched to conclude that there is an assumption that justice will ensue if the rule of law and the other key governance indicators are deemed acceptable.

Malawi illustrates the obsession to measure good governance. The country is, in general, thought to have a liberal democratic constitution, which regards the rule of law, accountability, and transparency as its underlying principles. Despite such a favourable constitutional framework, however, there are still significant governance challenges, according to the dominant indicators on governance. According to such indicators, levels of accountability and responsiveness by government and other public functionaries are low. Over 8 in every 10, 81% of people, rate traditional structures as the most effective compared to formal institutions. For example, a 2011 DCP baseline survey to update that undertaken in 2006 found that 67.5% of people still turned to themselves to access services. The Democratic Accountability Survey⁶⁴ found that the Proportion of the public who are able to demand their rightsmany times or at least once was 4% in both instances.The proportion of people understanding the link between human rights and service delivery such as education, housing, health, water and food are low, respectively at 65.9, 47.4, 64.3, 63, 58.7 The proportions of people indicating improvements regarding access to food, shelter, infrastructure, education, health services, basic resources, employment show significant disparities among these aspects of the right to development, respectively at 54.8, 45.2, 45.7, 44, 35.5, 31.9, 15. No wonder then that the World Bank’s indicators on good governance for the country, viewed over times, show fragility in good governance. On a scale of -2.5 to +2.5, Malawi has scored below 0 on all the indicators except for political stability between 2004 and 2006, as the Figure 1 shows.⁶⁵ If the indicators are valid, they show a pattern that underlines that good governance has been poor and fragile in Malawi, at least between 1998 and 2010. This observation can lead to many troubling questions, of which none have been answered by experts who measure justice: Since good governance by the State has been poor in Malawi’ s case, why have there not been processes, until most recently, to facilitate justice with regard to such State failure?

The second issue an assumption of the relationship between justice and the rule of law. Indeed, in some cases, there is a conflation of the rule of law with justice. An argument by two members of the Program in Criminal Justice Policy & Management at the Harvard Kennedy School, Foglesong and Stone,⁶⁶ illustrate such a conflation. In response to critics that justice cannot be measured by an index or an indicator, the two experts insist that such is the case only regarding “justice with a capital J, but not necessarily of its component parts; and it is on those component parts of justice that reformers must work.” The very next sentence in their article, however, is revelatory of the tendency to conflate justice with the rule of law: “In each individual country, province, and city, reformers must strengthen the rule of law one part at a time.” A key aspect of such measurement is access to justice, as exemplified by the instructional manual by key donors such as DFID⁶⁷, OECD,⁶⁸ SMFA⁶⁹, sida,⁷⁰ UNDP and the EU⁷¹ ; and the expert institutions on measuring justice, such as the World Justice Project⁷² and the Vera Institute of Justice.⁷³ Indeed, linked to the obsession with good governance, the often unquestioned assumption is that good governance entails advancement of the rule of law, which in turn assures a justice system that delivers justice since access to justice will be guaranteed. Figure 2 depicts such an assumed virtue.

The third issue, from a global framework of a governmentality perspective, is the very virtuous hegemony of good governance over justice. The contemporary concept of good governance is a child of IFIs, which has become part of the global framework of governmentality involving both international institutions, national governments - as donors and donees - and NGOs. The link between the principle of good governance and its indicatorology emerges out of the arcane internal politics of the World Bank. Good governance has been the project of a group of lawyers within the Bank as an amelioration of the apparent problems of the bare structural adjustment strategy previously favoured by the Bank’s economists. As Sarfaty contends, the choice of an instrumental approach to human rights and justice has two values⁷⁴. Firstly, the new ideology is seen not as an end in itself but a functional one couched in terms of its impact on development. The second value is to get the economists on side. These dual aims naturally lead to indicatorology, for example, of the Bank’s Measuring Justice and Human Rights Indicator Project as a way of better producing an effective way of testing the impact of good governance policies. Sarfaty has this interesting quote from a World Bank lawyer:

The mainstream way of convincing and persuading people is an economistic way of seeing things. Unfortunately, all the other disciplines, like social development for example, are forced to use that language to make their case. And I think that’s unfair in a way. It’s raising the bar for their arguments to be accepted. [But] that’s really the dominant way of doing business [at the Bank].”⁷⁵

However, the converse of the desire to please the economists is that “the Bank economists have successfully managed to tame the unruly governance agenda, placing it firmly at the service of the Washington Consensus”< /span>⁷⁶. The result is that indicators lead to the construction of an image of good governance, human rights and justice which is subsumed by the dominant concerns of the economists. However, such an image does not stay confined within the portals of the Bank and the IMF. As the history of the IFIs indicates, a key function of the IFIs has been to globalize its ideology through model-mongers and missionaries⁷⁷ or “an emerging class of global experts with highly internationalized training (usually American) who claim to possess a universally applicable variety of expertise”.⁷⁸

Dissemination of the new ideology takes place in a variety of ways, but the most important principle of the new governmentality is that it should be internalized by the subjects as the new image of justice.⁷⁹ Thus there are conferences, workshops, seminars, courses all devoted to good governance and justice reform.⁸⁰ The most significant innovation is the Bank’s websites on Poverty, Good Governance and Law and Justice Institutions.⁸¹ The web pages with their wealth of material become the source of first resort even for those critical of the ideology. The sites’ interactive spaces provide a further dimension of engagement. It is thus not surprising that other organizations, governmental or independent, take their cue from the dominant discourse of governance and its accompanying indicatorology.⁸²

For governments, good governance becomes the new conditionality which they have to follow. Yet, the shift from Structural Adjustment to Poverty Reduction Strategy Papers involves that the governments and civil society ‘own’ the new images of good governance and justice and thus participate in their internalization through their own dissemination strategies. Indicators thus become the new signifiers of where each state stands in the pantheon of good governance and justice, enforced by the stick of reduction in aid funding, not just from the Bank and IMF, but from a variety of sources. For example, several governments, such as those of Zimbabwe, Uganda, and Malawi have had aid reduced or suspended because of declining indicators on justice and good governance. In the case of Malawi, a once donor-likeable regime of Bingu wa Mutharika eventually had aid withdrawn by various donors including the USA, the UK, the IMF and the World Bank because of perceived poor performance in good governance and the rule of law.

6. THE DILEMMAS OF MEASURING

A number of problems arise from the measuring of justice. Firstly, the very institutionalization of ideas such as human rights and justice by global institutions distances them from their traditional role of being spaces of emancipation and hope. Secondly, what is being measured is the ability of the local institutions to deliver justice, good governance and human rights. This assumes an entirely benevolent role for the global institutions at whose behest the measuring is being done. As Frost suggests “[i]t goes without saying that philosophical discourses about global justice have to start from and respond to the reality of global injustice”. ⁸³

Thus, let us take the apparently laudable aims of the concept of the World Justice Project with its emphasis on accountability of government officials, clear and fair laws protecting fundamental rights of persons and property, fair process of enactment and administration of law and access to justice. In the context of the controversy over peoples’ access to HIV/AIDS medicines in South Africa, in the period prior to the Doha Declaration, it could be argued that all four criteria were applicable in South Africa. However, many people had no right of access to medicine because the intellectual property law protected the rights of patent holders to their property. The main reason why an alternative result came about was that a mass of protests at international level argued against the fundamental injustice of international laws (the TRIPS Agreement) which protected patent holders at the expense of the victims of HIV/AIDS.

Pogge argues that an underlying reason for the ignoring of such injustice claims is that the debate about global injustice is cast in terms of charity, with the issue being whether the affluent countries have or do not have an obligation to help developing countries. He suggests:

Both sides easily take for granted that it is as potential helpers that we are morally related to the starving abroad. This is true, of course. But the debate ignores that we are also and more significantly related to them as supporters of, and beneficiaries from, a global institutional order that substantially contributes to their destitution.⁸⁴

For Pogge, everyone, including global institutions, developed and developing country governments, corporations, and citizens are part of the system which causes the poverty of injustice.⁸⁵ The significance of this approach, which is based on a negative human rights obligation, is that there is a strong accountability on the part of those who fail to fulfill the obligation. Yet, all the indicator systems discussed above have tended to ignore such accountability.

Baxi reinforces these assertions through another level of criticism of contemporary approaches to the improvement of justice. For him the existing discourses of improving human rights and justice camouflage the underlying promotion of “trade related market friendly human rights”⁸⁶. That is, what is involved ultimately is the promotion of a global market place which is attuned to the justice and human rights needs of corporate global interests, rather than those of social justice:

The protection of rights of foreign investors is to be of such a high order as to deconstruct all traditional and newly emergent human rights as ‘trade distorting’ policy obstacles that need to be overcome.⁸⁷

The significance of this structure is that it enables the domination of the discourse of justice, governance and rule of law by the same institutions to which Pogge and Baxi attribute responsibility for the creation of injustice. There is an added significance to this domination in that it is the language and methodologies of the dominant institutions which determines the nature, fulfillment and non-fulfillment of duties and hence their measuring. In fact, as we have seen above, Morse has attested to the power that indicators wield in promoting a governmentality which involves “rule by experts”.⁸⁸

Of course, the positions of Pogge and Baxi come up against the currently dominant capabilities approach of Sen.⁸⁹ Vizard suggests that Pogge’s idea of breach of a strong negative duty by the international institutional order is highly contestable. For her, Sen’s conception of a positive duty based on enhancing capabilities provides a greater basis for a rights discourse. She further asserts that while such duties involve an imperfect obligation, these become justiciable by applying the standard of reasonableness as indicated in a number of South African constitutional cases.⁹⁰

However, whether we commence with Pogge’s strongly interpreted negative duty not to impoverish or with Sen’s duties to reasonably promote human capabilities, the important consideration is the global responsibilities of governmental, non-governmental and market institutions alike. It is this aspect, the way in which institutional mis-measuring can promote injustice, that is more significant than the current debate between the Rawlsian “Social Primary Goods” approach and the Senian “Capabilities” approach to measuring justice.⁹¹ In this respect, Pogge’s concern with global institutional and personal responsibility for injustices and Baxi’s insistence on the process being one of ‘impoverishment’ are stronger criteria than more ‘ charitable’ approaches which conform with the global institutional rhetoric of ameliorating ‘poverty’. It is therefore important to consider how the actions and operations of these institutions can be involved in any system of accountability which is constituted by justice indicators.

Of course, there is significant movement within international organisations towards greater responsiveness to people, both at the substantive level with an emphasis on ‘poverty reduction’ on the part of the IFIs and a greater concern with local ownership through an emphasis on consultation. Ngaire Woods gives an example of the World Bank Inspection Panel process which can hear concerns raised about World Bank projects and policies directly from people affected by them. ⁹² As we have noted, organizations, such as the World Bank and the UNDP, have begun to take greater notice of the institutions which affect the people themselves, such as informal justice systems.

However, this is precisely, why the methodology and methods used for measuring justice become significant. Indicatorology/ism as a form of concession to neo-liberal economists becomes a significant obstacle to justice. There is vigorous critique of both the substance and the processes involved in the promotion of good governance mechanisms of opportunity, empowerment and security in which justice plays a significant role.⁹³ As Craig and Porter suggest, in relation to the World Bank Poverty Reduction Strategy Papers, the main engine of good governance:

We should not be blinded by the intentions, or even the practices: for as ever, the most powerful drivers of real opportunity, empowerment and security exist not primarily in Development’s institutions and programming but elsewhere in the more powerful structures and changing relationships of political economy: markets and societies, markets and territories, markets and capital.⁹⁴

7. ALTERNATIVES?

In this context, it is necessary to consider the relevance of alternative approaches to social and legal justice and accountability mechanisms. A significant alternative is provided by what Santos has termed counter-hegemonic or subaltern approaches⁹⁵. An aspect of these approaches is to measure justice or rather injustice from the perspective of the subaltern; those who are at the bottom of the hierarchy either as “subordinate nation-states, regions, classes or social groups”.⁹⁶

Therefore, while, in principle, both the mainstream approach and the counter-hegemonic approach are attempting to focus on the needs of the excluded, they emerge from very different political and strategic dimensions. While some measures of justice are likely to be common to both, the fundamental differences mean that both the indicators and the ways of measuring those indicators might be different. From this perspective, the following points can be stressed:

1. We should be mainly concerned with the perspectives of the oppressed and impoverished. In a different context, subaltern scholarship has been rooted in studies of struggles against injustice, as exemplified by groups within and without the World Social Forum.⁹⁷ Studies should be concerned as much with the successes and failures of these as with more formal institutions.
2. Such an approach to (in)justice indicators would therefore be not only of the delivery of justice but also the production of injustice from the perspective of the subordinated. Such an approach would not take for granted, as is the case with most systems of indicators, the virtuous relationship between the global norms such as the Human Development Index and the Millennium Development Goals on the one hand, and systems of good governance, accountability, opportunity, empowerment and security on the other. It would commence by ask
3. The key difference is a focus on impoverishment rather than poverty, on injustice rather than justice. One should not take for granted that better formal institutions will necessarily produce more justice or reduce poverty. As we have seen, justice can be the matrix of injustice in many cases.
4. ing questions of each of these to the extent that they produce justice or injustice. For example, do global human rights norms promote market-friendly human rights? They would enable questioning of the way in which global and national intellectual property rights regimes promote social injustice. They would consider the ways in which national instruments such as Constitutions and laws promote justice or injustice both in their normative frame and in their application. They would consider the nature of public duties created under these institutions (as we have done in our preliminary project); the production of justice or injustice through their normative and performance structures; and whether global corporate structures and governance structures exist in ways which are beyond the purview of juridical institutions.
5. Equally significantly, the indicators described above might be seen to give too much weight to public institutions and insufficient to plural institutions of governance and law. For example, it would be pertinent to ask whether plural forms of justice in the village or townships produce justice or injustice. How do they relate to formal justice institutions?
6. Indicators would need to take sufficient cognizance of the inter-relationship between legal and other institutions in the production of justice or injustice. In particular, there would need to be an awareness of the limited effectiveness of law (whether formal or informal) in relation to broader social economic factors.
7. Thus, indicators need to be seen as part of an interconnected framework measuring the delivery of justice or rather the seeping of injustice within a network of institutions.

8. CONCLUSION

These points lead us to a final dilemma. Indicators are intrinsically linked to quantification and quantification may have the Aristotelian tendency to mis-measure as aptly illustrated by Shakespeare. One approach would be to avoid quantification altogether. But perhaps the lesson of Shakespeare is that measures have to be measured holistically in terms of where they fit in the real world of (in)justice or the lived experiences of the people. In this respect, methodological primacy has to be given not to the quantitative but to the qualitative, not to quick interviews and expert opinions, but to detailed engagement with real lives and reall struggles.

In this sense, the ongoing research between the Warwick’s School of Law and the University of Malawi’s Faculty of Law may at least raise questions not only about the production of injustice or justice, but also about how to or not to measure justice or injustice. In this sense, contrary to the claims of innovations in measuring justice, such research finds such claims wanting at least in so far as they still fail to show who produces injustice and how that injustice affects the quality of life of the impoverished.