This material was initially prepared in 2010 as part of a background document for developing a national human rights report for Ethiopia. Its publication here is intended to serve as an input for individuals and groups interested in preparing a human rights monitoring report as well as informing discussion on the assessment of existing or future human rights monitoring reports. God willing, I hope to follow it up with a brief assessment on implementing access to justice in Ethiopia as per this framework.

Introduction

The conceptual and methodological framework for human rights monitoring should be informed by:

– best experience among international, regional and national human rights organizations with particular attention to treaty monitoring bodies, and National Human Rights Institutions;

– the international and regional human rights normative framework;

– the national human rights framework (i.e., the FDRE Constitution, the UDHR, and other international human rights instruments duly ratified by Ethiopia); and,

– the mandates of the body seeking to undertake the monitoring initiative (as defined under its establishment proclamation if it is a public body or its organizational objectives if it be a non-state actor) as interpreted by its high level management.

Accordingly, the author has conducted a review of relevant literature, legislation and practice on the basis, nature, structure and scope of monitoring by a range of actors, and human rights monitoring approaches applied by international, regional and national human rights institutions.

 

2  Conception of Monitoring

2.1  Meaning and Purposes

Monitoring means the close observation of a situation or individual case over a long period of time, with reference to accepted norms, with the purpose of providing an assessment as basis for further action. The following elements constitute monitoring:[2]

• It is carried out over an extended period of time.
• It involves collecting or receiving a large quantity of data.
• Close observation of the situation is done through constant or periodic examination or investigation and documentation of developments.
• Standards or norms are used as reference in objectively assessing the situation or case in question, especially in determining what is wrong with it.
• Tools or instruments are used in identifying how the situation compares with established standards or norms.
• The product of monitoring is usually a report about the situation.
• The report embodies an assessment of the situation which provides a basis for further action.

 

The most common general purpose of monitoring is to be able to pinpoint what is wrong with a situation or a case and to indicate what steps can be taken to remedy it. Monitoring is also undertaken to see whether steps that have been taken to improve a situation are working. Human rights monitoring has the following particular purposes, among others:

– To ensure compliance with international and domestic human rights law by government authorities and citizens;

– To provide remedies for the victims of human rights violations and address impunity for human rights abuses by collecting evidentiary material for court cases, investigations by national human rights institutions, etc …

– To identify patterns of human rights abuses and violations in terms of the types, frequency, and causes of human rights violation with a view to systemic solutions for addressing them;

– To inform and educate the public about human rights situations and ensure transparency for government and individual actions by establishing the human rights situation in a particular context thorough documentation; and,

– To offer validation to victims of human rights violations by amplifying the voices of victims and providing opportunities for those voices to be heard.

While sharing similar purposes, monitoring is distinct from investigation and documentation of human rights abuses. Monitoring involves the repeated collection of information often involving investigating and documenting a large or representative number of human rights events. Investigation, on the other hand, refers to the process of fact finding focused on an event which carries or is suspected to carry one or more human rights violations. The final stage in human rights investigations is documentation or the systematic recording of the results of the investigation as a basis for advocacy or comparison. Data documented over a period of time and covering a large number of specific cases can be analyzed so as to get a fuller picture of the human rights situation in the context of a monitoring process.

2.2  Human Rights Monitoring Bodies

Monitoring may be conducted by a wide profile of human rights actors among which three actors, namely inter-governmental bodies, NGOs and government organizations – especially national human rights institutions, take important roles.

Intergovernmental Organizations (IGOs)	- Treaty monitoring bodies; - The Commission on Human Rights - Special Rapporteurs and Working Groups, - Specialized agencies (e.g. ILO, WHO, UNDP, …), - Regional IGOs, and - The Sub-Commission on the Protection and Promotion of Human Rights	- Set standards - Monitor compliance of governments with their treaty obligations - Monitor certain situations involving violations
Governmental Organizations (GOs)	-Government agencies or sector ministries responsible for treaty-based reports, -National human rights institutions, -Policy monitoring executive bodies, -Specialized commissions/ agencies (e.g. anti-corruption commissions)	-Encourage own governments to adopt international standards -Monitor compliance of own governments with treaty obligations -Monitor violations
Non-Governmental Organizations (NGOs)	-International advocacy groups and organizations, -National human rights NGOs	-Lobby with IGOs toward setting standards -Lobby with governments toward adopting international standards -Monitor compliance of governments with their treaty obligations -Monitor violations

A national human rights institution is usually any of the following:

Human rights commission – a body composed of several members usually from diverse backgrounds which discharge various functions (usually different from one commission to another) that range from human rights education to investigation of complaints.

Ombudsman – an individual or group of persons generally appointed to protect the rights of individuals who believe themselves to be the victim of unjust acts on the part of the government

Specialized commission – a body composed of several members which function to protect the rights of a particular vulnerable group such as ethnic and linguistic minorities, indigenous populations, children, refugees or women.

National human rights commissions are generally mandated to study international standards, encourage their governments to adopt these, and call the attention of their governments if the adopted standards are not met. One way through which they monitor the compliance of their governments with obligations is by making contributions to the required periodic reports.

Another common duty of national human rights commissions as well as specialized commissions is to call the attention of the government to areas of violations and instances of discrimination in the country. Many human rights commissions, either national or specialized, and certainly the national ombudsmen, are mandated to monitor violations, especially if a complaints procedure is in place.

2.3      Approaches to Human Rights Monitoring

The approaches adopted by various actors in monitoring human rights may differ as a factor of what is monitored, thematic scope or focus, and the intended purposes.

Situation vs. Performance of Duties: The perspective adopted by a monitoring initiative may fall into one of three general categories: a situation monitoring; or a duty-bearer analysis. A situation report seeks to monitor progress in the realization of human rights, i.e., whether and the extent to which the rights are enjoyed by the rights-holders. As such, the focus is on the status of human rights and the situation of vulnerable groups. While such reports abound at the national level, the reports prepared under the Universal Periodic Review (UPR) mechanism within the framework of the UN Human Rights Council are also a good example. On the other hand, a duty-bearer analysis report, such as most of the treaty-based reports, monitors the fulfillment of obligations to realize human rights. Such a process thus focuses on mapping human rights actors, and examining actions taken by the State and other legal and moral duty-bearers to realize human rights. In some cases, these two perspectives may come together in a multi-perspective monitoring report dealing with the status of rights, situation of vulnerable groups and fulfillment of legal/moral duties by the duty-bearers.

Comprehensive vs. Specialized: Human rights monitoring processes and reports are also different in terms of the range of issues/rights to be covered. Some reports comprehensively cover the whole range of rights while others opt for a more in-depth coverage of selected thematic issues/rights. Most national human rights reports, however, have an overview section dealing with the whole range of rights/issues as well as specific sections for more in-depth discussion of selected issues/rights.

Situation vs. Case Monitoring: Human rights monitoring can be of two general kinds, depending on their focus: situation monitoring and case monitoring. Under each kind, there can be various forms, as summarized below:

Situation monitoring	-monitoring human rights violations -monitoring the drafting and passing of legislation -monitoring the implementation of laws and policies -monitoring the establishment and progress of human rights institutions
Case monitoring	-monitoring the legal process undergone by a case -monitoring relief and rehabilitation services provided to a client -monitoring other forms of intervention in a case

Situation monitoringfocuses on a situation in general in terms of the recurrence of violations, progress in relevant human rights legislation and the performance of human rights institutions. This form of monitoring is useful for the purpose of monitoring government compliance with treaty obligations as well as for domestic monitoring. Case monitoring, on the other hand, is very focused and victim-oriented and involves work for or on behalf of an individual victim or a group of victims. Follow up and documentation of developments in the case is an essential and integral part of case monitoring.

3     Human Rights Monitoring Methodologies

Two dominant methodologies in monitoring human rights situations are the "events" (or acts-based) methodology and the indicators-based methodology.[9]

The “events methodology” for monitoring involves identifying the various acts of commission and omission that constitute or lead to human rights violations. In other words, it is a concrete form by which the “violations” approach takes shape. This methodology involves investigating and documenting an event that is suspected of or confirmed to be consisting of one or more acts considered as violations.

A limitation of the “events” methodology is that it usually does not aim, or often fails, to arrive at a complete picture by giving the total number of violations, much less the proportion of actual victims to the whole population. There are two problem areas identified in this regard:

–      The monitoring body does not hear of all events involving the violations covered by its mandate; and,

–      Even if the monitoring body learns of events that are likely to contain violations, it is unable to investigate and document these for reasons such as ongoing military actions, hesitation of witnesses to come forward, and lack of resources.

Indicator based human rights monitoring, on the other hand, involves the use of performance standards for the core components of specific rights in the form of indicators and benchmarks to determine patterns and trends. The advantages of this methodological approach have been noted in terms of enabling the identification of problems and potential major violations, expressing the magnitude of the problems, comparisons over space, determination of the status of groups within a country, and facilitating the evaluation of trends over time. However, indicators and benchmarks may not be appropriate in addressing grave violations since they tend to aggregate the situation of individuals. Indicators-based methodology is especially weak in situations were victims require direct and individualized assistance.

In short, the combination of the “events” methodology and the indicators-based methodology should result in a comprehensive and detailed picture of a situation.

3.1 Identifying, Selecting and Benchmarking Indicators

An indicator is a tool that shows where something is, what direction it is leading to, and how far it is from that objective. It serves as a sign or symptom that tells what is wrong in a situation and helps in pointing out what needs to be done to fix the problem. A human rights indicator may be defined as

“a piece of information used in measuring the extent to which a legal right is being fulfilled or enjoyed in a given situation.”

Indicators may be based on quantitative or qualitative information and may assess inputs or outputs. Quantitative indicators measure change through numerical or statistical facts of physical outputs. A more accurate description would thus identify indicators as “quantitative or qualitative statements that can be used to describe situations that exist and to measure changes or trends over a period of time”. In this definition, indicators are seen as both qualitative and quantitative descriptions of situations and as elements that are employed to define a particular process.

International, regional, and national human rights institutions have developed widely accepted indicators for a range of rights. At the international level, UN agencies such as UNICEF, UNESCO, and WHO, Special Rapporteurs such as Paul Hunt on the Right to Health, and inter-agency working groups have developed rights-based measurement systems in their respective spheres of expertise. Most notable among these are the detailed CESCR Guidelines,[13] the draft harmonized reporting guidelines, the human development indicators and indices of UNDP, the national context-specific benchmarks under the Common Country Assessment (CCA) framework of UNDAF,  and indicators developed for the Millennium Development Goals (MDGs). Generally, three kinds of rights-based indicators have been identified for human rights monitoring. These are: structural, process and result indicators measuring commitment, efforts and results, respectively.

Structural indicators measure whether or not appropriate legal, regulatory and institutional structures considered necessary or useful for the realization of a human right are in place as an indicator of the extent to which human rights are guaranteed through ratification and adoption of legal instruments and existence of basic institutional mechanisms deemed necessary for facilitating realization of human rights (such as court systems, human rights commissions and ombudsmen, and formal complaint mechanisms). They capture commitments or the intent of the State in undertaking measures for the realization of the concerned human right to monitor the state obligations of conduct, i.e. the effort the government has put forth towards the realization of a human right. Structural indicators have to focus foremost on the nature of domestic law as relevant to the concerned right, i.e. whether it incorporates the international standards, and the institutional mechanisms that promote and protect the standards. Structural indicators also need to look at policy framework and indicated strategies of the State as relevant to the right. Most structural indicators are qualitative in nature, and a number of structural indicators may be evaluated by a simple “yes“ or “no“ answer, e.g. if a particular law or policy is in place or not. However, sometimes these yes/no answers need follow-up questions and additional clarification, to capture qualitative dimensions of the law or policy.

A process indicatoron the other hand measures the degree to which the state is complying with its obligations as well as the effectiveness of the structural realities, the laws and institutions that exist. In most cases, process indicators relate State policy instruments with benchmarks that cumulate into outcome indicators, which in turn can be more directly related to the realization of human rights. Process indicators capture:

– quality of a process in terms of its adherence to the key human rights principles (is the process non-discriminatory, accountable, participatory and empowering, and can duty bearers be held accountable?), and

– type of policy instruments, and public resource allocations and expenditures invested to further the progressive realization of a specific right.

Examples of process indicators include the amount of government spending on female primary education, and the degree of independence of the judicial system.

A result indicatormeasures the outcome of efforts, or the lack of them, to meet a particular obligation and success in bringing about greater equity and wellbeing. It is therefore an indication of the current status of the enjoyment of a certain right and a more direct measure of the realization of rights. Outcome indicators capture attainments that reflect the status of realization of human rights in a given context. For instance, a result indicator may relate to whether and to what extent individuals and groups enjoy freedom of religion and free speech, or an increase in government spending on female primary education actually yield improvements in the literacy rates of girls relative to boys.

 

Salient features of the OHCHR indicators framework

–      A common approach to identifying indicators for monitoring civil and political rights, and economic, social and cultural rights, thereby strengthening the notion of the indivisibility and interdependence of human rights;

– The framework translates the narrative on the normative content of human rights (starting with the related provisions of international human rights instruments and general comments by treaty bodies) into a few characteristic attributes and a configuration of structural, process and outcome indicators.

– The identified indicators bring to the fore an assessment of steps taken by the State party in addressing its obligations – from acceptance of international human rights standards (structural indicators) to efforts being undertaken by the primary duty-bearer, the State, to meet the obligations that flow from the standards (process indicators) and on to the outcomes of those efforts from the perspective of rights-holders (outcome indicators);

– The framework makes it is easier to identify contextually meaningful indicators for universally accepted human rights standards. It seeks neither to prepare a common list of indicators to be applied across all countries irrespective of their social, political and economic development, nor to make a case for building a global measure for cross-country comparisons of the realization of human rights;

– The framework focuses on two categories of indicators and data-generating mechanisms: (a) indicators that are or can be compiled by official statistical systems using statistical surveys and administrative records; and (b) indicators or standardized information more generally compiled by non-governmental sources and human rights organizations focusing on alleged violations reported by victims, witnesses or NGOs; and

– The framework also focuses on quantitative as well as qualitative indicators to assess the implementation of human rights effectively. Efforts have been made to keep the identified indicators simple, based on objective and transparent methodology and, to the extent feasible, there is an emphasis on the disaggregation of identified indicators by type of prohibited discrimination (e.g. sex, ethnicity, disability, etc) and by vulnerable or marginalized population groups.

UN- OHCHR’s work on indicators for human rights assessments,

Status Note August, 2007

Most existing frameworks, such as treaty body guidelines, focus on structural and process-level indicators rather than outcomes due to ease of measurement, and emphasis on government obligations. One visible exception is the Committee on Economic, Social, and Cultural Rights (CESCR) whose Guidelines require states to provide specific outcome level statistics disaggregated by social group, sex, geographical location and other criteria, and supplemented with information about the rates of change over the past five and ten years. However, the principle of interdependence among the whole spectrum of human rights requires that a system of rights-based measurement incorporate indicators for both enforceability (structure and process) and actual rates of enjoyment (outcome). As Craig Mokhiber of OHCHR has noted,

“… simply measuring status, or degree of realization, is not sufficient. There is a need to ensure the existence of an express right, and to monitor and measure effectiveness of institutions and mechanisms of redress and enforcement as well”.

Moreover, the use of structural, process, and outcome level information gives us a more comprehensive picture of the human rights situation as well as a clearer indication of accountability. These would in turn lay the basis for more relevant recommendations to realize human rights.

The various frameworks developed through the above indicated efforts provide us with a choice of indicators. Yet, there are a few more challenges that have to be addressed to ensure the contextual relevance and effectiveness in addressing the objective(s) for which the indicators are to be used. A major general concern in this respect is the fact that almost all available attempts were principally geared towards treaty-based reporting by States; a point that may give us pause in applying the same indicators for a report developed by an independent national human rights institution. Moreover, concerns relating to validity, reliability and transparency in terms of triangulation/cross-confirmation among data sources, capturing the essential aspects of the normative framework, and appropriate methodology have to be addressed. More specifically, the selection of human rights indicators is also necessitated by the following methodological considerations:

-        Simplicity, timeliness and number of indicators: Indicators that are too complex, take too much time to measure or are too numerous would be useless in practice in light of inherent limitations of expertise, time and other resources; after all, indicators are expected to indicate;

-        Objectivity: indicators should be directly observable and independently verifiable based on objective information and data-generating mechanisms following relevant international statistical standards, rather than perceptions, opinions, assessments or judgments made by experts/individuals

-        Comparability: monitoring indicators should be suitable for temporal and spatial comparison since we are interested in measuring changes and differences; and

-        Disaggregation: monitoring indicators should be amenable to disaggregation in terms of sex, age, and other vulnerable or marginalized population segments.

In general, indicators should be produced and disseminated in an independent, impartial and transparent manner and based on sound methodology, procedures and expertise.

Finally, the indicators we have selected have to be benchmarked to make them open to regular monitoring of progress over successive time spans that takes into account the current situation. A benchmarkis the level that is aimed to be met when using a certain indicator. An example of a benchmark, when using adult literacy rate as an indicator, is 75% literacy among adults nation-wide. Unlike human rights indicators which measure human rights observation or enjoyment in absolute terms, benchmarks generally refer to targets established by particular governments in relation to specific rights. The primary use of benchmarks is to offer a tool to assess the performance of states in reaching the goals they have set for a particular interval of time as part of the process of fulfilling their obligations.

3.2      Identification and Selection of Sources of Information

Human rights monitoring reports generally focus on two complementary sources of quantitative data: socio-economic and other administrative statistics; and, events-based data on human rights violations.

Socio-economic statistics refers to quantitative information compiled and disseminated by the State, through its administrative records and statistical surveys, usually in collaboration with national statistical agencies and under the guidance of international and specialized organizations. The use of a standardized methodology in the collection of information, whether it is through census operations, household surveys or through civil registration systems, and usually with high level of reliability and validity, makes indicators based on such a methodology vital for the efforts to bring about greater transparency, credibility and accountability in human rights monitoring.

Events-based data, on the other hand, is information relating to alleged or reported cases of human rights violations, identified victims and perpetrators. The most important sources are cases entertained by national human rights institutions and UN special procedures both of which normally process allegations in a standardized manner. While such data does not provide comprehensive information on violations and precludes comparison over time or across regions, such indicative data has significant potential in terms of elaboration. Other sources of qualitative information used in human rights reports include household perception and opinion surveys and data based on expert judgments.[27]

One should however be cautious and selective in the use of available secondary sources be aware of the possible limitations. At the outset, comprehensive information consistently covering all categories and dimensions of human rights is not available due to reasons attributable to limited objectives, scope and coverage. Moreover, the data sources often manifest problems relating to source biases, validity, reliability, transparency, variance truncation, and aggregation.

Information source bias relates to the availability of information and the possible biases stemming from the type of organizations that produce the information. Most available reports by human rights organizations, having access only to those violations that are reported, are based on reported cases of violation rather than a comprehensive database. Moreover, the very nature of the organizations may entail inherent bias in the way information is collected, compiled, analyzed and reported. For instance, foreign government reports such as those produced by the US State Department and the UK Foreign and Commonwealth Office will necessarily have certain biases and differ from information provided by international governmental and non-governmental organizations. There are also differences in reporting and interpretation among different human rights NGOs, such as Amnesty International, Human Rights Watch and Freedom House as well as between government agencies and CSOs. In relation to narrative and qualitative reports, there will be varying degrees of bias and uncertainty associated with differences in source material, ideological influences, and the fact that there are many incentives not to report human practices and problems accurately or at all.

Validity concerns the degree to which an indicator actually measures what it purports to measure. There may be some ‘distance’ between the category and/or dimension of a particular human right and the indicator that is being used to measure it. Certainly the use of proxy measures runs up against this problem.

Reliability concerns the degree to which the indicator can be produced consistently across different contexts by different groups at different times. Can the indicator be produced by different people using the same coding rules and source material?

Transparency concerns the degree to which the coding rules and procedures for producing an indicator are publicly available. For example, while some human rights data websites are explicit about its coding rules and sources for coding its different indicators, other are less transparent about the sources that are used for each country and how its checklists are used to produce their different scales.

Variance truncation concerns the degree to which information on human rights at the national level is forced into limited categories, such as those found in the standardized scales derived from expert judgments. These standardized scales, which fail to show changes over shorter periods, are only useful for those countries in which there has been great variation in human rights protection over time.

 

Aggregation concerns the ways in which indicators are combined as well as the degree to which they provide information on different groups of people in a country. For example, the HDI, which employs different aggregation and weighting rules for the various components, rarely provides information that helps identify the rights conditions for significant sub-populations within countries.