Nyet! No Tax Dollars for your Project

Well, the reviews are in, and once again the Political Science Panel at the National Science Foundation has declined to fund our project on National Human Rights Institutions.  You can read last year’s rejection and proposal here, and the proposal itself here.

On a more positive note, we are working on the User’s Guide for the Organizational version of the NHRI data, and with luck will get the beta version of that data released by the end of July.  We will present the first substantive paper using those data at The Domestic Politics of International Human Rights Agreements conference this fall at Princeton’s Niehaus Center for Globalization and Governance.

That said, the really cool (and difficult, time consuming) part of the project is the Behavioral data, and that remains stalled in Beta development of coding rules since without substantial  funding we will not be able to code enough data to pursue it on a shoe string (the Organizational data were much more simple).  I have yet to speak to my co-PIs (indeed, I have read only the Panel Summary), so cannot speak for the project, but my personal view is that after two rounds at NSF one punts.

The reviews this time around follow.

Proposal Number: 1330236

NSF Program: Political Science

Principal Investigators: Courtenay Conrad, Jacqueline HR DeMeritt and Will H. Moore

Proposal Title: Collaborative Research: National Human Rights Institutions: Organization, Behavior, and Effects on State Repression
Panel Summary:

The Political Science Advisory Panel extensively discussed the proposal, offering ideas about institutional design in which national human rights organizations are effective. The project develops original data to test and offers new perspectives on existing literature. Despite these positive aspects, panelists noted some concerns:

*Only looking at English language documents. What are the potential sources of bias from looking at English?
*Relying to some degree on states’ own reports of behavior of their own institutions
*Lack of theory of creation of institutions and there is large variation in forms and origins of these institutions
*Possible selection/endogeneity bias in determining impact (i.e., some origins will aid effectiveness). Is the existence of the institution itself informative about cultural/social latent support of human rights protections?
*Confusion about whether proposal is about effectiveness of institutions or why the institutions are adopted.

Broad impact: important information about policy/institutional means to reduce human rights violations. After discussion, panelists reached consensus that the project should be placed in the Not Competitive category. But, panelists encourage the PIs to revise the project with more attention to research design and the selection model, possibly extending the work from effectiveness to treaty adoption.

Panel Recommendation: Not Competitive

 

Review #1

Rating: Very Good

REVIEW:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

The project will speak to the scholarly literature on human rights and human rights violations by governments. The PIs will do so by making linkages to the formal theory literature on political institutions. I think that there is significant room for improvement in the literature on human rights, and I believe that this proposal will make an important contribution to this literature.

1. I believe that the proposed project has a very good chance to advance the knowledge and understanding of how political institution can shape governments’ incentives to violate or protect human rights. The benefits to society of such understanding would be significant.

2. I do not find the project to be especially innovative or creative. It appears to be a fairly standard request for funds to collect data.

3. The plan of action proposed by the PIs makes sense and seems to be well-reasoned. As far as I can tell, there is no obvious plan to assess the success of the project.

4. The team of PIs is absolutely qualified to do this work. They have all written in this research area and have experience with these data and other data collection projects.

5. There are sufficient resources for the PIs to complete this project. I am fully confident that if they are funded they will complete the data collection project in a timely fashion.

Summary Statement

The PIs are requesting funds to support a data collection project. They propose to collect data on factors that measure the activities and behavior of National Human Rights Institutions (NRHIs). They have already collected a pilot sample of the data and are using these data to develop hypotheses about how political institutions shape government behavior. My recommendation is that the project be funded by the
NSF. 

Review #2

Rating: Good
REVIEW:
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

INTELLECTUAL MERIT
The project under review purports to accomplish three related goals with the support of the NSF funds: (1) it aims to collect, organize, and make publically available a new dataset containing original data on the activities and behavior of National Human Rights Institutions (NHRIs); (2) it aspires to generate theoretical/conceptual understanding of the impact of NHRIs on state respect (or the lack of thereof) on individuals’ integrity rights; and (3) using the new data and theoretical/conceptual framework, the project proposes to implement an empirical study about the impact of the organization and behavior of NHRIs upon state repression.

In concur with the Project’s PIs in their assessment of the field of research on NHRIs, which has been dominated by legal, normative, and, largely, explorative scholarship. The proposed study of the impact of NHRIs on state conduct with regard to human rights is a fresh, timely, and promising undertaking. The institution of NHRI is a fairly new phenomenon in the realm of domestic and international human rights institutions; it is not surprising, therefore, that it has been underexplored theoretically and empirically. The project proposed for the NSF funding can advance the understanding of human rights researchers and practitioners about the role that domestic human rights agencies can play in promoting and defending human rights. It can also enhance a broader understanding of the possible impact and mechanisms of influence by parchment institutions on state conduct.

Below I comment on the many strengths but also some weaknesses of the proposal for illuminating its intellectual merit and suggesting ways for strengthening the proposed research.

On the one hand, by building on assumptions and expectations of rational choice institutionalism to theorize the effectiveness of NHRIs, the project purports to make specific contributions to the growing rational choice institutions literature.

The application of the time-tested rational choice institutionalist perspectives that have been central not only to the field of comparative politics but to the political science as a whole, helps to illuminate the mechanisms of NHRIs’ impact on state repression. In this way, the project promises to enhance our understanding of the pathways in which institutions shape state behavior and facilitate social change. On the other hand, born out of the study of American congressional behavior, rational choice institutionalism has been rightfully criticized for its ethno-centric focus and Western bias in ways it conceives of authority (as legal-rational) and relationship between states and citizens (as based on the ‘social contract’), among other things. A growing body of comparative research (including comparative studies of human rights) on Africa, Asia, Latin America, and post-Communist Eurasia suggests that in many parts of the world, particularly where human rights violations are still prevalent, the ‘rules of the game’ that structure political life are informal. Much current literature on institutions and human rights assumes that actors’ incentives and expectations are shaped primarily by formal rules (constitutions, judicial systems, elections, and other formal institutional arrangements). Such a narrow focus risks missing much of what drives political behavior of states that do not square well with the Weberean notion of a ‘modern’ state.

This criticism does not imply that the PIs should change their perspective: in the end, the rational choice ability to develop a conception of the relationship between institutions and behavior and generate highly generalizable set of concepts and expectations that lend to systematic theory building is widely acknowledged. However, the exclusion of any discussion of the ways in which rational choice institutionalist perspectives simplify the image of human motivation and the lack of acknowledgement of various types of instrumentality and non-instrumental forms of behavior affecting both individual and state conduct with regard to human rights may discourage scholars with considerable area studies expertise from using the data and resources that will be made available by this project. The least that the PIs can do is to elucidate these various dimensions of institutions and their relationships with individual preferences/motivations and states’ policies and provide some guidance for the researchers on how to deal with these complexities concerning both the understandings and functioning of human rights institutions in the diverse social contexts. One of the places where such a discussion can be brought in to the study is on p. 13 of the proposal. The PIs acknowledge that NHRIs do not appear in states at random; rather, some type of political or socio-political process lead to their adoption. Therefore, to explain the effect of NHRIs on human rights, the researchers have to consider why states adopt the institution in the first place. This is important to avoid tautological reasoning û states with an NHRI exhibit greater respect for human rights; but states that have greater respect of human rights are more likely to have an NHRI û that will bias the findings. The PIs, in one sentence, mention a couple of statistical procedures they intend to use to explore this question, but does not offer any further theoretical/conceptual elucidation of what substantive discoveries these procedures will allow them to achieve.

According to the theoretical narrative connecting NHRIs to state respect (or the lack of respect) for human rights, individuals’ preference and behavior are central to the NHRIs’ impact. The PIs postulate repeatedly that the NHRIs have the potential to change the landscape of human rights by mobilizing people [emphasis added] for human rights, both on its own and in conjunction with other domestic institutions. Again, on p.6, the PIs state that people must believe that should they challenge the state, they have a reasonable chance of being better off as a consequence, and this belief about the likelihood of a positive payoff to challenging the state is a function of the NHRIs’ independence and accountability. This is re-stated on p.8 as, ‘NHRIs are increasingly able to affect change as they encourage individual citizens to press their claims of physical integrity rights and therefore increase leaders’ perceptions of the cost of repression.’ In all of these chains of reasoning, the NHRIs’ ability to effectuate change in state’s behavior is contingent upon their ability to affect individuals’ perceptions and behavior. The latter is, however, assumed, throughout the project, but never measured or tested directly. The assumption about the NHRIs’ impact on individuals’ perception and behavior is taken for granted and not assessed critically or theorized sufficiently.

I have no doubts about the qualification and expertise of the team of PIs to conduct the project. They are all nationally and internationally known scholars recognized for their accomplishments in the area of human rights, as evidenced from the list of their excellent publications, frequent citations of their work, and previous funding for projects on human rights, among other markers of excellence of their work. The earlier projects are a testament of not only the PIs’ expertise in human rights but also their extensive experience in statistical analysis, research design, data gathering, and measurement of complex conceptual constructs.

Their approach to measuring the NHRIs and their impact is both creative and original. The PIs are working on two separate datasets of organizational and behavioral data on NHRIs. The behavioral data, in particular, features several new dimensions of impact, namely, awareness, independence, and accountability that are proposed as indices for measuring the NHRIs’ impact on states’ behavior in the realm of human rights.

I would, however, encourage the PIs to discuss the concept of NHRIs further. The PIs state the NHRI belong to the set of institution û such as domestic civil society, the judiciary, legislatures, and media û but they don’t elaborate what make the NHRIs different and unique from (or similar to) the existing domestic institutions that work to increase individuals’ beliefs that a claim, if pressed, will effectuate a change in
the leaders’ behavior.

If measures that the PIs intend to collect on the organizational dimension of NHRIs are fairly straightforward, the measures of NHRIs’ behavior are somewhat blurry (I was challenged to draw a conceptual tree for, for instance, ‘awareness’ linking its operational definition to conceptual attributes and, then, specific empirical attributes of this concept).

My greatest concern, however, is with limiting the coding of the NHRIs’ reports to only those published in English (that only reinforces the bias discussed above). The least that can be down in this regard is to identify 2-3 major languages (other than English) spoken in the countries included into the sample for this project and hire graduate/undergraduate researchers with sufficient reading skills in these languages.

The PIs argue for a longer view of NHRIs’ outcomes (footnote 10). The time period (t) over which the impact of NHRIs will be measures seems to be left un-theorized and unspecified just like the sources of data for other domestic and international institutions (TANs, NGOs, etc.) that may shape state behavior jointly with the NHRIs.

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

BROADER IMPACTS

The project does have a capacity to generate broader social and political impact. The enhanced understanding of the ways in which NHRIs can effectuate social and political changes within states independently or jointly with other domestic and international institutions has considerable potential to benefit society. The findings of research analyses carried out on the new data can be used to inform various public and international policy decisions. The dataset that will be in the open access will facilitate further research, but also teaching and training in the scholarship on human rights institutions. If the PIs were to hire research assistants with foreign language expertise for coding reports in languages other than English, the project would have facilitated foreign language acquisition and/or participation/collaboration with students with foreign language expertise (if translators were to be hired).

Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable

I’ve included those into the assessment of intellectual merit (see above).

Summary Statement

Overall, I support the proposal for NSF funding with an understanding that the PIs would take the reviewers’ comments to heart (to the extent possible) and work to improve the conception of their study.
Review #3

Rating: Very Good

REVIEW:

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

The proposal definitely has intellectual merit. It is actually quite shocking to me that more research hasn’t been done on the influence of NHRIs. The hypotheses seem simple and straightforward. The work could bridge a gap between the literature on the development of these institutions and the more nascent literature on their impact.

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

I think the research does have broad appeal and could have a broad impact. However, all of the hypotheses appear to be things that the average human rights worker would take as self-evident. This is not to say that the proposal will not have broader impact but just to urge the project to think about the novel value-added for the advocacy community from this project and how they will convey the importance of these findings.

Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable

If funds are available, the proposal should be funded. The proposal is very strong and clear. This is especially so for the organizational data; however, much of that data appears to already have been collected. I’m less clear on the behavioral data and I am not quite sure how some of that data will used to evaluate the hypotheses laid out in the proposal. I urge the authors to be more explicit about how the behavioral data is necessary for their hypotheses 5-7 and/or would lead to the ability to test additional hypotheses. Also, what is the likely overlap with independence/organization of the NHRI and behavior of the NHRI? If there is not a lot of overlap expected, why not? This is not to say that there is not value from English language coding of the complaints of the NHRIs but that there are built-in problems with this
approach, many of which are mentioned in the proposal. I want to know on what basis the authors conclude that “English has become the dominant language of the international human regime” (page 16).

Are there systematic regions where English language reports are more frequent? This could limit the idea that these reports are a “costly signal”(page 16) û at least, the cost is not uniform across NHRIs. I would urge the authors to include the “half measure” mentioned in footnote 15. Also, are there any selection issues with only focusing on NHRIs that have an “accreditation grade” (page 9)? Does the project encompass more NHRIs than Koo and Ramirez or less? Why? I’m really not sure on this û the Koo and Ramizez paper says there are 178 NHRIs that have been established while the proposal says the focus is on 101 û what explains the difference? Also, if as Koo and Ramirez point out, most countries have an NHRI, controlling for state capacity seems necessary in all models.

Summary Statement

This is a reasonable project. The authors are established on the subject of human rights in general. It would be worth considering for funding.
Review #4

Rating: Good

REVIEW:

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

This grant is designed to collect data on National Human Rights Institutions (NHRIs), which have been adopted by more than 100 countries since 1990. The PIs offer six hypotheses about the effectiveness of these institutions, arguing essentially that those NHRIs that are given more authority are also more likely to be effective. There is some mention of selection effects in this proposal but not enough. The states most likely to have strong NHRIs are of course going to be more likely to have better human rights records. It is also unclear how the PIs deal with legitimacy through time. NHRIs are recent innovations, but most of the literature on legitimacy suggests a constraining effect of ‘parchment institutions’ over time.

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.
The proposal is relatively strong in terms of broader impacts given the large amount of money going to undergraduates.

Please evaluate the strengths and weaknesses of the proposal with respect to any additional solicitation-specific review criteria, if applicable

Summary Statement

This is an interesting proposal and worthy of funding eventually.

Review #5

Rating: Multiple Rating: (Excellent/Very Good)

REVIEW:

In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit.

This research seeks to examine whether NHRIs limit state repression. The authors offer a theory of the conditions under which NHRIs are effective (focussing on independence, autonomy and awareness), and they design an effective statistical test to determine if it is true based on an original dataset. The data gathering and analysis project is ambitious, but the PIs have undertaken considerable pilot research to demonstrate that their project is feasible.

Beyond being theoretically informed and methodologically sound, this project is well grounded in a diverse literature. Interestingly, it takes a new perspective on that literature that adds to the legal, historical and anthropological literature on human rights in an interesting way.

The data generated by this project should make new theoretical and empirical research on human rights possible. This is an important and pressing question.

The proposed research team is impressive and well qualified to undertake the proposed research. I still worry about English language as a defining element, and would like to see the Pis think more about how this could bias results. I also worry about basing behavior measures on state’s own reports.
In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts.

Understanding how to restrain governments from violating the rights of their own people is a very important project (though of course it is not the only type of human rights violation we want to worry about). But figuring out how to reduce state violence against citizens is a very important goal. Im not sure that this project will provide the information we need to prevent occurrences such as the Syrian government’s attacks on its own people, but it should nevertheless provide useful information about institutional design and other critical questions that could inform policymaker’s and human rights advocates actions aimed at reducing state violence against citizens.

The Pis also seem to do a good job of involving students in their research, providing some additional impact through training.