CCVW: National Violence Monitoring System (Indonesia)

This is a joint post with Christian Davenport.

At the most recent session of the Conflict Consortium Virtual Workshop (CCVW) we introduced a new initiative to focus not on a paper but a new dataset – a CCVW, Data Feature. These conversations are loosely guided by the Conflict Consortium’s Standards and Best Practices document.


For this inauguration we led off with Sana Jaffrey (University of Chicago) and her National Violence Monitoring System (NVMS) project on Indonesian violence after armed conflict from 1999 to 2005 (download the Methodology Summary [PDF]). Anita Gohdes and Christopher M. Sullivan joined us (i.e., Will Moore and Christian Davenport), and Jaffrey deserves special credit for speaking with us from Indonesia in the middle of the night. You can watch a video here.

The NVMS dataset (based on a 14 year effort) includes millions of pages of information and 160,000 events, by the incident-day. The data were compiled by human content analysis of news reports from multiple district newspapers across nine provinces in Indonesia, supplemented with other materials.

The work is largely based on a conception of conflict rooted in the work of Lewis Coser, which includes value conflicts that concern the acquisition of status, power and resources through taking coercive action. Such an orientation broadens the general range of discussion away from civil war and armed conflict to include a wide variety of activities such as “riots, violent protests, revenge killings, vigilante attacks, extortion and organized crime.” Interestingly, the effort involved a concerted effort to disaggregate the codes, recording information about the specific news sources that covered the relevant events, the editorial practices of the relevant news agency and the degree of redundant coverage/overlap.


Jumping into conceptualization, participants quickly asked questions about the separability of the different event types coded in the research. Codings are provided for the various categories (see the table at the bottom of this post) and the focus on individual events—rather than aggregated summaries at a daily or higher temporal unit—was deemed to be extremely valuable, a key point made in the Standards and Best Practices document. Further, the NVMS shows promise as a source to study how the various tactics fit together and probe what Charles Tilly referred to as “repertoires”.

We discussed how useful the data would be for evaluating sequences as well as patterns of escalation and de-escalation. We also had an interesting discussion about the distinction made in the data between “types” (noted above) and “forms”. Discussion focused on how the two were related to one another as well as how distinct combinations might be related to one another. Rape was taken to be an impact of the violence (i.e., an outcome) and not explicitly an event type which differs from other projects like Sexual Violence in Armed Conflict.

Another issue raised in the Standards and Best Practices document concerns clear articulation of the population vs. sample for the data. As is generally the case, the NVMS documentation (available at the CCVW page) does not discuss the issue, but does address it implicitly with a discussion of source bias and how the project addressed it. This discussion is conventional in data work in the field, and relies on an implicit assumption that all violence is observable and a census could be collected via “triangulation” of a sufficient number of varied resources. The Standards and Best Practices document challenges this implicit argument, and makes a case for explicit discussion of both population and sample in which discussions about source bias need to be placed.


A different conversation about source coverage also took place. We praised the NVMS for the care taken to understand what different newspapers covered, as well as how. Indeed, we were all hard pressed to identify another project that had been as thorough since the pioneering efforts of Tilly back during the Contentious French project.

That said, questions were raised about how well the project examined and understood the impact of one type of event occurring in the same spot and how that influenced the coverage of other events (i.e., newshole effects). These are not likely independent as the relevant news agencies must allocate resources to cover different events. Additionally, some attention was given to how events in different locales might influence the coverage of events in other areas.

Interestingly, NVMS tried to use different sources to compensate for failed coverage of a specific area but it does not appear to be the case that they tried to examine why coverage was not extended in the first place. This provides an interesting area of additional inquiry, allowing for the systematic evaluation of data generation that is sorely needed in the conflict/peace field.

There were important implications regarding source coverage for the NVMS. For example, events were often categorized by the existence of prior activity/mobilization. If previous action was not covered, however, then it is possible that an event would be categorized incorrectly. This brought into discussion the use of other sources, but it was not always clear exactly how such information was systematically brought to bear.

Indeed, some discussion was made about how rather than simply controlling for biases researchers should systematically attempt to model it like perhaps in the preliminary way offered by Davenport in his book Media Bias, Perspective and State Repression. This was explicitly mentioned during the session as a possible collaboration. Such an opportunity was not unique. One of the positive outcomes of the discussion was the conclusion that a wide variety of topics could be addressed with the NVMS, significantly advancing the study of the topic.

In summary, the NVMS is an exciting venture that warrants the attention, use and scrutiny of conflict researchers. Everyone who is willing to work outside of pooled cross-national time-series data structure should invest some time exploring the data set and determining its value as a potential source they could use to advance their research agenda.

Regrettably, the data’s website was recently removed as a new one is under construction. We are working with Jaffrey to make the data publicly available in the interim and will update you accordingly.

@WilHMoo & @engagedscholar

Categories in the NVMS Coding Scheme

Code in Raw Data Types of Violence (Triggers) Description
88881 Resource Conflict Violence triggered by resource disputes (land, mining, access to employment, salary, pollution, etc.)
1102 Other resources Violence triggered other resource disputes.
1103 Land Violence triggered by land disputes. (public or private)
1104 Natural resource Violence triggered by natural resources such as mining, water etc. (public or private)
1105 Man-made resource Violence triggered by man-made resources. (public or private)
1106 Access Violence triggered by access to employment, markets route, customers, etc.
1107 Environment Violence triggered by environmental damage, air

pollution, noise pollution, etc.

1108 Salary/labor issues Violence triggered by complaints over pay, labor condition, industrial relations between laborers and the management, etc.
88882 Governance Conflict Violence is triggered by government policies or programs (public services, corruption, subsidy, region splitting, etc.)
2202 Other governance conflicts Violence triggered by other governance issues.
2203 Tender process Violence triggered by problems related to government tenders, including corruption in the tender process
2204 Corruption Violence triggered by corruption or misuse of government funds unrelated to tender process
2205 Public services Violence triggered by issues related to the quality of public services, such as education, healthcare, and other services provided by the government
2206 Commodity prices/subsidy Violence triggered by changes in commodity prices or subsidy allocation/distribution
2207 Government programs Violence triggered by problems pertaining to government programs OUTSIDE OF TENDER, CORRUPTION, PUBLIC SERVICES, AS WELL AS COMMODITY PRICES AND SUBSIDY. This includes execution of government programs, funding priorities and complaints regarding implementation or unmet needs, salary issues and government employment.
2211 Region splitting Violence triggered by regional splitting or re-districting
2212 Law enforcement Violence triggered by disputed arrests, problems pertaining to actions by security forces, or dissatisfaction with court proceedings/decisions
88883 Elections and Appointments Violence triggered by electoral competition or bureaucratic appointments.
3302 Other election and public office conflicts Violence triggered by other competition for position and power
3303 National election/appointment Violence triggered by electoral competition or bureaucratic appointments at the national level (e.g.: national parliament members, ministers, President or Vice President etc.)
3304 Provincial election/appointment Violence triggered by electoral competition or bureaucratic appointments at the provincial level (e.g. provincial parliament members, provincial govt. positions, Governor and Vice Governor etc.)
3305 District/municipality election/appointment Violence triggered by electoral competition or bureaucratic appointments at the district/municipal level (e.g. district parliament, district govt. positions, district head and mayor etc.)
3306 Sub-district appointment Violence triggered by sub-district level government appointment (e.g. dispute over the office of head of sub-district (camat))
3307 Village/kelurahan elections/appointment Violence triggered by village/kelurahan level election or appointment (e.g. village head and village council elections, village level appointments)
3308 Other government office Violence triggered by election or appointment at other level of government
3309 Office/influence/power in political parties Violence triggered by election/appointment within political parties
88889 Separatist Conflict Violence triggered by efforts to secede from the Unitary State of the Republic of Indonesia (NKRI)
9903 Separatism Violence triggered by independence/separatist struggle to secede from Unitary State of the Republic of Indonesia (NKRI)
88884 Identity-based Conflict Violence triggered by group identity (religion, ethnicity, tribe, etc)
4402 Other identity Violence triggered by identity of other groups
4403 Inter-ethnic/tribal Violence triggered by ethnic/tribal dispute ( regarding cultural attributes or symbols of diaspora, language and so forth)
4404 Inter-religious Violence triggered by disputes between members of different religious groups
4405 Intra-religious Violence triggered by disputes over interpretation within a religion (e.g. between sects)
4406 Between migrants/refugees and locals Violence triggered by issues pertaining to migration/diaspora/refugees
4407 Between migrants/refugees and locals and certain ethnicity Violence triggered by issues pertaining to migration/diaspora/refugees as well as ethnicity/tribalism
4408 Geographical Violence triggered by long-standing enmity between residents of particular villages/neighborhoods
4409 Gender Violence triggered by gender related issues (including LGBT)
4410 Supporters of sports clubs Violence triggered by issues between supporters of different sports clubs/teams
4411 School/university identity Violence triggered by issues between students of different schools/faculties/universities (e.g. mob fights between schools)
88885 Popular Justice Violence perpetrated to respond to/punish actual or perceived wrong (group violence only)
5502 Other issue Violence perpetrated to retaliate over other issues
5503 Retaliation over insult Violence perpetrated to respond to/punish insults/embarrassment/loss of face
5504 Retaliation over accident Violence perpetrated to respond to/punish traffic accidents
5505 Retaliation over debt Violence perpetrated to resolve/punish debt disputes
5506 Retaliation over theft Violence perpetrated to recover /punish theft/fraud or other financial damage
5507 Retaliation over vandalism Violence perpetrated to respond to/punish vandalism
5508 Retaliation over sexual indiscretion Violence perpetrated to respond to/punish sexual indiscretion for example fornication/adultery/affairs
5509 Retaliation over assault Violence perpetrated to respond to/punish a previous murder/assault/beating/rape
5510 Attack on places of vice Violence perpetrated to stop gambling/drinking/narcotics/prostitution
5511 Retaliation for sorcery Violence perpetrated to respond to/punish sorcery/black magic
88880 Other Conflicts Violence triggered by other issues
1 Unclear Trigger of violence is not clear
2 Other types of violence Violence triggered by issues other than those listed in the coding key



Posted in Uncategorized | Tagged , , , | Leave a comment

A Quick Bit on the DA-RT Kerfuffle


Regarding DA-RT: Are we better off or worse off, as a species, for the Convention Against Torture?  It may surprise many that I am not terribly sure.  It’s a tough question to answer in no small part because there are lots of counter-factual CATs that might have existed had they delayed its approval and had more discussion.  And it is hopefully surprising to nobody (and here I am not a cynic, but a Pollyanaish fool) that those who opposed the CAT back in 1984 did so for diverse reasons ranging across views such as

  • opposition to “hard law” (norms > parchment institutions);
  • it was too vague;
  • it lacked adequate enforcement mechanisms;
  • it violated state sovereignty;
  • states parties could sign reservation clauses;

and so on.  Some may view these reasons as insincere, ill conceived, etc.  They may also view those who championed the views as shills, vapid, or otherwise disparage their character.  I sincerely hope they would not view them as a monolith, but since I study conflict processes for a living, I am familiar with the standard psychological processes such as “image of the enemy” and so on.  So hand me those rose tinted glasses, damnit, because I don’t like what I am seeing.

Having said that, what is my position on the DA-RT initiative?


For those who prefer (poor) graphic presentations…


And finally, for those who dislike abstract and numeric math (frequently, and maddeningly, referred to as qualitative types)…


Why no fucks?  The short answer, is

(1) I don’t think herding journal editors is the best mechanism in which to invest energy;

(2) I think that attempting to herd journal editors may well maximize blowback; and

(3) I have reached my life limit on reading nonsense by insincere hacks who wish to protect their incomes and that of their offspring (some of those signing the petition and writing about it fit this description), and am thus unable to credibly commit to becoming informed.

For what it is worth, I did skim/read most of the DA-RT Working Group‘s publications and websites as the project moved forward between 2011 and 2015 (e.g., here, here, and here, among others).  So while I have not read neither the JETS statement nor the petition to pause, I have a reasonable sense of the former

That said, I do give a fuck about transparency in science, and standards of data collection in particular.  Indeed, Christian Davenport and I successfully sought NSF funding, held two Workshops, and produced a document that declares a set of Standards & Best Practices for collecting conflict data (PDF here).  You might be interested to read it.

And with that, I leave you so that I can finish the revisions to a submission that I have been lamely sitting on.


Posted in DA-RT, Data Standards | Leave a comment

A Moral Hazard in Counter-Terror Policy?

Do restrictions on civil liberties reduce terror attacks? Earlier this week the CC Virtual Workshop featured Tiberiu Dragu‘s paper, “The Moral Hazard of Terrorism Prevention.” Regrettably, technical issues prevented us from recording the session to YouTube. That aside, we had a fun and interesting discussion.

And why not? Dragu’s provocative answer is “probably not.” More specifically, he argues that once we take into consideration the strategic interaction of government security agents and dissidents, the former will underinvest in detective work (thereby creating demand for their services), and the latter will sometimes plan more attacks than they would have had the government not restricted civil liberties (e.g., speech, association, assembly but also enhancing surveillance and domestic spying capabilities).

To produce these implications Dragu studies a dynamic game theory model involving security agency of a government and a dissident group using terror tactics. The game has two periods: in the first, the group decides whether to execute attacks, and the agency decides how much effort to expend foiling potential plots. As in any game theory model, the trick is to consider how each actor will behave in each period based on its expectations of what will happen over the course of the interaction. This is where the interesting stuff happens as the actors generate expectations about what is likely to have in subsequent periods influencing what they do in the present.

In particular, Dragu compares two circumstances: one in which, following a terror attack in period one, the government does not restrict civil liberties versus another in which, following a terror attack in period one, the government restricts civil liberties. The results of the model indicate that the security agents will work harder to foil plots in the situation with no restrictions than the one with restrictions. In part, this is explained by the fact that without the restrictions these actors have to work a bit harder to figure out what is going on. Higher levels of repression make security force agents lazy, reducing repressive action. Lower levels of repression make security force agents exert more effort, increasing the amount of repressive action that one would see. Further, depending on the values of some of the model’s parameters, the dissidents will plan more attacks. If you know that you are going to be in a situation where it will be harder to meet, plan, train and execute, then you would ramp up your dissident behavior. The paper challenges the conventional wisdom that civil liberties restrictions reduce challenges and advances the agenda that repression needs to be simultaneously disaggregated tactically as well as considered together.

The paper is part of a book length project in which Dragu draws also from his articles published in AJPS (pdf here) and APSR (pdf here). In those works, he expands the discussion to include the incentives of politicians, and the book will engage an array of models that permit him to enrich the sparse models in each of the papers. This should be an important piece.

Terrence Chapman, Ursula Daxecker and Monika Nalepa served as discussants, and raised a number of excellent suggestions ranging from modeling choices to framing and presentation to empirical implications and beyond. Dragu also engaged the group in discussion about some of his ideas for the book length manuscript as well as this individual paper. A good time was had by all.

@WilHMoo & @engagedscholar

Posted in Uncategorized | Tagged | Leave a comment

International Law as Coordinating Mechanism to Reduce Massive Rights Violations?

Today’s Conflict Consortium Virtual Workshop paper, “How does Human Rights Law Work? Institutions, Norms and Focal Factors” by Tiberiu Dragu & Yonatan Lupu, suggest a third mechanism by which international human rights law reduces violations. The first two mechanisms–sanctions (aka the logic of consequences) and norms (aka the logic of appropriateness)–are well known and documented, though a number of recent widely publicized books have challenged their impact.  This paper points us to an overlooked third mechanism: the logic of expectations.

Screenshot from 2015-10-21 08:52:46

The idea is straight forward to anyone familiar with coordination games involving lots of players.  Lots of potential solutions exist, but the players have difficulty to communicating which solution they should pool on.  I was reminded of the discussion during the 2004 Democratic primaries for US president in which many voters said that the liked a candidate other than John Kerry better, but planned to vote for Kerry because he was the only one who could win.

Why is that a coordination problem?  Because rather than simply act on their personal preference these voters expressed concern for wanting to pool their votes with those of other voters (mistakenly) believing that doing so was important for beating the Republican nominee that November.  A more common example of coordination games involves which side of the street to drive cars on: either the left of the right side would work.  What is critical is that everyone adopt the same standard, thereby dramatically reducing collisions.

Dragu & Lupu argue that this logic applies to agents of coercion in situations where the executive has to call on large numbers of them to put down a popular uprising.  Consider, for example, countries like Algeria, Bahrain, Egypt, Morocco, and Tunisia during 2011.  All of those regimes faced large scale popular protests, and each government called upon police and/or military to play a role in policing/suppressing the demonstrations.

Dragu & Lupu suggest that the agents of repression face a coordination problem in such situations: if all other personnel repress, than each agent wants to join his/her comrades.  However, if they are going to balk (as the military did in Egypt), then each individual soldier does not want to repress.  The difficulty in such situations is that repressing is a better choice if the leader is going to survive (and punish anyone who sat out) but not repressing is a better choice if the leader falls (and the future regime or a court takes action against those who committed violations).  Yet the success of the executive depends on how the agents of coercion respond.

That’s where international human rights law, and perhaps the lobbying efforts of watchdogs and activists, can play a role.  Not by persuading soldiers that they will be punished (logic of consequences) or that it is wrong to repress (logic of appropriateness), but instead that their comrades in arms won’t repress.  The third mechanism by which international law and activism can reduce violations is by providing information that suggests a focal point solution to a coordination problem among large numbers of actors who cannot readily communicate with one another.

Ann Marie Clark, Amanda Murdie, Keith Schnakenberg and Ryan M. Welch joined me to provide Lupu feedback.  It was a fun and productive session.  But of particular interest, I had asked Scott Edwards if he might “pinch hit” for Christian Davenport, my co-convener who could not make it due to a scheduling SNAFU.

Edwards, who has worked with Amnesty International USA for a number of years, was unable to join us, but he read the paper and caught part of the discussion on YouTube.  Afterwards he fired some comments to Dragu, Lupu and I that are quite interesting.

I was a fan of the model because it formalizes a causal path that I’ve witnessed (and directed) in minimizing compliance with illegal orders; which is essentially to create doubt among rank and file about whether compliance or non-compliance will be the norm…we’ve done this through big, highly visible media presence, with talking points that stress that there will be costs for commanding officers, inter alia.
Importantly, I don’t have enough fingers to count the number of times I’ve gone on AJ or BBC, with my intended audience being security forces and combatants, rather than general viewers. Intent was to create the very doubt/belief that is the causal nexus in the model.

If it sounds interesting, you can watch it on YouTube here.


Posted in Uncategorized | Tagged , , , , | Leave a comment

Trading Territory for Security & Economic Well Being? To the Survey Experiment, Israel Stylee!

In yesterday’s Conflict Consortium Virtual Workshop Stacie Goddard, Alan Dafoe, Jakana Thomas, Alex Braithwaite and I discussed  Devorah Manekin‘s working paper “Symbolism or Materialism? A Public Opinion Approach to Territorial Conflict,” (co-authored with Guy Grossman and Tamar Mitts).

Devorah Manekin (click to watch on YouTube).

Devorah Manekin (click to watch on YouTube).

Manekin, Grossman & Mits have conducted a series of cool survey experiments in Israel to gain some leverage on the extent to which Israeli’s will trade-off territory for economic growth, security, and other good things in life.  In a nutshell, the results suggest: not so much (or, less than one might think).

Naturally, there is a distribution of opinion across the Israelis who completed the survey.  And the authors want to unpack what helps explain who are those who are “stubborn” rather than “flexible,” and what sorts of benefits generate “flexibility” (I am stealing Braithwaite’s suggested, admittedly oversimplified, dichotomy).

I learned about conjoint experimental designs,[1] and the group had a number of interesting observations that seemed to help Manekin better understand where they can shore up the presentation.  We also had a number of concrete, and more than a few vague, suggestions for probing their survey for additional information to further illuminate that distinguishes “types” of Israelis, and what the “flexible” folks respond to.  Inevitably, we also came up with suggestions for designing future surveys.  Doing so is de rigeur, oui?

If it sounds interesting, check it out here.  And be sure to keep an eye on the upcoming schedule.


[1] I learned this, admittedly, over lunch with Manekin after the workshop, where I could fire a few questions at her to clarify where I had gotten lost in the paper.

Posted in Uncategorized | Tagged , , | Leave a comment

The Conflict Consortium’s Virtual Workshop 3.0 (i.e., year 3)

Roughly five years ago, Christian Davenport and Will H. Moore hit something of an impasse and began discussions about what might be done. We were dissatisfied with how political scientists continue to use 20th Century practices and organizations to provide feedback to one another on work in progress. Conferences have primarily become rent-seeking forums: people attend for subsidized travel and reunion with friends (actual and potential), some professional networking, and a pinch of research interaction. As expectations of useful feedback have dropped, the effort people invest in presenting well conceived, well executed work declines with it. “It’s just a conference paper. But did I tell you about the restaurant/art exhibit/ball game/etc. I visited?” Eye. Roll.

Our professional associations diluted the research value further by putting more and more people in each session with 5-6 presentations and 1-2 discussants, the better to drive up association membership. While great for association coffers, this leaves less and less time for presentation and discussion. Indeed, panels increasingly feel like “academic speed dating” as each presenter zooms through the presentation hitting the highest points of the chorus, but leaving the gems of the lyrics in the paper, which probably wasn’t uploaded to the conference webpage, and is likely to go unnoticed if it is.

With this as backdrop we began to search for things we might do to step into the breach. The current process was created back in the days when academics traveled by train, produced copies of their working papers by having a human being type two copies (an original and a carbon copy) at a time, and then paid the postal service to carry it in a stamped envelope to potential readers. Telephone calls were prohibitively expensive, so it made economic sense for researchers to coordinate annually over a several day period when they could travel to a common location and respond to one another’s ongoing work. That we would continue this as our best practice today is an exemplar of path dependent collective stupidity. It doesn’t need to be our grandparents’ 1950’s politico-social. Surely we can do better.

The Past as Prologue

Given our intellectual background, it made sense that our discussions invariably led to a conversation of Charles (Chuck) Tilly (who had recently passed at the time of the conversation referenced above). Chuck had run a workshop at Michigan, the New School and then Colombia that followed some basic rules. Upon being asked about exactly what was involved, these were outlined below by Roy Licklider – a constant participant when the workshop was in New York:

I think we all agree that the seminar/workshop that Chuck created and ran was a remarkable phenomenon. It might be useful to compile its rules in the hope that they might be helpful to others trying to do similar things. Of course, the rules were never written down, and one of the issues with unwritten constitutions is that people often disagree about their content (unlike, say, written constitutions, but that’s another story).

Anyway, I thought I would put down my version, and everybody else can explain where I got it wrong. I’ve put brackets around my comments and specific illustrations from my own experience.

The overriding purpose is to improve a piece of research. Critics are not supposed to show how smart they are by humiliating the author [there was no point to it anyway since Tilly was smarter than anyone else in the room]. A good comment doesn’t just point out a weakness in the project; it also suggests what should be done to make it better (constructive criticism).

There is no overriding topic or theme. Basically it is all about how to do good social science research. [The final title was Workshop in Contentious Politics, and there isn’t much that couldn’t be included under that heading. The lack of a topic made it different from most other seminars and, especially at Columbia, made it difficult to attract members who would keep coming back; Tilly’s reputation helped a lot, and some of us became infatuated with the whole approach, but as noted below this became a problem.]

Within the seminar everyone is treated as an equal. First names are used by everyone for everyone. Everyone is an author and a critic; every regular member of the seminar is expected to present (ideally once a year, although that may not be possible) and to comment on everyone else’s work every week. Specialized knowledge on the topic is useful but not necessary, and often the best comments and questions come from people who know nothing at all about the topic.

Papers are never presented; they are written and distributed a week ahead of the session. There is a reciprocal arrangement; authors limit themselves to fifty pages or less, and all members read the papers in advance. [Chuck once said it was okay if you didn’t read the paper, but you couldn’t say so and then make a comment.] The paper should include an introductory page putting the research in context and explaining its audience (is this a dissertation, a potential article or book, a conference paper, etc.).

At the beginning of the session the author is allowed but not encouraged to say a few sentences, usually about the context of the research (which should be covered in the introductory page but sometimes isn’t). But the session really starts with extensive comments by two preselected critics, at least one of whom does not have a Ph.D. [In recent years these comments were often written in advance and read aloud, with a copy going to the author either before (my preference) or after the oral presentation. This allows the author to not have to worry about taking notes and facilitates discussion. Chuck and I disagreed about reading the comments; I felt that, at least for native speakers of English, people should talk about the comments rather than reading them, which would be good practice for conferences and teaching classes.]

After the two critics have made their remarks, the author is given a substantial amount of time to respond.  The floor is then open to comments and questions. Members attract the attention of the leader by raising their hand (one-finger question); the leader keeps a queue of names and calls on them in the order in which they have been seen, except that the first three comments after the critics must be made by people without Ph.D.s. It’s okay for an individual to raise several separate questions at once. A second kind of intervention is the two-finger question–it must be directly on the point under discussion and thirty seconds or less. Asking a two-finger question does not change your position in the regular queue.

In addition to oral comments, members are encouraged to submit written comments. These fulfill at least two different functions: (1) they communicate specialized knowledge, bibliography, etc. which would not be of general interest to the group and (2) by repeating the oral questions or points, they again free the author from trying to take notes while answering a barrage of very different questions and issues and give them a record of the discussion which will be useful later when trying to recall what went on. [I have actually tape recorded several sessions where I was the author for the same reason. I learned from Chuck to try to keep my own comments until late in the session; with any luck others would make the points on their own and learn more from the experience than if we led the discussion.] Repeating a point made earlier, it is a firm rule that, no matter how wrong-headed the paper is [and there were some dillies], discussion is courteous, friendly, respectful, and directed at improving the project at hand rather than showing that the commentator is brilliant or that the author is insane or dangerous (although all of these may be true). Ideally the author is presented with several different ways in which the paper can be further developed, often contradictory ones which gave some choice.

After the seminar (which is scheduled for two hours), everyone is invited to go out to dinner somewhere nearby (it obviously helps if the seminar is scheduled late in the afternoon). The check is shared, but the author doesn’t pay. [I used to explain that they had provided the entertainment. This may not be haute cuisine; Chuck would alternate between two inexpensive restaurants (usually ethnic). He justified this by saying he wanted to encourage graduate students to come by keeping it cheap. When he didn’t attend during the last semester, the seminar went somewhere else to eat, although not to a much more expensive place, so maybe he was on to something. Once, when only faculty showed up, we went to a better restaurant. Occasionally, if he had gotten a nice check (as he would put it), he would pay the whole bill himself.

I think he regarded the dinner as the high point of the experience, and certainly many of us did. I made a point not to sit next to him to give graduate students a shot at him; at Columbia they were sometimes a little shy, but they soon got over it.]

With Chuck’s Workshop in mind, three years ago we decided to create something in line with the spirit outlined above but do it virtually, online – taking the older model into the 21st Century. The Conflict Consortium Virtual Workshop was born.

Challenges & Successes: Our Experience to Date

Given where we were coming from, we anticipated that there might be some issues to overcome.

For example, who the heck would want to put themselves through what some view as a grueling performance of academic “Survivor,” which is one thing in a room of 5-50 people who after leaving the room basically forget the whole experience. This is quite a different matter when half a dozen people have 90 minutes to discuss your working paper, live on the Internet, and the whole thing is recorded to CC’s YouTube Channel for posterity. Have you been to a panel where a discussant rips into the presenter and the audience (seeing the “blood in the water”) moves in for a kill like some version of “Academics Gone Wilding”?


In contrast, we wanted to “nurture the youth,” and limited CCVW applications to working papers by non-tenured faculty and PhD students. We were interested in getting people comments on their work and doing this in the supportive way that Chuck had managed to pull off. We began by adopting Chuck’s “Rules of Engagement” (as articulated by Licklider), bounced back and forth drafts of carefully worded emails and a blog post, shared our vision face to face with senior folks we know play well with non-senior folks, and selected kind, engaging and communicative participants to serve as Discussants. We also continued to remind one another to keep a watchful eye on interactions while sessions are underway, task the Chair (one of us) with mediating (should the need arise), and hold a dyadic debrief after each session to discuss how we might improve. Without exception all of these interactions have, indeed, been marvelous.

That said, we have not been deluged with applications in response to our Calls for Papers, and as such, are still working on getting the word out. The CCVW provides a unique opportunity: where else can one get half a dozen conflict researchers to spend 90 minutes giving you feedback on your working paper? We have discussed collecting testimonials, and we ask our past participants to spread the word virtually and face-to-face. As with any new endeavor, however, progress can be a bit plodding and we can always get more assistance in moving things forward (did we mention that we were being backed by the NSF?).

Another potential challenge concerned soliciting free labor: who would be willing to sit and participate 1½ hours to discuss someone else’s work (we very consciously try to build networks by selecting participants who do not know one another well, but given the small size of the community our success varies).

Here, too, Chuck inspired us. He enjoyed speaking and was an incredibly good as well as entertaining speaker. But, one thing that stood out to us about Chuck (and many other, given the tributes written after he had passed) was his desire to help people do better social science. Davenport, who was able to participate in several Workshop sessions over the years, gained tremendously from his interactions with Chuck. Davenport came to realize that Chuck would never really tell you what he thought was the most promising approach. Rather, he would provide 2-3 alternative ways that seemed equally promising. Now, his opinion might have existed somewhere in the three (like some intellectual shell game) but Davenport never found the shell that contained the item he preferred. Always teaching, Chuck knew better than to just hand one a solution. With that approach he engaged researcher’s work and joined them in their search to find the right argument, the right data, the right test or the right conclusion. He truly enjoyed the journey and as we would find, many of us similarly enjoyed it as well.

To our delight we have found the people in the conflict and peace community remarkably gracious with their time and supportive of the endeavor. Plenty of people are unable to accept a specific invitation due to a travel or other commitment, but with the exception of a few “non-responses” we have people not only willing, but enthusiastic to contribute time reading the paper and then an hour and a half online in discussion.

Equally important, if you watch some of the sessions we are sure you will agree that the quality of comments, suggestions, and discussion are very high quality: much stronger than the type of exchange we tend to generate at our megameetings. This has been, perhaps, the most gratifying part of the experience for us.

Third, might 90 minutes be too long? Given the short amount of time that most academics get to discuss their work, we wondered if we could provide an environment where actual conversations could emerge – online, among strangers.

Scheduling the sessions for 1½ hours proved a good decision. The conversations are engaging, content-rich, insightful, occasionally funny and quite useful – not just for the presenter but for all that observe the interaction. Individuals come away with a sense of how the presenter as well as other participants think but also how one responds, how research designs are structured, how data is collected and how results are written – all of the components of a decent research paper.

The conversations also seem to have an interesting rhythm to them. The beginning typically leads to partial immersion, followed by a bit of a lull, then a deeper dive to full immersion, some reflection, some probing, and then at about the hour mark there is often something of a 7th inning stretch, after which the full energy returns for the final twenty five minutes, and we almost always ending up cutting off the conversation due to time rather than having it “run out of steam.”

We have also encountered the standard sorts of diversity that confront science. In an October 2013 blog post, Moore lamented a particularly lopsided gender session, and reviewed the numbers to date. Our process has become one where Moore takes the lead generating a list of six people to invite as Discussants and a list of four to six “alternates” to pursue as we get declines. Davenport then reviews, and revises the list. Given that the two of us, who are male, participate, we have found that four women and two men in our initial six works best for striking a gender balance. We also try to get at least one full Professor and two other tenured faculty, filling out the roster with Assistant profs and PhD students. We also like to find at least one person who is from a different field (generally Economics, Sociology or Psychology). And we want our panel of discussants to represent different subfields, research networks, and so on. Did we mention race? Language? The global south? The fact that the time of day systematically excludes Asia is an issue as well (which is asleep at the time of our e-event)?

That sounds like a fun set of dimensions to maximize, right? Needless to say, we do not maximize that multi-dimensional space. Instead, we start with a list of names, eyeball our criteria, and start cutting and adding, cutting, and adding people (sometimes searching our CC Member List, Google Scholar, References of papers, and so on). It is a very seat-of-the-pants (or skirt) process, and hopefully we haven’t sucked at it.

To CCVW and Beyond!

With two year’s experience under our belt we are pleased with where the CCVW is. And we have begun to seek out other uses for the virtual format. One extension concerns what we call “Data Features.” Deviating from the standard workshop, these will involve a short data presentation, but immediately afterwards we will open the “floor” for an invited panel to ask questions about how the data were collected, what could be done with it, what has been done with it and what would they have done differently or next.

Another extension acknowledges that megaconference panels that actually “work” are frequently just too brief, and that there is really no reason why we could not continue these conversations off-site and online. We are calling these “Conflict Consortium Continua” as the presentations and conversations should be thought of as moving along a continuum of interaction. We will be adding these to the mix over the next year. We also discuss additional extensions, and welcome your ideas and feedback.

Please Steal this Idea

In closing, we hope that others will follow our lead, launch virtual workshops for their communities, and produce even better innovative public goods for nurturing research. Indeed, the Legislative Studies Virtual Workshop and Virtual IPES are already up and running. The International Methods Colloquium offers another model. Some will want to invest their energy in changing the existing megaconferences, and there is nothing wrong with that (basically). This said, we hope to see more entrepreneurs thinking of novel ways to leverage communications technologies in the service of the production of scientific knowledge. As the voice-over for old tv show The Six Million Dollar Man so presciently reminded us, “We have the technology.” Now it is time to use it.


This post is cross-posted at Analog (the anti-blog) and Will Opines.

Posted in Uncategorized | Leave a comment

“Et tu, Brute?” or “Congratulations, you have an all male Reference list!”

The other day I got an email that contained a great implicit question.  My interlocutor wanted to assign my recently published Presidential Address to Peace Science (ungated proof here), but expressed reluctance to do so given the paucity of work written by women cited within it.  The subtext seemed to be: given your activity supporting greater awareness of sexism in the discipline, what’s up with that reference list?


A bit o’ hyperbole: 90% male is close enough.

I had not eyeballed the gender composition of the References in the piece, and thus did so.  Eighteen of the twenty publications (90%) were authored by one or more males, one by a female, and one by a mixed pair (see table below the notes).  D’oh!  That is an arresting percentage.  So, indeed, what’s up with that?

Confronting the Implicit Biases Systems Impart

Like any systemic bias, sexism is a complex web and I am not going to try to briefly sketch it: we all have grown up in a sexist society, and it impacts all institutions and human relations, including, of course, political science and its many subfields.  That said, I can think of several specific domains in the profession where I have sought to address implicit biases.[1]

  • PhD Student & Faculty Recruitment
  • Mentoring (students & colleagues)
  • Letters of Recommendation
  • Syllabus Construction

But it turns out that I have always been ambivalent about vetting References in my writing and I have never pushed myself to explore my ambivalence.  The outcome, this recent email forced me to realize, is that I have not made it a habit to scrutinize my References for gender (or other) composition.  Which raises the questions: Why not? and Should I change?

The Gender Citation Gap

It would be fabulously weird, given sexism in society, if there was not a gender citation gap (for documentation of said gap, see herehere, here, and here), and it would be equally surprising if that gap did not decline over the decades.  The question is: what should we do about it?

I think syllabus construction is an easy case to assess.  First, when we create a syllabus we are producing a public good.  When we publish we are making a case for our ideas in a competitive marketplace of ideas.  When we produce syllabi we are training and socializing.  In this latter role we have an obligation to make our students aware of a variety of ideas, and this is especially true when it is the Core course / Proseminar.

As long as one does not reject the exist of sexism, it is not difficult to observe that (1) women were systematically excluded from the profession through the 1960s, and have only gained equal access to PhD programs during the 1990s,[2] and (2) this fact will dramatically impact women’s representation on syllabi.  The syllabi we create are strongly influenced by the syllabi of the courses we took in graduate school, and as a consequence, active intervention is needed to prevent change from moving at a snail’s pace.  In brief, this is why I vet my syllabi for under-representation[1] and urge others to do so as well.

Let us now turn to the work we engage in our research, and lands in our References.  Writing at the Duck, Dan Nexon conducted a self experiment and argues in favor of actively reviewing one’s reference and then making adjustments to include more work authored by women.  What is the case for ambivalence toward such arguments?

The truth is, I don’t want to talk about this publicly.  Why?  Because I harbor something close to contempt for the modal citation practice in political science.  I think the community cites poorly.  And don’t get me started on the nonsense policies of misguided journal editors to place limits on the number of citations (at the precise historical moment when Adobe’s PDF language had made paper irrelevant–you have to be fucking kidding me!).  But I digress.

If you have an opportunity to buttonhole one of my PhD students, ask her/him whether I gave them static about their crappy citation practices during graduate school.  But, as you can now see, I have no incentive to share this: randomly ragging on large groups of people tends to be counter productive.  Yet, I can see no way out of the quandary, so damn the torpedoes.

Citation, in my book, tells a carefully crafted story and should never be used strategically.   When we cite we are acknowledging an intellectual debt and we are producing a narrative that reveals the development of knowledge in a given area of inquiry.  That is precisely what makes the fuktup citation practice in an article like Collier & Hoeffler (2004) [5,384 cites on Google Scholar] so mind-bendingly frustrating.  The research reported there is interesting and valuable.  The fealty to community is bankrupt.[4]  And while that paper is an outlier, the modal paper I read falls well short of what I believe we should strive for (and those I review usually fall even farther from the mark).

So, in thinking about the source of my ambivalence over the past few days, I have narrowed it down to a clash of values.  Ceteris paribus I like the idea of vetting References for exclusion of under-represented classes of voices, but my concern about a different value—the careful construction of the story of the germination of my ideas—has taken precedence, leading me to not even conduct the review.

Show Us Some Data

OK, so what does the gender citation pattern look like for someone who has paid zero attention to it?  I decided to look, and restricted the sample to the six solo-authored articles I have published (1990 Political Communication & Persuasion, 1995 PRQ, 1995 JCR, 1998 AJPS, 2000 JCR and 2010 PS).[3]  Four were published in the 90s, one in the 00s, and one in the current half decade.


Note: the red comment boxes indicate 11 self citations.

What do you think?  I’m not sure what to make of the specific values, but the pattern seems to be precisely what one would expect, especially for a white male pursuing a rational institutionalist research agenda studying dissent-repression using statistical models to conduct hypothesis testing.

Hello, Can we Consider Marginalized Groups who aren’t Privileged White Women?

As Jeff Colgan observed in a footnote to his recent post on the gender composition of work assigned in IR graduate syllabi:

Diversity in IR, beyond gender diversity, merits attention and study. The data are a lot harder to collect. I know, that’s a crappy excuse.

But I am less concerned about the paucity of data than the paucity of discussion.  So since I am navel gazing, let me state that the authors of the 313 works I cited in my .  Put another way, the ideas of people of color and people who live and think in the Global South are poorly represented in my research.  That fact makes my work quite typical for the literatures to which I contribute–even absent data, I am confident that the data will reveal this to be so–but that doesn’t make it any less of a problem.

Whether that problem is better understood as a problem of access to training and tenure-track positions that are biased against people of color and those from the Global South or the intellectual hegemony of white people and the Global North I leave to the reader to consider.


So, what will my practice be going forward?  My navel gazing leads me to conclude that the my ambivalence was overdrawn and the post draft review proposed by Nexon makes sense.  I should have incorporated it earlier.  I can’t say how much of an impact it will make on my citations, but a practice that pushes one to confront implicit bias will do not harm.  It is not obvious that I cannot “have my cake and it eat it too” (i.e., the trade-off I have implicitly imagined may not exist).

In closing, I want to state that I do not wander this cold orb under an illusion that I know what’s best.  The best thing about a scientific community is that none of us can dictate: the community decides.  This is, of course, true of all democratic systems.  Any given argument I make or position I take may be ill considered, stupid, or otherwise flawed.  And I reserve the right to change my mind as I learn from others.

Finally, I am in my early 50s, and folks in their 20s and 30s have been socialized in a more equal, less biased world than I.  As such, their age cohort has higher expectations than mine. A non-trivial portion of socio-political change involves younger people pointing at their elders and jeering at their flaws.  This is a good thing.  So if my position here (and elsewhere) warrants, point and jeer away.


[1] Not only with respect to sexism, as I briefly note further down in the post.

[2] This is a blog post, so I am killing nuance: it is part of the genre.  Yes, that claim uses a dichotomy that obscures, fails to address faculty, and a variety of other issues.  I’m living with that.

[3] I cannot vouch for the extent to which my co-authors did (not) pay attention to the gender composition of References, and thus exclude co-authored articles from the sample.

[4] The specific problems are a garish use of straw-“men” characterizations and absence of interest in an enormous amount of relevant published work from a generation of scholars who published during the 60s-80s, and some in the 90s.


Citations in Moore (2015, CMPS)

Posted in Uncategorized | Tagged , , | Leave a comment