Researcher’s Diary: Polling in a Conflict Zone - Lessons From Iraq

By John Moreira | Insights | Series II | No. 7 | November 2014

As I listened over a scratchy Skype connection to survey supervisors in Baghdad while training them for a political poll, I wondered if the line would go dead again for the umpteenth time that day.

But dropped calls were hardly my biggest worry setting up a poll in Iraq, which presents a complex set of challenges for survey researchers. These include ensuring the safety of consultants, interviewers, and respondents, while creating an environment that allows respondents to provide their true opinions and attitudes.  Iraq is the nec plus ultra, the acid test for polling in what are euphemistically called “non-permissive environments” or “denied areas.”

If Iraq poses tough problems, it also is a place where we have developed answers on how to conduct valid polls in a conflict zone.  Having done survey research there since 2010 for the National Democratic Institute (NDI), I’ve seen that there are methods that can produce surprisingly accurate results. These allow groups like NDI to work with political parties and leaders to increase their awareness of needs of the people.  Just as important, they make Iraq a model of how survey research techniques can be adapted to work even in a country at war.

The security issues in Iraq begin with the role of foreign pollsters, since the very hands-on approach I’d normally take to training and briefing supervisors and interviewers there is untenable.  NDI is very strict on security; they wouldn’t allow me to go to Baghdad or Kirkuk.  Nor is it easy for Arab interviewers to get into the Kurdish areas, where foreigners can go but Arabs are viewed with suspicion.  So there wasn’t much point in my going to Iraq for the poll.

The solution was distance training, as noted above.  All the training I’ve done for the surveys has been over Skype with the main supervisors, who then trained the interviewers. The biggest problem was getting disconnected and having to regain a connection, which happened time and again. The audio would also come in and out at times, with some echoes.  But the supervisors spoke English and we were able to get the job done.

When survey interviewers go into the field, they face risks from both government and unofficial local militia forces. Frequently our interviews are conducted right outside the selected household so they are quite visible. Interviewers are commonly detained, and it’s no joke: they are interrogated by suspicious the Kurdish Asayish (the police force in the North), Iraqi security forces, or local militias. Often we need to get third parties involved to get them released.

To keep the field force safe, we take three specific steps that reduce potential risks.

• Substitute locations based on local knowledge.  Our team of local supervisors has a very detailed and intricate understanding of the changing landscape in their specific region or province.  As we randomly select locations to conduct surveys, we constantly are assessing threats. We recommend alternate similar locations (e.g. similar ethnic, religious, social-economic, and population density levels) if the situation on-the-ground poses unreasonable risk. 
• Get credentials:  We work in advance with the local and regional governments to obtain credentials.  Thus we try to ensure that when our interviewers are stopped by government forces, they have the papers they need to alleviate concerns.
• Go in and out fast: We develop surveys that are not too long, so our interviewers can complete the survey before the local militias get word that a group of people are going door-to-door and asking questions. The quicker our interviewers can conduct the interviews without compromising the results, the quicker they can move on without encountering any potential hazards.   

Anbar province in western Iraq is an example where we had to use these measures in order to work.  Anbar was, and is, the site of some of the heaviest fighting in the country.  It is also the second largest province in the Sunni-dominated West of the country.  Prior to the Iraqi national election earlier this year, we wanted to keep Anbar province in our sample design, though we worried about the safety of our interviewers and respondents. We worked with our local interviewers to determine “safe” areas and analyzed which of these areas were similar in composition to the unsafe areas in order to produce as representative a sample as possible. In this way, our results proved accurate and all our interviewers got home safely.

A successful poll also requires that respondents feel comfortable, in order to freely express their true opinions and not acquiesce to either the interviewer’s or the authorities’ opinions. In countries such as Iraq, where expressing one’s opinions can lead to serious repercussions, especially if they run counter to authority, this is no easy task.

Nonetheless, it is remarkable how often Iraqis want to express their opinions in surveys and focus groups.  Rarely are average citizens in their country asked their opinions and often they appreciate an opportunity that gives them a sense of empowerment.  To get to this point, however, takes interviewers who understand respondents’ apprehensions.

Using local interviewers who understand the local norms and dialects is critically important.  Interviewers need to build an immediate rapport with respondents.  They need to put them at ease, let them know the questions are for a legitimate organization that wants to understand the public’s views, and assure them that all responses are completely anonymous.  Interviewers who know the area, dress in local attire, and can easily begin a dialogue with a respondent are essential to obtaining accurate responses.  For this reason, among others, our response rates in Iraq are typically over 90%.  (This must be the envy of pollsters in over-polled America, where response rates have sunk to single digits!)

A prime example of the difference local interviewers can make came in Kirkuk in Northern Iraq.  Kirkuk province, and in particular the city of Kirkuk, is one of the most diverse areas of Iraq. The city is a blend of Kurds, Arabs, Turkmen, and several other minority groups, all with their own unique cultural norms.  Because of this, our interviewers in the Kirkuk region were the most diverse set of interviewers in the country.  Although Kirkuk city as a whole is diverse, the various sects and ethnicities largely live within their own areas.  Based on our local interviewers’ backgrounds, we allocated them to parts of the city where respondents were likely to feel more comfortable and open in responding to them.  

Questionnaire design also helps ease respondent fears.  For example, we tend to put more controversial questions near the end of the questionnaire after respondents have built a reasonable comfort level.  Additionally, we ask questions in different ways and at different points in the survey to validate a respondent’s response and attitudes. 

For the most controversial questions, particularly a vote question, we use “secret ballots.”  We create a version of the official ballot, have a respondent mark whom they would support, fold the ballot, and place it in an enclosed box or bag along with other ballots so their “vote” cannot be attributed to them without understanding a hidden key placed on the actual ballot.  When respondents show any sign of apprehension about doing this, our interviewers attempt to put them at ease and then use standard “follow-up” vote questions to encourage a response.  This has led to very low percentages of people refusing, leaving the ballot blank, or stating they are undecided when they actually know which party they would support.

Lastly, we always have our interviewers code the “openness” of respondents.  This involved whether they perceive a respondent to harbor deep resistance to answering questions or whether they are very open with no reservations about expressing their opinions.  Less than 5 percent of our Iraqi respondents are typically categorized as being “very concerned and likely did not express true opinions.” Not surprisingly, there are regional differences in “openness.”

Many of the lessons learned in conducting public opinion research in Iraq can easily apply to other conflict and post-conflict area research.  These lessons can help protect both interviewers and respondents while assisting organizations and governments to better understand public opinion – and perhaps even to make governments more responsive.

OTHER ARTICLES THAT MAY INTEREST YOU

Polls Key to Understanding and Defeating ISIS

What Syrians Think: Views on the Conflict, Negotiations, and Transitional Justice

Ask the Survey Doctor: Surveys in Conflict Zones and Non-Permissive Environments