Alice, documenting the strikes

Alice Ross is a reporter at the Bureau of Investigative Journalism and runs a team that looks at drone strikes, particularly at off-battlefield locations in Yemen, Pakistan, and Somalia.

She talks to us about how she and her colleagues collect data from both public domain sources and field investigations, the groundswell of media attention on the drone issue in recent years, and what narratives they have been able to draw from their data about patterns of strikes. She also asks key questions about the legitimacy of a state-sanctioned targeted killing program?

How long have you been working on the issue of drone strikes?

The Bureau started investigating drones in early 2011and I joined the team the same year.  We launched the first stage of our data in July, there was another phase in August 2011, and nearly since then we've been tracking drone strikes and running investigations about our findings.

Why did you decide to go into collecting and documenting all this data for drone strikes?

My colleague Chris Woods had been looking at drones as a matter of interest for some time, and he felt that of the available recording organisations there was a lack of emphasis on civilian casualties. We set out to reexamine all of the materials that were available in the public domain, with a specific emphasis on civilian casualties. What we found is that civilian casualties appear to be significantly under-reported in the independent estimates to date. Although the CIA was claiming that this was an exceptionally accurate and surgical weapon that caused almost no civilian casualties, in fact it appeared that hundreds of civilians were dying.

What is the process you use for collecting and documenting drone strikes?

It is a continuous process of adding to our knowledge about the drone strikes. Our starting point will be news reports, tweets, and press clippings, that kind of information. At a later stage many strikes are investigated, for example by field investigations run by news agencies or perhaps investigations runs by NGOs, and our processes continually add to our understanding of strikes. We never look at a particular drone strike and think we know everything there is to know. We will always be open to going back and advising in the light of new information.

What kind of data do you have? What are you looking for? Is it a database?

What we've put together is essentially a list of drone strikes. For each drone strike we record how may casualties are reported to have taken place, how many of those casualties were reported to have been civilians, and how many people are reported to have been injured.  We also look at all of the available reports and put together a narrative indicating what's reported to have happened at what time. That narrative can be very unclear -  while one source might say a vehicle was hit, another might say a compound was hit. We try very hard not to make a decision about that.  We simply say some sources said this, and other sources said that. We also include all of the references, all of the sources that we've used, so people can really look through the material and come to a decision on their own. From those individual strike records, we also put together a tally of the total number killed. 

Because in a single strike some reports might say that seven people have been killed and other reports say nine people have been killed, over time you end up with a huge range from a minimum to a maximum number of people killed. Data for Pakistan at the moment will show between 2,500 and 3,500 people killed.  So we can't say with certainty what that number is, but we feel it is somewhere in that range based on the reporting we have.

Aside from the news reports, tweets, and NGO reports, there have been a couple of leaks of both Pakistani and US data, and we have reconciled those with our data as well. My colleague Chris Woods  obtained a Pakistani document showing their estimates of every single strike, or series of strikes, between 2006 and 2009.  So we have included those as well.  Our data can take all sorts of forms and it's not what you might conventionally think of as data. It's gleaned from all over the place, then we condense and try to make sense of it.

But there are still major sources of information which are completely inaccessible to us, and they are the CIA's own estimates; the American government's own estimates and accounting of how many people it feels are being killed, its own assessment of individual incidents. They are really the missing piece of the puzzle of the drone campaign, which at the moment are completely secret.

How do you verify and identify disinformation?

is a very, very complicated issue. We work with reports created by credible news organisations, so their verification processes are our verification, if you like.  If we are relying on tweets, we will rely on people we are familiar with. There are simple ways to verify tweets..

For official sources, in a way they tend to come with their own pedigree. There are certainly sources we have found to be unreliable in the past, so we keep those ones in a sort of quarantine and won't include them in our references. We try to work with really trusted journalists.

What is the most difficult story you have obtained from that information collection process?

The biggest challenge for us is that there is so much information available on drone strikes that it is essentially a full-time occupation for one member of our team to stay on top of all the material that is coming out. It's a huge challenge simply being able to digest everything that has been put into the public domain. However, information on specific drone strikes can be very vague and sometimes two accounts can be completely contradictory. So we make efforts to return to journalists, to ask them to revisit their notes.

Even when you reconcile those accounts, you can still end up with some incidents where simply very little is known because it has happened in a very far off place, an area that is out of bounds to journalists and to international media.  It is dangerous even for local reporters because of the level of militancy in the area.  So really the biggest challenge is the lack of information on particular incidents, particularly the lack of information about who is being killed.  Often we will have the name of one individual, maybe a militant leader, but he will be one of six or seven people who was killed and we will know nothing at all about the people who have died alongside him. The Bureau has just launched a project called '

Naming the Dead

Featured on our Resources pages, Naming the Dead is a new project aiming to create a comprehensive database of which strikes happened and who was killed.


Read more
Explore Resources
Naming the Dead' where we are attempting to identify as many people who have been killed as possible - it's really an attempt to address those gaps that exist,  where we have an estimate of the number of people who have been reported to have been killed, but more often than not we don't know who they are.

You are also know for your investigations on the ground. How does that work?

We work with press reports and court records and all sorts of documentary evidence as our starting point for investigating drone strikes.  However when we feel that there is a pattern, for example strikes that are very problematic or incidents that are of real concern, we will often use field researchers to go and investigate those strikes for us, to come up with an independent account of what is reported to have happened. My colleague Chris, for example, used this technique to investigate reports that drones were attacking rescuers and funerals. We investigated that twice now and both times we have found that drones have been attacking those who are rescuing people from the original drone strikes.  To do that, we work with trusted journalists in the tribal areas. In certain instances, we have been able to rule out reports that civilians have been attacked while rescuing people from wreckage. So what we find in our field investigations doesn't always mirror what's is in the initial reporting, but in the majority of cases we have found that it does tend to reflect and add detail to what's already been reported.

Can you tell us more about how your data has been used so far?

The most common use of our data is as source of information for news reports. Every day we will see news reports citing our overall statistics on the drone campaign and in newspapers around the world. We also publish a monthly report which get picked up quite widely by the press. The data is also being used to inform academic reports and studies, for example the report, which was published last year by Stanford and NYU universities. Chris' investigation into attacks on funerals and rescuers was cited by two UN Special Rapporteurs on terrorism and extra-judicial killings as a potential war crime. The UN Special Rapporteur on counter-terrorism has now opened an investigation into drone strikes.

How important is information visualisation in your work?

Visualisation is a very, very important part of communicating, and some of the most powerful work that is being done using our data has been some of the visualisation.  There is a visualisation called '' that alone has probably had as many re-tweets and as many hits and visits as some of our larger investigations. It reached people in a very immediate way where words sometimes don't.  The real ability of the visualisation is to lay out for people the shape of this campaign, and the numbers of deaths that are reported to have taken place.

Do you let others use your evidence and data?

We work closely with all sorts of other organisations, whether they are researchers, visualisers, students or technology experts who might be putting together different projects. We provide our data to them and it is fascinating to see the types of project that they come up with.  For example a developer called Josh Begley came up with a project where he attempted to tweet every drone strike in the last ten years in 24 hours, which ended up taking him several weeks because there were so many drone strikes. That was a project that really captured peoples' imagination, as the project combines information from our data about individual drone strikes with a simple photo of where that strike took place. Again, a photo is an incredibly simple and powerful way of conveying some of the impact of the reality of the strikes.

Can you tell me a bit more about two forms of impact your work has had - on the one hand the impact on actions and decisions in the West, but also the impact when the data goes back to where you gathered it from?

When we first published our data in 2011, there was fairly little media attention in the United Kingdom. There was some media attention in the United States, but it was really quite self-contained.  It didn't make a huge impact, but what we have seen over the time that we have been working is that work by us and other organisations has led to a groundswell in the amount of focus on drones. This happened to the extent whereby, even though the drones in Pakistan are a covert CIA program, Barack Obama stood up and made a speech about them in May in which he acknowledged that drones have caused civilian casualties. He acknowledged that there is a difference between the US estimates of civilians casualties and those of independent observers. Those are really important steps that have brought the drone campaign out of this secret form of bombing into something which is a real political issue in the United States. Obviously accompanying that has been a real increase in activism around drones and a real increase in the debate. 

One thing that we have seen in Pakistan, particularly since Imran Khan based his presidential campaign on opposition to drones, is a surge of anti-drone sentiment. They have become very hot political issue inside Pakistan. It's hard to gauge exactly what the response is within Pakistan because it is a very diverse culture and a very diverse country. 

There have been a number of reports which claim that 90 percent of everyone killed in drone strikes is a civilian, and obviously our work counters that quite firmly.  We find that around 20 percent of those killed in drone strikes are civilians. It's very hard to say where our work feeds into this but we do receive a lot of coverage from Pakistani sources, and I think that the overall tracking and quantifying of the harm done by drone strikes gives an evidential base for some of those reports. 

Rather than simply documenting in a database, you work like journalists, and journalists are about telling stories. What kind of narratives do you seek out of the evidence you gather?

Over the time that we've been looking at drones we've seen their use change quite considerably, certainly in Pakistan.  Civilian casualties have fallen significantly since we started looking at drone strikes. In 2010 at least a hundred civilians were killed in drone strikes, according to the report that we have. So far in 2013 (up to September) we haven't seen a single confirmed drone strike that killed civilians in Pakistan. That is a remarkable change. What we've also seen is that drone strikes have become much less frequent, and when they do strike they tend to kill senior named militants along with a number of other people whose names we might never discover.  Drone strikes have also got smaller; each drone strike now kills fewer people on average. Those are important trends and we feel that what they reflect is a more careful targeting practice, and much closer attention to avoiding civilian casualties of the kinds that once took place on a very frequent basis. If you look at drone strikes in 2009, 2010, or during the Bush era, civilians died all the time. That is now a comparative rarity.

Another narrative that you can see over time is the impact on US relationships with Pakistan. For example in 2011 there was a series of events that hit the US relationship with Pakistan very badly -  CIA agent Raymond Davis shot two Pakistanis in Lahore, and there was the Osama bin Laden raid of course, then there was a friendly fire incident that killed over twenty Pakistani soldiers. There was also a seven-week pause in drone strikes at the end of 2011, leading into 2012.  So you can see how patterns of what is going on in the political sphere will really impact how drones are used. 

Those are really important patterns, I feel. 

What we have seen is drones being used to target not just senior terrorists, which is the narrative that the US presents, but quite often to target militant groups who are carrying out counterinsurgency over the border in Afghanistan. President Obama himself acknowledged that. In May, he said that this would now wind down as US troops withdraw from Afghanistan. What we are interested to see in the coming months is what the impact of the withdrawal from Afghanistan will be. Will drone strikes slow down completely?  US Secretary of State John Kerry went to Pakistan at the beginning of August 2013 and he said that drone strikes will stop very, very soon.  That was almost immediately contradicted by his own department, who said that they would never deprive themselves of a weapon or a tool that might prove useful. But I think it's an indication that overall drone strikes are on the wane in Pakistan, that we may have seen the worst of it. Obviously you can never predict that, but the general trend now is for a decline in drone strikes, meaning a decline in deaths.

Is your data telling you anything that would enable you to somehow reconstruct the US list of terrorist names, the disposition matrix, that is alleged to be used? Does it tell you anything about the 'black box' of intelligence gathered and how the decision is made to strike?

The thing is that in the absence of official transparency about the drone strikes, we are stuck trying to piece together a picture afterwards, like doing a jigsaw puzzle from the glass of a broken window or something. You are trying to piece together the clearest picture that you can, but ultimately the most detailed information about the drones, who they are targeting, why they are targeting that individual, what is known about the other individuals who might die alongside them, is all completely secret. So there is very, very little that we can say with confidence about the killing program that is in operation. Although drone strikes in general are in decline, and although we can say that is an improvement, the fact remains that this is a targeted killing program. 

These drone strikes are not taking place on a battlefield.  The legal framework under which they are taking place remains very, very contested.  So even though we can that civilian deaths are falling precipitously, the question then is - is that good enough?  Is it acceptable to have what is essentially a targeted killing program?  Is this legitimising the practice of targeted killing?

If you were to name the major activist working on trying to reveal the drone programs in a different context, who would that be?

There was a peer research poll recently, which found that of all the types of warfare that they polled, women always opposed war more than men, across different nations and across different forms of war. But on drones the gender gap is greater than on any other weapons platform across almost all countries, and it's remarkable how large the difference is. I wouldn't want to speculate on why that is, but what you do find is that a number of the leading experts and campaigners on drones are women. You've got Hina Shamsi of the ACLU, Naureen Shah at Columbia Law School and now with Amnesty International, Jennifer Gibson and Kat Craig at Reprieve, Medea Benjamin and her colleagues at CodePINK...  there is a trend.

Is there any link between the military application of drones and those used as weapons of surveillance? In the sense that they are being sold to the public as safe and good for politics, so everybody likes them...

Image - covertdrone.pngI think there is a distinction that doesn't get made very often, that drones are used in three main ways.  They are used for surveillance certainly, both in the military context and increasingly in civilian applications.  Then there are military armed drones used alongside conventional weapons in Afghanistan, Iraq or Libya for example. Then there is this third category of covert drone - drones which are used away from the battlefield to kill specific (or sometimes not that specific) individuals or groups who are believed to pose a threat. That is the area that we have studied because it seems to us the most problematic area of the use of drones, because it happens in a legal framework that is certainly open for debate. It happens with no official transparency. 

It was eight years into the Pakistani drone campaign, and ten years after the first non-battlefield drone strike, that Obama first acknowledged that drones were being used in that way. So for ten years drone use was a more or less open secret, but it was nonetheless a secret program.  It's this covert use of drones that seems to us to be very problematic because there are issues about transparency and accountability.  Who takes responsibility if someone's home is destroyed, or if a strike goes wrong and the wrong people are killed or wounded?  Is it legitimate to simply kill people with no prior judicial process? It's effectively execution with no due process. These are really, really big important questions. At the moment there are very few nations that have armed drones, but the technology is advancing very fast and these are questions that need to be settled sooner rather than later, because there is a real danger of some very worrying precedents being created.