At the heart of the most high-profile investigation currently under way in the United States is an extremely tedious, time-consuming and technologically complex task: sifting through thousands of videos and photos, looking for the people responsible for the Boston Marathon bombing.
In the days since the violent attack that left three dead and approximately 180 injured, the Federal Bureau of Investigation has examined video from surveillance cameras and the thousands of witnesses near the Boston Marathon finish line. On Thursday, investigators announced two unidentified suspects in the high-profile case.
Working in the FBI’s favour is the sheer amount of footage available from the day of the bombing. The marathon is one of the biggest public events of the year in Boston, and is run on Patriots’ Day, a civic holiday. The two explosions occurred near the finish line at around the four-hour mark of the race – traditionally a point when a large number of runners cross the finish line – and many bystanders had their cellphone cameras out, recording the runners as they completed the race.
But with such a massive amount of footage to sift through – and more new video coming in every day – investigators face a difficult task in trying to separate useful information from noise. Much of the video from nearby security cameras panned over large crowds, where picking out an individual face becomes trickier. Detailing the movements and body language of potential suspects is also particularly challenging given that, in the immediate aftermath of the explosions, the first instinct of many people was to flee from the area of the blast.
One of the biggest hurdles standing between the FBI and a positive identification, according to facial-recognition experts, is the imperfection of everyday video and photos. Images recorded in a controlled setting, with the subject facing directly forward – for example, driver’s licence photos – tend to be very useful for facial recognition. But non-standard images present a host of difficulties.
“Facial recognition is like any biometric fingerprint: It works very well on a face if you have the whole face in a very controlled matter,” said Paul Schuepp, chief executive officer and president of Animetrics, a New-Hampshire-based facial-recognition firm that works extensively with U.S. law-enforcement and government agencies.
“But anybody’s camera can be very random, very uncontrolled.”
Besides the problems of “angulation” – when a subject isn’t looking directly at the camera and part of the face is hidden – photos and videos from the public often come in a wide variety of resolutions and formats, rather than just high-definition images. Lighting and shadows can add more complexity and anything that masks the face, such as a wide-brimmed hat, makes a positive ID even less likely. Sunglasses are particularly problematic because they hide the eyes, the first facial feature some recognition software is trained to pinpoint.
To solve the problem of angulation, companies such as Animetrics have developed in recent years a set of programs that turn two-dimensional images into composite 3D models. Other software also helps “fill in the blanks” of a partially obscured face, making the image more useful. However, the technology to zoom in to a photo without loss of definition – a staple of police procedural TV shows – doesn’t work nearly as well in real life.
Many of the problems the FBI will face in its investigation mirror those faced by the Vancouver Police Department after the 2011 Stanley Cup riot. In that case, investigators searched some 6,000 hours of video.
“We basically spent weeks tagging people in the videos,” said Vancouver police spokesman Constable Brian Montague. “It was very tedious work where dozens of people sat at computer terminals and where everyone was given a video and instructions on tagging.”
Investigators tagged individual people in the videos by labelling them according to gender, clothing and facial features such as mustaches and beards. Once the footage was tagged, investigators could search for people in minutes, instead of hours or days. “We had to front-load the investigation,” Constable Montague said.
With suspects pinpointed, the FBI is likely to check images of the two men against a host of information sources, such as the agency’s own multimillion-entry criminal database. Nonetheless, investigators on Thursday made public a set of images of the two suspects in the bombing – part of an appeal to the public to help find or identify the men. In addition, the FBI has set up a dedicated website to collect raw images.
“We have been collecting video from a variety of sources – as you’d imagine at the finish line of the Boston Marathon there’s lots and lots of video,” Janet Napolitano, the U.S. Secretary of Homeland Security, told the House Homeland Security Committee in a statement. “There is some video that has raised the question of those the FBI would like to speak with. …We need the public’s help in locating these individuals.”
But online, some members of the public have already taken it upon themselves to try to identify the perpetrators. On the hugely popular website Reddit, a forum dedicated to the task erupted in arguments about the consequences of posting images of potential suspects that had not been verified by the investigators on the case. In one case, a Web sleuth flagged a “suspicious” eBay user who had purchased two pressure cookers similar to those used in the Boston attack.
Perhaps the most high-profile act of misguided sleuthing appeared on the front page of the New York Post on Thursday. The newspaper published a photo of two men in a crowd, claiming federal investigators were searching for them. Ultimately, the two men were not mentioned in the FBI’s press conference announcing the suspects in the case. Instead, a senior FBI official made a point of mentioning the dangers of publishing images not released by investigators.
Now, having made a public appeal for information, the FBI is likely to focus on the time-consuming work of collecting as much information as possible about the perpetrators – and handling public pressure to do so as quickly as possible.
“This is not an NCIS episode,” Ms. Napolitano said. “Sometimes you have to take time to properly put the chain together to identify the perpetrators.”