1300 222 547
Geeks Crack Spy Trade
For anyone interested in the sophisticated art of computer forensics and analysis, whether you work for a government agency or a business, an article in The Wall Street Journal on September 4 had particular relevance. One of the Journal’s writers, Siobhan Gorman, revealed a major technological breakthrough, one that intelligence insiders claim is the world’s most effective analytical tool for investigating terrorist networks. And it’s come from a virtually unknown software start-up.
Based in Silicon Valley, Palantir Technologies has created a user-friendly search tool that can scan multiple data sources at once, something that previous search tools couldn’t do. That means an analyst who is following a tip about a planned terror attack, for example, can more quickly and easily unearth connections among suspects, money transfers, phone calls and previous attacks around the globe. Palantir’s software has helped root out terrorist financing networks, revealed new trends in roadside bomb attacks, uncovered details of suicide bombing networks and discovered a spy infiltration of an allied government, according to current and former US officials familiar with the events. It is now being used by the CIA, the Pentagon and the FBI.
Yet Palantir remains an outlier among government security contractors. It rejected advice to hire retired generals to curry favour with the agencies and hired young government analysts frustrated by working with slow-footed technology. The company’s founders knew little about intelligence gathering when they started out. Instead, they went on a fact-finding mission, working with analysts to build the product from scratch. The technology they’ve produced is increasingly valuable to spies confronting an information explosion, where terrorists can hide communications in vast data streams on the Internet. A former US Assistant Secretary of Defense has said that this is a new way of war fighting.
Spy agencies like the CIA and military intelligence organizations have hundreds of databases each, most of which aren’t linked up. A single database might contain reports from field agents or lists of known terrorists or companies thought to be financing terrorism. To conduct an investigation, analysts have to query individual databases separately, then try to make sense of the data, frequently with pen and paper. With many of the existing search tools, analysts also can’t access some files on suspects or other threats because a bit of data in the file is classified at a level higher than they are allowed to see. That is a problem, because making connections between new clues and existing data is a key to foiling terrorist plots.
Palantir’s software plugs these gaps by using a “tagging” technique similar to that used by the search functions on most Web sites. Palantir tags, or categorizes, every bit of data separately, whether it is a first name, a last name or a phone number. That means if only one piece of data in a file is classified top-secret, an analyst with a lower level of clearance can still see the rest of the data. It also allows analysts to quickly tag information themselves as it arrives in the form of field reports from spies overseas, and to see who else in the agency is doing similar research so they can share their findings. Some experts say Palantir’s strength is in helping analysts draw inferences when confronted with an enormous amount of disparate data.
In the past two years, the firm’s work in Washington has expanded from eight pilot programs to more than 50 projects. The Australian government is now a client, and the NSA is eyeing Palantir, as is the UK, current and former government officials say. The company expects to turn a profit from its government work this year, and for revenues to reach $US100 million within two years. It recently started working with financial companies.
As he builds up his US East Coast office, Palantir’s 41-year-old chief executive, Alexander Karp, says he is still figuring out “how to live in those two worlds” of Silicon Valley and Washington. Rival software contractors seem disturbed by Palantir’s expansion, dismissing it as the “next sexy thing”. They argue that it won’t be able to make it in the government contracting business.
Whether it does or it doesn’t, it is clearly a valuable new tool in the world of forensic analysis and investigation and has a fascinating range of applications.
