Israel's security agencies: technology, integration and 'Big Data'

Some news reports appeared recently in Israel that read almost like press releases from a high-tech company. But these reports weren't about any high-tech company. They were about Israel's secret anti-terror agency, Shin Bet.

The Shin Bet's role in Israel is to prevent terror inside Israel. It's Israel's Internal Security Agency (ISA). These news reports spoke about recent Shin Bet accomplishments, the most important of which (at least, for Israeli civilians) might be that, year-to-date 2018, the Shin Bet has thwarted some 250 terror attacks (arutzsheva, here).

For us ordinary folk, that's good news. It suggests our Shin Bet hasn't been sleeping on the job. It's been working so we can sleep.  

The surprising part of these reports was how un-police-like they seemed. For example, one Shin Bet official sounded more high-tech than hound dog. He said, "As a learning, integrating, advanced, and technological organization, we've emphasized the value of integration and the value of innovation [emphasis mine]" (ibid). 

The Shin Bet was also quoted as describing how it's been able to reduce 'lone wolf' attacks. This statement wasn't about police-work. It was about using "big data abilities" (jerusalempost, here). 

The Shin Bet adapts itself to new "technological, intelligence and operational tools [emphasis mine]" (timesofisrael, here). Clearly, today's Shin Bet isn't like your father's terror-fighter. It's changed.  

This internal security agency is no stand-alone operation. The entire Israeli security apparatus has become a kind of high-tech intelligence/security cooperative. It revolutionizes the world of intelligence and surveillance. 

To get a glimpse of how Israel's intelligence community has become a high-tech, innovation-oriented organization (as suggested by the quotes above), consider what's happened at Israel's Northern border with Lebanon, where Israel's enemy Hezbollah sits like a wolf bidding its time before attacking. 

At Israel’s Northern border, IDF intelligence units have used a series of techniques to gather intelligence. To a great extent, the IDF has used individual surveillance operators, mobile surveillance resources and air resources to meet its surveillance goals.  

  

It appears that, before 2009, the main intelligence-gathering technique for the IDF at its Northern border was, essentially, optics-based. This meant having individual soldiers use optics to gather intelligence.

We infer from sources that, pre-2009, visually-acquired intelligence reigned supreme for the IDF at the Northern border (here). Using this elementary technology, a surveillance operator would scan a sector multiple times during his shift. Each scan took several minutes to complete, which meant that, as each point within the sector was scanned, the other points in the sector remained, by definition, un-scanned. This process created gaps in surveillance which, at least in theory, meant the enemy could catch the IDF ’blind’, and move about undetected in those temporarily un-scanned spots.

In 2009, the IDF introduced a new intelligence technology to its Northern border—a new Radar system. This technology did its surveillance more thoroughly than individual optic operatives—and faster.

Radar represented a major leap forward for surveillance. It changed the operational concept of intelligence gathering (arirusilia, ibid). It certainly made data collection more complete. It also Ieft fewer gaps in the data-gathering process. Fewer gaps meant better intelligence.

IDF radar technology has gotten better. For example, the IDF introduced radar that can see through foliage—all the better to watch Hezbollah movements.

Still more technology went to the border. This refers to something called, the MARS system. This technology uses multiple surveillance cameras for different ranges and with different resolution characteristics. It monitors a very broad sector on multiple levels.

These technologies work quickly--and are getting faster. They scoop up masses of data. So much data flooding into an intelligence HQ has the potential to confuse analysts, not help them. This is where, I suggest, the technology known as ‘Big Data ability’ comes to bear.

‘Big Data’ is relatively new. It may still be, in 2018, cutting edge. It uses a new computer science concept called, parallel programming theory. It's a way to make sense of what is called, ‘oceans of information’—and to do that quickly.  It can uncover hidden patterns and unseen correlations (searchbusinessanalytics, here), two advantages that could help terror-fighting enormously. 

The only way to make these different technologies (the ‘optical system’, radar, foliage-penetrating radar, MARS and 'Big Data', among others) work together to enhance intelligence analysis is to integrate them. This means that all of these technologies are blended together so that when, for example, a radar system finds something of interest in a sector, two things can happen quickly. First, the optical operators can be alerted to play closer attention to that sector and, second, one or more of the remaining radar systems (and other data gathering technologies) can be tuned into that part of the sector to take a series of ever-closer looks from different angles, different ranges and different levels of detail.

The IDF calls this integration process, “Intelligence in Context” (arirusila, ibid). I believe the IDF may use some form of  'Big Data' analytics to analyze data collected to create a ‘unified status picture’ (ibid). This ‘big picture’ attempts to make sense of the data. When such a single picture takes shape, battalion decision-making becomes sharper and better-informed. Indeed, even when a newly-created 'unified status picture' doesn’t tell field commanders where exactly an enemy is situated, it can guide field forces simply to ‘look over there’ (ibid). 

Sometimes that's enough. Almost always, it's better than anything that was available in 2009.

This report is just the tip of a very large intelligence 'iceberg'. But it suggests how Shin Bet can use technology, 'Big Data analytics' and the process of integration to solve its own intelligence challenges.      

The better able Shin Bet is to address its terror challenges through technology, Big Data analytics, innovation and integration, the safer we will be. That's good for Israel.