Sep 13, 2008

DNI Open Source Conference 2008

I attended the DNI Open Source Conference yesterday but left right after the keynore, as soon as I realized that "Open Source Intelligence" is not at all what we mean by "Open Source Software". We mean pipes. They mean contents. And I find DNI's meaning deeply disturbing.

Part of it was the keynote speaker's "double humped camel" analogy where the gap between humps was the budget cuts of the 1990s. He followed with a "moment of silence for 9/11 victims" which I realized was a triumphant celebration of his camel's second hump in politically-correct disguise.

DNI's meaning of "open source" is basically anything that's not nailed down (as distinct from "closed source" which is). He alluded to that meaning in "there’s real satisfaction in solving a problem or answering a tough question with information that someone was dumb enough to leave out in the open".

He's not talking about software. He's talking about sifting thru mountains of irrelevant information about people's daily lives to draw half-baked conclusions from anything they find there. Airline records. Credit card records. Speed cameras. Street cameras. Anything that's not nailed down.

And that scares the bejezus out of me. Obviously because of the police state implications but also because of the dubious quality of this information. Yes, its free, and worth every penny. Arithmetically better refining of exponentially lower-quality data is just not as effective as putting boots on the ground to develop quality intel resources.

Didn't 9/11 teach us what comes from relying on high-tech SIGINT at the expense of low-tech HUMINT? Especially with even more second-hump resources to get in each other's way?

Grumpf. Oh my country.

10 comments:

das said...

I think you're still misunderstanding what open source is in the context of intelligence, and drawing unwarranted conclusions.

"Airline records. Credit card records. Speed cameras. Street cameras."

Those are not examples of open sources, and he is not at all talking about sifting through that information in this context! Those are records that, when obtained in relation non-US Persons outside the United States, still represent closed sources and covert collection. Now, if someone misconfigures their systems and leaves such data openly accessible, then it is an "open source", but that's not really what we're talking about here.

The other issue is that when we talk about sifting through that information (this is outside of the open source discussion), there is a massive distinction between such collection with regard to US Persons and foreign individuals outside of the United States. You talk of a "police state", assuming that a particular collection technique is to be applied domestically, while forgetting the largely foreign mission of our intelligence agencies.

Further, you seem to be confusing covert SIGINT and data-mining mountains of data obtained via SIGINT with OSINT. There can be overlap, but OSINT only represents data collected from OPEN sources. The publicly-accessible (free or commercial) internet, the airwaves, media, and so on. OSINT can in fact overlap with any other INT.

This discussion isn't about SIGINT — or anything else — supplanting HUMINT. It's about open sources being the source of first resort, and used to refine, target, or verify information obtained by the other INTs, which should all be working in concert.

Additionally, much of the OSINT movement within the US Intelligence Community also uses open source software to manage it. For example:

Intellipedia uses MediaWiki
Intelink Blogs uses WordPress
Intelink Forums (before they were shuttered in favor of other mechanisms to manage the content) used phpBB
Intelink's instant messaging system uses Jabber (XMPP)
...and I could go on.

There are quite a number of people in the Intelligence Community who value open source software, and the low cost and ease of deployment and management of open source software often intersects quite nicely with those working on open source intelligence.

Just so there is no misunderstanding about what's going on here, I think readers may do well to read the transcript of actual keynote for themselves. It's too bad that you left the conference which thousands of people were turned away from; you might have actually gotten an understanding about open source intelligence. Then again, I must admit I'm a bit puzzled how you came away with what you did from the opening address...

Brad Cox said...

Is it a "misconfiguration" that my traffic (as a US citizen) traverses AT&T servers that DNI/NSA/whoever can tap on the grounds that non-citizens use them?

And you didn't really address my main concern, that this smacks of police state tactics to gain reams of very poor quality intel.

das said...

No, that's not a "misconfiguration", but that is also not open source intelligence. Open source intelligence represents unclassified collection methods from open, publicly accessible sources. The idea here is rooted in the notion that upwards of 80% — some say 90% — of the information needed to inform policy decisions can be obtained from open sources.

The monitoring issue is another issue altogether, and has obviously been steeped in controversy to varying degrees.

It was never as clear cut as being "legal" or "illegal". Court decisions have cast doubt on some activities, but those who claimed that it was clearly "illegal" before any such decisions were rendered did so largely for political reasons. The other mistake is equating "traffic that *could be* listened to" with "traffic that *is* listened to" — unfortunately, they are not at all the same. This also ignores that to even determine whether traffic is subject to legal collection, it must — to be blunt — actually be able to be collected.

Having the capability to instantaneously examine traffic of international origin, where one or both endpoints of a communication are international, necessitates such wholesale monitoring capability. However, such capability being present does not imply its use for all traffic.

There are two issues here:

1. Monitoring the contents of a communication

2. Monitoring the metadata or "envelope" (source and destination information) of a communication

The first is allowable without a warrant or court oversight when one or both endpoints of the communication are international, and when the target of such monitoring is a non-US Person outside of the United States. Such foreign signals intelligence collection does not require a warrant or court oversight.

The second point above has multiple functions. One is using advanced data mining techniques to look for troubling patterns in communications.

he other critical function of the second point above is determining whether one or both endpoints are international in origin, and is related to the support of initial and ongoing monitoring of a non-US Person outside of the United States. This is important.

Federal law and executive order define a US Person as:

- a citizen of the United States
- an alien lawfully admitted for permanent residence
- an unincorporated association with a substantial number of members who are citizens of the US or are aliens lawfully admitted for permanent residence
- a corporation that is incorporated in the US

In order to reasonably determine that one or both ends of a communication is outside the United States and to be able to affirm that one or both international ends also do not constitute US Persons, with any degree of certainty as is required by law, having such communication metadata available for examination in support of foreign intelligence collection is critical.

What FISA covers is foreign intelligence collection within the US, traditionally on foreign persons or entities within the US. The difference today is that increasing levels of traffic that ordinarily are not subject to court oversight travel physically through the United States — but even that traffic is beginning to trend away from the US.

he now-sunset Protect America Act of 2007 explicitly allowed such monitoring, but in its haste made some provisions too ambiguous. Democratic Congressional leaders proposed the RESTORE Act, now known as the FISA Amendments Act of 2008 (HR 3773), sponsored by Rep. John Conyers (D-MI) and cosponsored by 7 other Democratic colleagues, which:

1. Clarifies that no court warrant is required to intercept communications of non-United States Persons when both ends of the communications are outside the United States. (Even when the interception occurs within the US.)

2. Requires an individualized court warrant from the FISA Court when targeting persons in the United States. (Same as current law.)

The FISA court will review targeting and so-called "minimization" procedures for such surveillance. But to be clear, this bill explicitly reiterates that no warrant is required for the monitoring of communications where the target is a non-US Person outside of the United States — even when the interception happens within the United States. The FISA Amendments Act of 2008 has been passed by the House and Senate, and is now playing amendments tag to resolve some final differences.

While it seems that some are convinced that such collection is clearly illegal or unconstitutional, the facts are that monitoring communications of non-US Persons outside of the US is allowable under the Constitution and under the law, albeit with some legal gray areas, some of which are being resolved in current legislation. However, the mechanisms must necessarily be in place to support such legal collection of data traffic.

Having mechanisms in place to monitor such traffic is of paramount importance. The prospect of traffic shifts away from the United States is worrying from an intelligence perspective.

All of this has nothing to do with open source intelligence, of course. The methods used to gain open source intelligence are unclassified, and the information obtained is the result of collection from open sources. Neither of these fundamentals are shared with covert/classified SIGINT monitoring. But even those capabilities are more important that you might imagine, and aren't intended to supplant other disciplines, e.g., HUMINT.

das said...

No, that's not a "misconfiguration", but that is also not open source intelligence. Open source intelligence represents unclassified collection methods from open, publicly accessible sources. The idea here is rooted in the notion that upwards of 80% — some say 90% — of the information needed to inform policy decisions can be obtained from open sources.

The monitoring issue is another issue altogether, and has obviously been steeped in controversy to varying degrees.

It was never as clear cut as being "legal" or "illegal". Court decisions have cast doubt on some issues, but those who claimed that it was clearly "illegal" before any such decisions were rendered did so largely for political reasons. The other mistake is equating "traffic that could be listened to" with "traffic that is listened to" — unfortunately, they are not at all the same. This also ignores that to even determine whether traffic is subject to legal collection, it must — to be blunt — actually be able to be collected.

Having the capability to instantaneously examine traffic of international origin, where one or both endpoints of a communication are international, necessitates such wholesale monitoring capability. However, such capability being present does not imply its use for all traffic.

There are two issues here:

1. Monitoring the contents of a communication

2. Monitoring the metadata or "envelope" (source and destination information) of a communication

The first is allowable without a warrant or court oversight when one or both endpoints of the communication are international, and when the target of such monitoring is a non-US Person outside of the United States. Such foreign signals intelligence collection does not require a warrant or court oversight.

The second point above has multiple functions. One is using advanced data mining techniques to look for troubling patterns in communications.

The other critical function of the second point above is determining whether one or both endpoints are international in origin, and is related to the support of initial and ongoing monitoring of a non-US Person outside of the United States. This is important.

Federal law and executive order define a US Person as:

- a citizen of the United States
- an alien lawfully admitted for permanent residence
- an unincorporated association with a substantial number of members who are citizens of the US or are aliens lawfully admitted for permanent residence
- a corporation that is incorporated in the US

In order to reasonably determine that one or both ends of a communication is outside the United States and to be able to affirm that one or both international ends also do not constitute US Persons, with any degree of certainty as is required by law, having such communication metadata available for examination in support of foreign intelligence collection is critical.

What FISA covers is foreign intelligence collection within the US, traditionally on foreign persons or entities within the US. The difference today is that increasing levels of traffic that ordinarily are not subject to court oversight travel physically through the United States — but even that traffic is beginning to trend away from the US.

The now-sunset Protect America Act of 2007 explicitly allowed such monitoring, but in its haste made some provisions too ambiguous. Democratic Congressional leaders proposed the RESTORE Act, now known as the FISA Amendments Act of 2008 (HR 3773), sponsored by Rep. John Conyers (D-MI) and cosponsored by 7 other Democratic colleagues, which:

1. Clarifies that no court warrant is required to intercept communications of non-United States Persons when both ends of the communications are outside the United States. (Even when the interception occurs within the US.)

2. Requires an individualized court warrant from the FISA Court when targeting persons in the United States. (Same as current law.)

The FISA court will review targeting and so-called "minimization" procedures for such surveillance. But to be clear, this bill explicitly reiterates that no warrant is required for the monitoring of communications where the target is a non-US Person outside of the United States — even when the interception happens within the United States. The FISA Amendments Act of 2008 has been passed by the House and Senate, and is now playing amendments tag to resolve some final differences.

While it seems that some are convinced that such collection is clearly illegal or unconstitutional, the facts are that monitoring communications of non-US Persons outside of the US is allowable under the Constitution and under the law, albeit with some legal gray areas, some of which are being resolved in current legislation. However, the mechanisms must necessarily be in place to support such legal collection of data traffic. The prospect of traffic shifts away from the United States is worrying from an intelligence perspective.

But all of this has nothing to do with open source intelligence. The methods used to gain open source intelligence are unclassified, and the information obtained is the result of collection from publicly accessible sources. Neither of these fundamentals are shared with covert/classified SIGINT monitoring. But even those capabilities are more important that you might imagine, and aren't intended to supplant other disciplines, e.g., HUMINT.

Bonny said...

Yeah, Open Source Intelligence is distinct from open source software.

And Das is right -- open source intelligence comes from open sources (e.g. newspapers on the Internet), not closed sources. But I'm just an anonymous Internet person, so you probably don't believe me either. Oh well.

Brad Cox said...

das and I kicked this to death via chat this am without either side convincing the other. He trusts govt to do the right thing and I find what what they're doing deeply disturbing. What it is or isn't "open source" is peripheral to that, which boils down to an issue of trust.

deichmans said...

Accusing the U.S. government of "police state tactics" is a bold claim, especially when there are laws that prevent the abuse of intelligence systems. I recommend you review Executive Order 12333, in particular Section 2.

The professionals in the intelligence community take "Intelligence Oversight" very seriously. If you have evidence of abuse of position or authority, I urge you to bring it to the attention of your Congressional representatives and the Attorney General.

However, if you don't, then I ask that you ease up on the rhetoric.

Brad Cox said...

deichmans, its precisely those "executive orders" that scare me so much, and the very reason I called this a "police state". That's not mere rhethoric, its precisely the way I (and most of the world now it seems) actually see it.

Jesse said...

Brad - Then your argument is that you don't trust the government, and that's as far as it goes. And that's fine. We should NEVER give government a warrant to do anything it wants. But because you don't trust the government doesn't mean you should make claims about what the government is doing without having researched the issue and the rules the IC operates under. What Das said is exactly correct. The IC is extremely strict (read the Intelligence Oversight requirements!) on what can be collected when it comes to US persons. The IC is not using open sources (e.g., newspapers, radio, etc.) to condcut research Americans. They are using it to learn about foreign countries, their leaders, intentions, etc. The mission of the IC is to bring understanding to policymakers so they can make better decisions. That is a noble mission, that does not involve spying on Americans.

Brad Cox said...

Jesse, I have researched the issue; that's exactly what's causing the concern. Not how seriously intelligence agencies take the laws, but the laws themselves. All this compounded every way you look, from border security confiscating laptops to TSA confiscating nail clippers. All things that annoy honest citizens will having no discernable impact on bad guys who are likely to travel on container ships anyway. Yes, I'm aware of the bill to reign in the laptop snoopers, and find it small comfort that that bill is even needed. Remember, Naziism was perfectly legal too. That didn't make it right.