al

Interesting paper: "The value of Online Privacy," by Scott Savage and Donald M. Waldman.

Abstract: We estimate the value of online privacy with a differentiated products model of the demand for Smartphone apps. We study the apps market because it is typically necessary for the consumer to relinquish some personal information through "privacy permissions" to obtain the app and its benefits. Results show that the representative consumer is willing to make a one-time payment for each app of $2.28 to conceal their browser history, $4.05 to conceal their list of contacts, $1.19 to conceal their location, $1.75 to conceal their phone's identification number, and $3.58 to conceal the contents of their text messages. The consumer is also willing to pay $2.12 to eliminate advertising. Valuations for concealing contact lists and text messages for "more experienced" consumers are also larger than those for "less experienced" consumers. Given the typical app in the marketplace has advertising, requires the consumer to reveal their location and their phone's identification number, the benefit from consuming this app must be at least $5.06.

Interesting analysis, though we know that the point of sale is not the best place to capture the privacy preferences of people. There are too many other factors at play, and privacy isn't the most salient thing going on.

by Chris Borgen

All Things Considered ran an interview this past Monday with Alex Fowler, the chief privacy officer of Mozilla (developer of the Firefox web browser), stemming from a blog post Fowler had written critiquing President Obama’s speech last week concerning NSA activities. When asked about the “most glaring reform needs” that were not addressed in the President’s speech, Fowler said:

right now, we have a policy approach in Washington which is focused on not closing security holes but actually [on] hoarding information about security backdoors and holes in our public security standards and using those then to exploit them for intelligence needs. In our perspective, and I think certainly those of your listeners – as you think about the news related to Target data breaches and breaches with Snapchat and other common tools that we use every day – that what we really need is to actually focus on securing those communications platforms so that we can rely on them. And that we know that they are essentially protecting the communications that we’re engaged with.

This relates to the market for so-called “zero-day exploits,”  where the U.S. government pays hackers for information about holes in software security that its intelligence and law enforcement agencies can then use for surveillance. (The market for zero-day exploits is described in greater detail in this previous post.) The U.S. also pays the sellers of these exploits to keep the holes secret, not even warning the company that has the security hole, so that the exploit may remain useful to the U.S. government for as long as possible. Unfortunately, this also means it will remain open for criminal hackers who have also discovered the hole.

The injection of U.S. government funds has transformed a formerly loose, reputation-based, market into a lucrative global bazaar with governments driving up prices and the formation of firms with business models based on finding and selling exploits to the U.S. and other governments. Although cash-rich companies like Microsoft are responding by trying to out-bid state actors for information about zero day exploits in their own products, the money in the market has shifted from rewarding security into incentivizing insecurity:

…nearly everyone seems to be in the market for zero-days; a report earlier this year claimed that the U.S. government is the biggest buyer of zero-day vulnerabilities. Even the NSA contracts with zero-day exploit vendors like the French firm Vupen Security. In fact, Professor Ross Anderson, of the University of Cambridge, previously told TechWeekEurope that “researchers are purposefully placing bugs in open source software during the development stages, so that when code appears in completed products, those same researchers can highlight the flaws and profit from them where companies are willing to pay.”

As mentioned in my previous post on the exploits market, there have been calls for regulation.  The Internet Governance Project has written about one possible form for such regulation:

We suggest focusing policy responses on the demand side rather than the supply side. The zero-day market is largely a product of buyers, with sellers responding to that demand. And if it is true that much of the demand comes from the US Government itself, we should have a civilian agency such as DHS compile information about the scope and scale of our participation in the exploits market. We should also ask friendly nations to assess and quantify their own efforts as buyers, and share information about the scope of their purchases with us. If U.S. agencies and allies are key drivers of this market, we may have the leverage we need to bring the situation under control.

One idea that should be explored is a new federal program to purchase zero-day exploits at remunerative prices and then publicly disclose the vulnerabilities (using ‘responsible disclosure’ procedures that permit directly affected parties to patch them first)…

In other words, instead of engaging in a futile effort to suppress the market, the US would attempt to create a near-monopsony that would pre-empt it and steer it toward beneficial ends. Funds for this purchase-to-disclose program could replace current funding for exploit purchases.

[Emphases added.]

It’s an interesting idea that uses a combination of regulation and the U.S. dominant position in the market to set new norms, the most important of which would be the U.S. disclosing the vulnerabilities it purchases. This would essentially reverse the goal of the purchases from keeping exploits secret and unpatched to publicly disclosed and fixed. The latter would allow the affected company to increase its security and consumers to beware or possible loss of privacy/ data.

But what would be the government incentive to pay money to do this?  Couldn’t this be left to the companies themselves to purchase exploits in order to fix their own systems; in other words, the market as it existed before the large-scale entrance of the U.S. government? The problem is that the entrance of the U.S. into the market has brought other governments into the bidding as well. It may not be easy, or as cheap, for companies to defensively purchase exploits, even without the U.S. bidding up the price. China, Russia, or some other countries may be there to outbid private actors. So there may well be a need for U.S. government action in the exploits market to help secure systems.

Moreover, the IGP writers noted the possibility of coordinating with other countries, This is a point worthy of emphasis. Although the U.S. government is, for the moment, seemingly the biggest purchaser of exploits, multilateral cooperation may be crucial for effective market regulation.  A group of states acting together could have much broader reach in protecting the data of their companies and the privacy of their citizens. However, it would make spying by the NSA and other such agencies more difficult.

Right now, the U.S. cyberstrategy is largely offensive: finding as many ways to scoop up data and exploit target computer systems as possible.  What the IGP and others have suggested is switching to a defensive strategy: trying to make the internet as secure as possible.  Some have balked that robust data security on the internet is not feasible given its architecture, and so offense has to be the default strategy.  I am not a computer scientist and I don’t know how to resolve that debate.

I do understand that the President would not want to discuss the exploits market in his surveillance speech, particularly when the U.S. is itself the main market actor on the demand side.

But,  as we have an ongoing public debate about our cyberstrategy in the coming weeks, we need to  consider the costs and benefits of pursuing a primarily offensive or defensive strategy (or various combinations of the two). And, assessing such strategies will require addressing  the hackers’ bazaar of zero-day exploits, and not passing it by in silence.

This makes a lot of sense.

Viviane Reding dismissed recent fines for Google as "pocket money" and said the firm would have had to pay $1bn under her plans for privacy failings.

Ms Reding said such punishments were necessary to ensure firms took the use of personal data seriously.

And she questioned how Google was able to take so long to getting round to changing its policy.

"Is it surprising to anyone that two whole years after the case emerged, it is still unclear whether Google will amend its privacy policy or not?" she said in a speech.

Ms Reding, who is also vice-president of the European Commission, wants far tougher laws that would introduce fines of up to 5% of the global annual turnover of a company for data breaches.

If fines are intended to change corporate behavior, they need to be large enough so that avoiding them is a smarter business strategy than simply paying them.

All of the recent news about Target, Neiman Marcus, and other businesses being hacked might be a surprise to many but it’s no surprise to us. Truth is that practice of security has devolved into a political image focused designed satisfy technically inept regulatory requirements that do little or nothing to protect critical business assets. What’s worse is that many security companies are capitalizing on this devolution rather than providing effective solutions in the spirit of good security. This is especially true with regards to the penetration testing industry.

We all know that money is the lifeblood of business and that a failure to meet regulatory requirements threatens that lifeblood. After all, when a business is not in compliance it runs the risk of being fined or not being allowed to operate. In addition the imaginary expenses associated with true security are often perceived as a financial burden (another lifeblood threat). This is usually because the RoI of good security is only apparent when a would-be compromise is prevented. Too many business managers are of the opinion that “it won’t happen to us” until they become a target and it does. These combined ignorant views degrade the overall importance of real security and make the satisfaction of regulatory requirements the top priority. This is unfortunate given that compliance often has little to do with actual security.

Most regulatory requirements are so poorly defined they can be satisfied with the most basic solution. For example PCI-DSS requires merchants to undergo regular penetration tests and yet it completely fails to define the minimum level of threat (almost synonymous with quality) that those tests should be delivered at. This lack of clear definition gives business owners the ability to satisfy compliance with the cheapest most basic of testing services. To put this into perspective, if the standards used to test bulletproof vests (NIJ and HOSDB test methods) were replaced by PCI–DSS then bulletproof vest testing could be satisfied with a squirt gun.

These substandard regulatory requirements combined with business owners lacking in true security expertise formed a market where exceedingly low quality, low-threat, easy to deliver security-testing services are in high-demand. This market has been answered by a horde of self-proclaimed security experts that in almost all cases are little more than marginally capable script-kids and yet they inaccurately market their services as best in class. Take away their third party tools (scripts, scanners, Metasploit, etc.) and those vendors will be dead in the water. Take the tools away from a bona fide researcher or hacker and they’ll write new tools then hack you with a vengeance.

The saturation of the penetration testing industry with charlatans makes the process of identifying a quality vendor difficult for business managers that actually care about security. In many cases the consumer is a non-technical (or non-security expert) buyer and not able to truly assess the technical capabilities of the vendor. As a result they often make buy decisions based on the non-technical exploration of things like the number customers serviced, annual revenue, size of company, etc. While these are important factors when evaluating any business, they are by no means a measure of service quality and testing vendor capability. With regards to penetration testing services, quality of service is of the utmost importance and it is a highly technical matter. This is why we wrote a guide to vendor selection that sets a standard of quality and was featured on Forbes.

It is unfortunate that most business owners don’t seem to operate in spirit of good security but instead operate with revenue focused tunnel vision. The irony of this is that the cost of a quality penetration test is equal to a small fraction of the cost of a single successful compromise. For example, in 2011 Sony suffered a compromise that resulted in over 170 million dollars in damages (not including fines). This compromise was the result of the exploitation of a basic SQL Injection vulnerability in a web server (like Target). The average cost of Netragard’s web application penetration testing services in 2013 was $14,000.00 and our services would have detected the basic SQL Injection vulnerability that cost Sony so much money. Perhaps its time to rethink the value of genuine penetration testing? Clearly genuine penetration testing has a positive revenue impact through prevention. Specifically the Return on Investment of a genuine penetration test is equal to the cost in damages of a single successful compromise.

So what of Target.

We know that target was initially compromised through the exploitation of a vulnerability in one of their web servers (just like Sony and so many others). This vulnerability went unidentified for some time even after the initial malicious compromise. Why did malicious hackers find this vulnerability before Target? Why was Target unaware of their existing paths to both compromise data exfiltration? Who determined that Target was PCI compliant when PCI specifically requires environmental segregation and Target’s environment was clearly not properly segregated?

A path to compromise is the path that an attacker must take in order to access sensitive information. In the case of Target this information was cardholder data. The attackers had no issue exploiting a path to compromise and propagating their attack from their initial point of compromise to the Point of Sale terminals. The existence of the path to compromise should have resulted PCI failure, why didn’t it?

A path for data exfiltration is the method that an attacker uses to extract data from a compromised network. In the case of Target the attackers were able to extract a large amount of information before any sort of preventative response could be taken. This demonstrates that a path for data exfiltration existed and may still exist today. As with the path to compromise, the path for data exfiltration should have resulted in a PCI failure, why didn’t it?

We also know that Target’s own security monitoring capabilities were (and probably still are) terrible. Based on a Krebs on Security article, the hackers first uploaded malware to Targets points of sale. Then they configured a control server on Target’s internal network to which the malware would report cardholder data. Then the hackers would login to the control server remotely and repeatedly to download the stolen cardholder data. Why and how did target fail to detect this activity in time to prevent the incident?

If we use the little information that we have about Targets compromise as a light-weight penetration testing report we can provide some generic, high-level methods for remediation. What we’re suggesting here is hardly a complete solution (because the full details aren’t known) but it’s good advice nonetheless.

• Deploy a web application firewall (ModSecuirty or something similar). This would address the issue of the web application compromise (in theory). If one is already deployed then it’s in need of a serious reconfiguration and whoever is charged with its monitoring and upkeep should be either trained properly or replaced (sorry). Most web application vulnerabilities are not exploited on the first try and their exploitation often generates significant noise. That noise likely went without notice in the case of Target.

 

• Deploy a host-based monitoring solution on public facing servers. This would further address the issue of web server security and would help to prevent distributed metastasis. For companies that want an open source solution we’d suggest something like OSSEC with the appropriate active response configurations. For example, tuning OSSEC to monitor and react to system logs, web application firewall incidents, network intrusion incidents, etc. can be highly effective and its free. OSSEC can also be used to build blacklists of IP addresses that continually exhibit hostile behavior.

 

• Deploy a network intrusion prevention solution (like snort) on the internal network at the demarcation points of the cardholder environment. If done properly this would help to block the path to compromise and path for data exfiltration. This solution should be tuned with custom rules to watch for any indication of cardholder data being transmitted out over the network. It should also be tuned to watch for anomalous connections that might be the product of rootkit’s, etc. In the event that something is detected it should respond in a way that ensures that the affected source and targets are isolated from the rest of network.

 

• Deploy a second host based solution like bit9 on the Points of Sale (PoS). (No we’re not a reseller and have no affiliation with bit9). This solution should be deployed on the points of sale assuming that bit9 will run on them. This will address the issue of malware being deployed on the points of sale and being used to steal credit card information. Especially malware that was first created in 2013 (BlackPOS).

 

• Hire a real Penetration Testing vendor. Given what we know about this compromise, Target hasn’t been selecting penetration testing vendors using the right criteria. Any genuine penetration testing vendor that delivers quality services at realistic levels of threat would have identified many of these issues. In fact, its fair to say that following the methods for remediation that come from a genuine penetration test would have prevented the compromise of cardholder data.

 

Finally, who do you think did a better job at testing, the hackers that compromised Target or the Penetration Testing firm that said that they were PCI Compliant?

The post Don’t become a Target appeared first on "We protect you from people like us.".

When I originally described the flexible Qubes Odyssey framework several months ago, I mentioned that we would even consider to use “Windows Native Isolation” mechanisms as a primitive type of isolation provider (“hypervisor”) for some basic edition of Qubes for Windows. The idea has been very attractive indeed, because with minimal effort we could allow people to install and run such Qubes WNI on their normal, consumer Windows laptops.

Sure, the inter-process isolation provided by a monolithic kernel such as Windows or Linux could never be compared to the inter-VM isolation offered even by the most lousy hypervisors. This is simply because the sizes of the interfaces exposed to untrusted entities (processes in case of a monolithic kernel; VMs in case of a hypervisor) are just incomparable. Just think about all those Windows system calls and GDI calls which any process can call and which contains probably thousands of bugs still waiting to be discovered by some kid with IDA. And think about those tens of thousands of drivers, which also expose (often unsecured) IOCTLs, as well as parsing the incoming packets, USB devices infos, filesystem metadata, etc. And then think about various additional services exposed by system processes, which are not part of the kernel, but which are still trusted and privileged. And now think about the typical interface that needs to be exposed to a typical VM: it's “just” the virtualized CPU, some emulated devices (some old-fashined Pentium-era chipset, SVGA graphics adapter, etc) and virtualized memory.

Anyway, knowing all this, I still believed that Qubes WNI would make a whole lot of sense. This is because Qubes WNI would still offer a significant boost over the “Just Windows” default security, which is (still) essentially equivalent to the MS-DOS security model.  And this is a real pity, because Windows OS has long implemented very sophisticated security mechanisms, such as complex ACLs applicable to nearly any object, as well as recent mechanisms such as UIPI/UAC, etc. So, why not use all those sophisticated security to bring some real-world security to Windows desktops!

And, best of all, once people start using Qubes WNI, and they liked it, they could then pretty seamlessly upgrade to Xen-based Qubes OS, or perhaps Hyper-V-based Qubes OS (when we implement it) and their system would look and behave very similarly. Albeit with orders of magnitude stronger security. Finally, if we could get our Odyssey Framework to be flexible enough to support both Qubes WNI, as well as Xen-based Qubes OS, we should then be able to support any hypervisor or other isolation mechanism in the future.

And so we decided to build the Qubes WNI. Lots of work we invested in building Qubes WNI was actually WNI-independent, because it e.g. covered adjusting the core Odyssey framework to be more flexible (after all “WNI” is quite a non-standard hypervisor) as well as some components that were Windows-specific, but not WNI-specific (e.g. could very well be used on Hyper-V based Qubes OS in the future). But we also invested lots of time into evaluating all those Windows security mechanisms in order to achieve our specific goals (e.g. proper GUI isolation, networking isolation, kernel object spaces isolation, etc)...

Sadly this all has turned out to be a story without a happy end, as we have finally came to the conclusion that consumer Windows OS, with all those one-would-think sophisticated security mechanisms, is just not usable for any real-world domain isolation.

And today we publish a technical paper about our findings on Windows security model and mechanisms and why we concluded they are inadequate in practice. The paper has been written by Rafał Wojdyła who joined ITL a few months ago with the main task of implementing Qubes WNI. I think most people will be able to learn a thing or two about Windows security model by reading this paper.

Also, we still do have this little hope that somebody will read the paper and then write to us: “Oh, you're guys so dumb, you could just use this and that mechanism, to solve all your problems with WNI!” :)

The paper can be downloaded from here.

Last month, changes to the Wassenaar Arrangement on Export Controls for Conventional Arms and Dual-Use Goods and Technologies (“Wassenaar Arrangement”) placed “zero-days,” other computer exploits, and potentially more categories of software under this multilateral export control regime. These changes take place following reports that the U.S. government purchases “zero day” computer security vulnerabilities—previously unknown exploits—for use by the NSA’s targeted hacking team. Support for these recent changes has come from policymakers and privacy advocates concerned with keeping exploits and network surveillance tools out of the hands of repressive regimes, economic spies, and other bad actors internationally.

Despite these goals, efforts to restrict distribution of surveillance and network exploitation tools must define the tools under control narrowly enough to leave security research tools and other valuable software unregulated while stemming the proliferation of the targeted software. The Wassenaar Arrangement changes attempt to do so by identifying particular characteristics of software as potentially malicious and subjecting software with these characteristics to export controls. Whether the recent Wassenaar amendments draw the line well remains up for debate.

The Wassenaar Arrangement has 41 participating states including the U.S. It creates uniform “control lists” of dual-use technologies, facilitates information sharing on dual-use transfers, and serves as a consultation mechanism for members on national export policies and denials of export license applications.

The recent changes include adding two new classes of export-regulated software to the dual use provision regulations:

Intrusion software:

“Software” specially designed or modified to avoid detection by ‘monitoring tools’, or to defeat ‘protective countermeasures’, of a computer or network capable device, and performing any of the following:

a. The extraction of data or information, from a computer or network capable device, or the modification of system or user data; or

b. The modification of the standard execution path of a program or process in order to allow the execution of externally provided instructions.

and

IP network surveillance systems

5. A. 1. j. IP network communications surveillance systems or equipment, and specially designed components therefore, having all of the following:

1. Performing all of the following on a carrier class IP network (e.g., national grade IP backbone):

a. Analysis at the application layer (e.g., Layer 7 of Open Systems Interconnection (OSI) model (ISO/IEC 7498-1));

b. Extraction of selected metadata and application content (e.g., voice, video, messages, attachments); and

c. Indexing of extracted data; and

2. Being specially designed to carry out all of the following:

a. Execution of searches on the basis of ‘hard selectors’; and

b. Mapping of the relational network of an individual or of a group of people.

The Wassenaar Arrangement focuses primarily on the transparency and harmonization of national export control regimes but is not a treaty, and therefore is not legally binding. Changes to the Wassenaar Arrangement must be implemented separately by each member state, so the precise impact of the recent additions is not yet clear.  Yet, the changes to the Wassenaar Arrangement propose the first major restrictions on widely used, commercially available software since the introduction of restrictions on encryption products to the Arrangement in 1998.

If implemented in the U.S., the changes to the Arrangement would institute export controls for intrusion software and IP network surveillance systems, requiring certain sellers to obtain licenses from the U.S. Department of Commerce. The U.S. Department of Commerce manages the Export Administration Regulations (“EAR”) that govern exports and transfers of “dual use items” (goods, technology and software) with both civilian and military applications. These EAR regulations currently are not applicable to exploits, intrusion software, or surveillance systems, but will likely change to reflect the new Wassenaar terms.  The 2014 National Defense Authorization Act also has required an interagency group to produce additional policy recommendations to control proliferation of cyber weapons.

The Wassenaar Arrangement changes are already having an impact on companies. VUPEN, a leading zero-day exploit firm and known supplier of exploits to the NSA, announced on its website that, in response to the Wassenaar Arrangement changes, it would restrict exploit sales, supplying only approved government agencies in approved countries.  The firm indicated it considered the newly adopted “intrusion software” restrictions applicable to its products. VUPEN also announced it would automatically exclude countries subject to European Union restrictions and countries subject to embargoes by the U.S. or the United Nations.

The newly introduced definitions of restricted software, particularly of intrusion software, could be interpreted to include an overly wide range of legitimately traded and used network security tools. The new language focuses on whether the targeted items are “designed to avoid security features on a device.” While this limitation may initially seem sensible, common beneficial tools avoid security features in order to, for example, install software without the user’s intercession. Auto-updaters may be one such tool, depending on the details of operation. Some experts have wondered whether anti-virus software would fit the intrusion software definition. Given the ambiguity in the definitions of regulated categories, firms may have to consult attorneys before exporting tools that arguably may be covered.

The new restrictions on software exports also face policy debates similar to previous debates regarding encryption export controls and proposals to regulate the market in computer vulnerabilities more generally. Some may raise objections that these software export controls restrict free speech, as with encryption controls. Exploits can be traded as knowledge (of a vulnerability in a particular software program or system) or as weaponized exploits (code written to exploit said vulnerability). The intrusion software clause intends to apply to weaponized exploits, but will it also restrict the exchange of knowledge of vulnerabilities?

The IP network surveillance system provision raises similar questions. What would this restriction mean for vendors of the “data loss prevention” tools used by many businesses to exercise control over information entering and leaving their networks?

More broadly, the Wassenaar Arrangement has been criticized for its weakness as an international tool. Particularly, its lack of harmonized implementation may allow defectors to access export opportunities adherents have forgone. Additionally, the existence of thriving black markets in vulnerabilities and the easily transmissible nature of such software bring into question the suitability of a traditional export regime for controlling these transient, easily transferrable goods. As countries begin to implement the new Wassenaar restrictions on a national level, the effects of the changes and objections to them should become clearer. We plan to continue to follow this evolution and write about it here.

Bug bounty programs, for the most part, have been the domain of large software vendors and Web companies such as Google, Mozilla, Microsoft, PayPal and Facebook. But some smaller companies are now getting involved, with the latest one to announce a bounty being Wickr, the maker of secure messaging apps for Android and iOS, and the potential payoff is huge: up to $100,000.

Wickr’s bug bounty program is quite similar to the one announced last year by Microsoft. That program offers hackers up to $100,000 for new offensive techniques that can defeat the memory protections in the latest version of Windows. The Microsoft bug bounty program also offers $50,000 to researchers who develop a defensive technique that can stop an existing mitigation bypass.

Wickr is doing something similar, enticing hackers with a payment of up to $100,000 for submitting a new vulnerability “that substantially affects the confidentiality or integrity of user data.” The company also is offering additional cash for a defensive technique, submitted at the same time, that can protect against the new vulnerability. Wickr makes a text messaging app for both Android and iOS that is designed to be secure and protect users’ privacy by shredding deleted files on users’ devices.

“The Wickr Bug Bounty Program is designed to encourage responsible security research in Wickr software. It is impossible to overstate the importance of the role the security research community plays in securing modern software. White-hats, academics, security engineers and evangelists have been responsible for some of the most cutting-edge, eye-opening security revelations to date. Their research speeds the pace of advancing security to the benefit of all. With this program and partnership, we pledge to drive constant improvement relating to the security interests of our users, with the goal of keeping Wickr the most trusted messaging platform in the world,” Robert Statica, co-founder of Wickr, wrote in a blog post announcing the bug bounty program.

Bug bounties have been quite successful for a number of the companies who have established them in recent years, with Google and others attributing the contributions of external researchers to the improved security of their products. In October, Microsoft paid its first bounty to researcher James Forshaw, and the company also recently extended its system to include incident response teams and forensics investigators.

Wickr’s program requires that researchers submit their vulnerabilities privately to the company and not publicly disclose them within three months of the submission. It’s open to anyone, of any age, who isn’t a resident of a country that’s on the United States embargo list, and the rewards will range form $10,000 to $100,000.

Image from Flickr photos of Pascal. 

The workshop kicked off with two talks on OPACITY. OPACITY is a protocol suite with two protocols: ZKM and FS. Both of these are Diffie-Hellman-based authenticated key-agreement protocols, geared toward smartcards, that incorporate some privacy measures. It was developed by the company ActiveIdentity, is apparently supported by the US Department of Defense, and has been standardized.

Marc Fischlin began with a talk about a cryptographic analysis of OPACITY. Marc pointed out that the fact that the protocol is standards-compliant does not imply anything about whether or not it is secure; following a standard is no substitute for a rigorous analysis/proof. (This is clearly true, since the standards don’t contain or imply any particular security definition.)

Both protocols use two rounds, and assume the smartcard holds a long-term, certified public key. The ZKM protocol is a static-ephemeral Diffie-Hellman protocol, where the card reader generates an ephemeral key and the smartcard uses its long-term (“static”) Diffie-Hellman public key to derive a shared key. Interestingly (or surprisingly), there is no authentication from reader to client, and anyone can easily impersonate a reader. The FS protocol is similar, though in this case both the smartcard and the reader have long-term, certified public keys. Here, two ephemeral-static Diffie-Hellman protocols are run to derive shared values, and these are then used in a “cascading” fashion to derive shared keys.

One complicated part of both protocols is that, for efficiency reasons, they allow what is called “persistent binding”: the card reader can cache the public key of a smartcard it has interacted with before. There is also a way for the smartcard to “blind” the certificate it sends, which is supposed to provide some measure of privacy for the smartcard.

In trying to analyze the protocol, Marc found that the security goals were unclear. (This is a general problem when security goals are stated informally, and demonstrates why formal models/definitions are useful even in the absence of a security proof.) Marc and his co-authors used a well-known security definition for authenticated key agreement by Bellare and Rogaway, adapted to capture one-way auhtentication (which is all we can hope for ZKS to provide). Marc was able to prove that ZKM achieves one-way authentication, and that the derived, shared keys are hidden from a passive attacker. FS also satisfies some notion of identity hiding for the card. In both cases, the analysis break down if persistent binding is used (it ws not clear to me whether there is a concrete attack in that case).

The second talk in the session was by Eric Le Saint, who introduced the OPACITY protocol. He noted that OPACITY was designed for protecting contactless transactions between chip-based mobile devices and terminals, where currently no security mechanisms are in place. For this reason, the protocol was designed to be fast: the reader/terminal can pre-process things so that it is ready to sent the first message of the protocol as soon as a card comes into close proximity; the card can then quickly compute and send the second (and final) protocol message. This can all be done in the time it takes to physically swipe a card at a reader, without inconveniencing the user holding the card.

Eric addressed a statement from Marc’s papers to the effect that the security guarantees of OPACITY are rather weak. In response he noted that one-way authentication may be sufficient for the intended applications, and resistance to eavesdropping may also be enough in practice. (Frankly, the justification for this did not make sense to me, but perhaps it was just poor presentation.) He also claims (without any explanation) that the notion of identity hiding achieved by ZKM is considered strong enough for US government applications.

He concluded by stating that he was currently working on an improved version of OPACITY that would achieve stronger security goals. He described the protocol using a high-level block diagram but I was unable to follow the details. I hope he (or someone else) will be attempting the prove security this time around.

The second session was on encryption. Jonathan Trostle kicked things off with a talk about authenticated encryption. Besides the “standard” security goals, he mentioned the idea of misuse resistance which requires, roughly that nonce reuse by the sender only leaks whether the same message is encrypted twice (using the same nonce) but otherwise does not breach privacy or authenticity. Jonathan’s talk focused on “lightweight” authenticated encryption schemes, that aim to minimize network overhead (i.e., ciphertext length) as well as the amount of energy expended per encrypted byte.

One point he made, which is obvious but nevertheless bears repeating (especially in light of the CAESAR competition), is that there is unlikely to be a single “best” authenticated-encryption scheme. The best scheme to use depends on the application (e.g., how many messages will be encrypted using a given key) or the deployment environment (e.g., hardware/software, or the relative cost of computation vs. transmission). It would be useful, therefore, to have a set of algorithms standardized, suited for different uses.

Tom Shrimpton spoke next about format-transforming encryption. (I had heard a previous talk on this topic by his student, Kevin Dyer.) The basic goal here is to make encrypted data “look like” other data so as to prevent it from being filtered using deep-packet inspection (DPI), as several countries do. DPI looks at payload-level information, including the data itself as well as information about what applications are being used. “Regular” encryption does not help here, since the very fact that encryption is being used may cause the packet to be filtered out; more specifically, the DPI may be using a whitelist of allowed applications and/or words. Previous work in this space was slow, inflexible, and lacked empirical validation.

So, the goal is to develop a (private-key) cryptographic transformation that makes arbitrary traffic look like “benign” traffic from some other application. The transformation will take as input a [specification of a] set of strings that the DPI will classify in the desired way, and output ciphertexts that will match the desired classification. This is very flexible, since the desired format can be changed, as needed.

Almost all the DPIs evaluated use regular expressions (or something close to that) to perform filtering. So the specification of strings that will be allowed to pass the DPI corresponds to a regular language, that furthermore can be learned from the traffic that is allowed to pass.

To map ciphertexts to words of the regular language, the key insight is to using ranking, that is, an efficiently computable/reversible map from integers to words in the regular language. (This was shown to be possible by Goldberg and Sipser in 1985. It turns out that this gives exactly the rank of a word in the lexicographic ordering of the words in the language) Given this, the solution is relatively straightforward: encrypt the plaintext as usual to get some ciphertext; view the ciphertext as a series of integers; then map these integers to words of the language. There are, as is to be expected, several practical difficulties that emerge in trying to get this to work in practice, but Tom and his co-authors show that this can be done.

One question I was left with was whether the approach can be applied to DPI based on blacklists. While it is true that the complement of a regular language is still regular, the complexity of ranking depends on the size of the regular language [sic — see below] and so may get too large. (In the Q&A, Tom claimed that everything still works, so I guess I’ll have to look in the paper for details. Update: after reading my original post, Tom clarified that the complexity depends on the size of the DFA for the language, not the size of the language. Since a regular language and its complement have DFAs of the same size, everything works out fine.)

In the final talk of the session, Mike Bond spoke about “Crypto as a service,” i.e., providing crypto for other organizations. (This is something his company, Cryptomathic, is working on.) This raises a number of practical questions, such as backing up keys, updating keys, and achieving key separation. There is also the question of making the crypto API useable (and understandable) to the client.  He gave a walkthrough of a banking application that might use the service. Admittedly, this was a bit outside my expertise.

The Open Technology Institute is currently engaged in a joint project on export controls with Privacy International and Digitale Gesellschaft. The blog post below was published in response to the recent changes announced to the Wassenaar Arrangement. The original blog post is available here.

What is the Wassenaar Arrangement?

The Wassenaar Arrangement on Export Controls for Conventional Arms and Dual-Use Goods and Technologies (the "Wassenaar Arrangement") is a multilateral export control regime in which 41 states participate.

The Wassenaar Arrangement was established on 12 July 1996 in Wassenaar, the Netherlands by 33 founding members to contribute to regional and international security and stability. It is the successor to COCOM, a NATO based group that discussed arms exports to non-NATO states, though the membership today is a lot more broad.

The Wassenaar Arrangement promotes transparency and greater responsibility in transfers of conventional arms and dual-use goods and technologies, complements and reinforces the existing control regimes for weapons of mass destruction and their delivery systems, and uses export controls as a means to combat terrorism. It is not directed against any state or group of states in particular; it aims at harmonising the export regimes of all participating states.

What does it do?

The Wassenaar Arrangement works with two different control lists: the List of Dual Use Goods and Technologies and the Munitions List. The Munitions List contains items designed for military use, including but not limited to items such as tanks and other military armed vehicles, combat vessels and aircraft. The List of Dual-Use Goods and Technologies contains several categories of goods that can both have a military and a civilian use, including electronics, computers, and telecommunications and information security equipment. In addition to the control lists, states also agree on best practices on certain topics as a way in which to harmonise procedures.

Each year both control lists are updated when representatives of the participating states meet at the plenary meeting, which usually happens in December, though technical and policy-related matters are thrashed out in working groups prior to then. Afterwards, a new version of the lists is published on the website of the Wassenaar Arrangement, which is followed by implementation of the new controls at the national level.

Participating states have agreed to maintain national export controls on listed items, so the implementation and interpretation by each state in their national laws is crucial. The decision to allow for or deny the export of any item is the sole responsibility of each participating state.  Such a decision will be made in accordance with national legislation and policies, on the basis of national discretion.

What is the reach of the Wassenaar Arrangement?

Since its establishment in 1996, the Wassenaar Arrangement has grown to include 41 Participating States: Argentina, Australia, Austria, Belgium, Bulgaria, Canada, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Japan, Latvia, Lithuania, Luxembourg, Malta, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Republic of Korea, Romania, Russian Federation, Slovakia, Slovenia, South Africa, Spain, Sweden, Switzerland, Turkey, Ukraine, United Kingdom and United States.

As these participating states have agreed to maintain national export controls on listed items, their export regime will reflect the Wassenaar control lists, although the way in which these are implemented at the national level differs per country.

However, the reach of the Wassenaar Arrangement is not limited to the participating states. The Wassenaar Arrangement undertakes significant efforts to encourage voluntary adherence to its standards by other states as well. China’s transfer controls regarding small arms and light weapons for example are broadly in line with those laid down in the Wassenaar Arrangement.

Countries such as Israel have gone further and have chosen to align their export regime with the Wassenaar Arrangement, although they are not formally a participating state. By referring to the two control lists of the Wassenaar Arrangement in the Israeli Defense Export Control Law, Israel de facto has the same status as signatory countries.

Are new items on the Control Lists automatically controlled in the participating states?

No.

The Wassenaar control lists do not have binding legal force throughout the participating states. Controls have to be implemented in national laws to have effect. It is up to the participating states to decide how they will do this. Depending on the implementation mechanism in the relevant state, it may take a while before new additions to the control lists enter into force as export controls at the national level.
The participating EU member state will, for example, refer to Council Regulation (EC) No 428/2009 of 5 May 2009 (the “Dual-Use Regulation”) in their national laws, which has direct binding legal force throughout the EU member states and does not have to be implemented in national law. The Dual-Use Regulation pulls in dual-use items that are agreed and included within the control lists of the Wassenaar Arrangement, the Missile Technology Control Regime (MTCR), the Nuclear Suppliers’ Group (NSG), the Australia Group and the Chemical Weapons Convention (CWC). The EU has not kept up with the rate at which items contained within the multilateral export control regimes have been updated; a Council Regulation amending the EU dual-use list to reflect changes made to the multilateral control lists throughout 2010, for example, was only enacted across the EU in April 2012. It is unclear when the most recent version of the Wassenaar Dual-Use List will be implemented in the Dual-Use Regulation and thus when effective controls for new additions to the Wassenaar control list will be functional in the EU.

Other countries may refer to a certain version of the Wassenaar control list, such as South Africa, which has included a reference to the 2010 Wassenaar Control Lists in its national export control regime. Until the relevant schedule to the law is changed to refer to the most recent version of the control lists, additions to the lists will not be controlled.

There are however countries where the national export control regime refers directly to the most recent version of the Wassenaar control lists. New items on the control lists will immediately be controlled in these countries. Israel’s export control law for example refers to the relevant Wassenaar control list ‘as periodically updated’.

What are the recent developments?

The Wassenaar control lists have been updated after the plenary meeting in early December 2013 to include surveillance and law enforcement/intelligence gathering tools and Internet Protocol (IP) network surveillance systems or equipment.

This shows the consensus among the participating states that these tools and systems may under certain conditions be detrimental to international and regional security and stability.

It is also the go-ahead for the participating states and countries to interpret and implement the new controls to create what we hope will become an effective mechanism to control the trade, which will ensure that human rights concerns are given sufficient weighting in the license-granting process.
 
In addition to the links below, further recommended reading includes the Privacy International Surveillance Industry Index and Against Hypocrisy: Updating Export Controls for the Digital Age by OTI's Danielle Kehl and Tim Maurer.

The 13th annual Workshop on the Economic of Information Security will be held at Penn State June 23-24, and the call for papers is now open.

I’m on the program committee this year, and am looking forward to great submissions.

Today at the Chaos Computer Congress (30C3), xobs and I disclosed a finding that some SD cards contain vulnerabilities that allow arbitrary code execution — on the memory card itself. On the dark side, code execution on the memory card enables a class of MITM (man-in-the-middle) attacks, where the card seems to be behaving one way, but in fact it does something else. On the light side, it also enables the possibility for hardware enthusiasts to gain access to a very cheap and ubiquitous source of microcontrollers.

In order to explain the hack, it’s necessary to understand the structure of an SD card. The information here applies to the whole family of “managed flash” devices, including microSD, SD, MMC as well as the eMMC and iNAND devices typically soldered onto the mainboards of smartphones and used to store the OS and other private user data. We also note that similar classes of vulnerabilities exist in related devices, such as USB flash drives and SSDs.

Flash memory is really cheap. So cheap, in fact, that it’s too good to be true. In reality, all flash memory is riddled with defects — without exception. The illusion of a contiguous, reliable storage media is crafted through sophisticated error correction and bad block management functions. This is the result of a constant arms race between the engineers and mother nature; with every fabrication process shrink, memory becomes cheaper but more unreliable. Likewise, with every generation, the engineers come up with more sophisticated and complicated algorithms to compensate for mother nature’s propensity for entropy and randomness at the atomic scale.

These algorithms are too complicated and too device-specific to be run at the application or OS level, and so it turns out that every flash memory disk ships with a reasonably powerful microcontroller to run a custom set of disk abstraction algorithms. Even the diminutive microSD card contains not one, but at least two chips — a controller, and at least one flash chip (high density cards will stack multiple flash die). You can see some die shots of the inside of microSD cards at a microSD teardown I did a couple years ago.

In our experience, the quality of the flash chip(s) integrated into memory cards varies widely. It can be anything from high-grade factory-new silicon to material with over 80% bad sectors. Those concerned about e-waste may (or may not) be pleased to know that it’s also common for vendors to use recycled flash chips salvaged from discarded parts. Larger vendors will tend to offer more consistent quality, but even the largest players staunchly reserve the right to mix and match flash chips with different controllers, yet sell the assembly as the same part number — a nightmare if you’re dealing with implementation-specific bugs.

The embedded microcontroller is typically a heavily modified 8051 or ARM CPU. In modern implementations, the microcontroller will approach 100 MHz performance levels, and also have several hardware accelerators on-die. Amazingly, the cost of adding these controllers to the device is probably on the order of $0.15-$0.30, particularly for companies that can fab both the flash memory and the controllers within the same business unit. It’s probably cheaper to add these microcontrollers than to thoroughly test and characterize each flash memory chip, which explains why managed flash devices can be cheaper per bit than raw flash chips, despite the inclusion of a microcontroller.

The downside of all this complexity is that there can be bugs in the hardware abstraction layer, especially since every flash implementation has unique algorithmic requirements, leading to an explosion in the number of hardware abstraction layers that a microcontroller has to potentially handle. The inevitable firmware bugs are now a reality of the flash memory business, and as a result it’s not feasible, particularly for third party controllers, to indelibly burn a static body of code into on-chip ROM.

The crux is that a firmware loading and update mechanism is virtually mandatory, especially for third-party controllers. End users are rarely exposed to this process, since it all happens in the factory, but this doesn’t make the mechanism any less real. In my explorations of the electronics markets in China, I’ve seen shop keepers burning firmware on cards that “expand” the capacity of the card — in other words, they load a firmware that reports the capacity of a card is much larger than the actual available storage. The fact that this is possible at the point of sale means that most likely, the update mechanism is not secured.

In our talk at 30C3, we report our findings exploring a particular microcontroller brand, namely, Appotech and its AX211 and AX215 offerings. We discover a simple “knock” sequence transmitted over manufacturer-reserved commands (namely, CMD63 followed by ‘A’,'P’,'P’,'O’) that drop the controller into a firmware loading mode. At this point, the card will accept the next 512 bytes and run it as code.

From this beachhead, we were able to reverse engineer (via a combination of code analysis and fuzzing) most of the 8051′s function specific registers, enabling us to develop novel applications for the controller, without any access to the manufacturer’s proprietary documentation. Most of this work was done using our open source hardware platform, Novena, and a set of custom flex circuit adapter cards (which, tangentially, lead toward the development of flexible circuit stickers aka chibitronics).

Significantly, the SD command processing is done via a set of interrupt-driven call backs processed by the microcontroller. These callbacks are an ideal location to implement an MITM attack.

It’s as of yet unclear how many other manufacturers leave their firmware updating sequences unsecured. Appotech is a relatively minor player in the SD controller world; there’s a handful of companies that you’ve probably never heard of that produce SD controllers, including Alcor Micro, Skymedi, Phison, SMI, and of course Sandisk and Samsung. Each of them would have different mechanisms and methods for loading and updating their firmwares. However, it’s been previously noted that at least one Samsung eMMC implementation using an ARM instruction set had a bug which required a firmware updater to be pushed to Android devices, indicating yet another potentially promising venue for further discovery.

From the security perspective, our findings indicate that even though memory cards look inert, they run a body of code that can be modified to perform a class of MITM attacks that could be difficult to detect; there is no standard protocol or method to inspect and attest to the contents of the code running on the memory card’s microcontroller. Those in high-risk, high-sensitivity situations should assume that a “secure-erase” of a card is insufficient to guarantee the complete erasure of sensitive data. Therefore, it’s recommended to dispose of memory cards through total physical destruction (e.g., grind it up with a mortar and pestle).

From the DIY and hacker perspective, our findings indicate a potentially interesting source of cheap and powerful microcontrollers for use in simple projects. An Arduino, with its 8-bit 16 MHz microcontroller, will set you back around $20. A microSD card with several gigabytes of memory and a microcontroller with several times the performance could be purchased for a fraction of the price. While SD cards are admittedly I/O-limited, some clever hacking of the microcontroller in an SD card could make for a very economical and compact data logging solution for I2C or SPI-based sensors.

Slides from our talk at 30C3 can be downloaded here, or you can watch the talk on Youtube below.

Team Kosagi would like to extend a special thanks to .mudge for enabling this research through the Cyber Fast Track program.

David Modic and I have just published a paper on The psychology of malware warnings. We’re constantly bombarded with warnings designed to cover someone else’s back, but what sort of text should we put in a warning if we actually want the user to pay attention to it?

To our surprise, social cues didn’t seem to work. What works best is to make the warning concrete; people ignore general warnings such as that a web page “might harm your computer” but do pay attention to a specific one such as that the page would “try to infect your computer with malware designed to steal your bank account and credit card details in order to defraud you”. There is also some effect from appeals to authority: people who trust their browser vendor will avoid a page “reported and confirmed by our security team to contain malware”.

We also analysed who turned off browser warnings, or would have if they’d known how: they were people who ignored warnings anyway, typically men who distrusted authority and either couldn’t understand the warnings or were IT experts.

When we use terminology that is too broad, too coarse-grained, we make discussion more difficult. That sounds obvious, but it’s easy to miss in practice. We’ve made this mistake in spades with crypto. Discussing the field as one broad topic is counter-productive and leads to needless bickering.

I see 3 major kinds of crypto: b2c crypto, b2b crypto, and p2p crypto. I suggest that we use this terminology consistently to help guide the discussion. We’ll spend less time talking about differences in our assumptions, and more time building better solutions.

b2c crypto

Business-to-Customer Crypto (b2c) is used to secure the relationship between an organization and a typical user. The user roughly trusts the organization, and the goal of b2c crypto is to enable that trust by keeping attackers out of that relationship. Both the organization and the user want to know that they’re talking to each other and not to an impostor. The organization is usually acting like an honest-but-curious party: they’ll mostly do what they promise. The b2c-crypto relationship is common between Internet service providers (in the broad sense, including Google, Amazon, etc.) and typical Internet users, as well as between employees and their employer’s IT department.

Web-browser SSL is a great example of b2c crypto. Users start with a computer that has at least one web browser with a set of root certs. Users can continue using that browser or download, over SSL secured by those initial root certs, another browser they trust more. Users then trust their preferred browser’s security indicators when they shop on Amazon or read their Gmail.

A critical feature of b2c crypto is that users don’t ever manage crypto keys. At best they manage a password, and even then they’re generally able to reset it. Users make trust decisions based on brands and hopefully clear security indicators: I want a Mac, I want to use Firefox, and I want to shop on Amazon but only when I see the green lock icon.

b2b crypto

Business-to-Business (b2b) crypto is used to secure the relationship between organizations, two or more at a time. There are two defining characteristics of b2b crypto:

• all participants are expected to manage crypto keys
• end-users are generally not involved or burdened

DKIM is a good example of b2b crypto. Organizations sign their outgoing emails and verify signatures on incoming emails. Spam and phishing are reduced, and end-users see only the positive result without being involved in the process. Organizations must maintain secret cryptographic keys for signing those emails and know how to publish their public keys (usually in DNS) to inform other organizations.

OAuth qualifies as b2b crypto. Consumers and Producers of Web APIs establish shared secret credentials and use them to secure API calls between organizations.

Another good example is SSL certificate issuance. Both the web site seeking a certificate and the Certificate Authority have to generate and secure secret keys. The complexity of the certification process is mostly hidden from end-users.

p2p crypto

Peer-to-Peer (p2p) crypto is used to secure communication between two or more crypto-savvy individuals. The defining characteristic of p2p crypto is that the crypto-savvy individuals trust no one by default. They tend to run code locally, manage crypto keys, and assume all intermediaries are active attackers.

PGP is a great example of p2p crypto. Everyone generates a keypair, and by default no one trusts anyone else. Emails are encrypted and signed, and if you lose your secret key, you’re out of luck.

so how does this help?

This naming scheme provides a clear shorthand for delineating crypto solutions. Is your wonderful crypto solution targeted at the general public? Then it’s probably a combination of b2c crypto for users and b2b crypto for organizations that support them. Are you building a specialized communications platform for journalists in war zones? Then it’s probably p2p crypto.

The implementation techniques we use for various kinds of crypto differ. So when some folks write that Javascript Crypto is considered harmful, I can easily respond “yes, dynamically-loaded Javascript is a poor approach for p2p crypto, but it’s great for b2c crypto.” In fact, when you look closely at a similar criticism of Javascript crypto from Tony Arcieri, you see this same differentiation, only with much more verbiage because we don’t have clear terminology:

Before I keep talking about where in-browser cryptography is inappropriate, let me talk about where I think it might work: I think it has great potential uses for encrypting messages sent between a user and the web site they are accessing. For example, my former employer LivingSocial used in-browser crypto to encrypt credit card numbers in-browser with their payment processor’s public key before sending them over the wire (via an HTTPS connection which effectively double-encrypted them). This provided end-to-end encryption between a user’s browser and the LivingSocial’s upstream payment gateway, even after HTTPS has been terminated by LivingSocial (i.e. all cardholder data seen by LivingSocial was encrypted).

This is a very good thing. It’s the kind of defense that can prevent the likes of the attack against Target’s 40M customers last month. And that’s exactly the point of b2c crypto.

most users can’t manage crypto keys

I use the term p2p crypto because I like to think of it as “Pro-to-Pro.” Expecting typical Internet users to engage in p2p crypto is, in my opinion, a pipe dream: typical users can’t manage secret crypto keys, so typical users must rely on organizations to do that for them. That’s why successful general-public crypto is a combination of b2c crypto between individuals and the organizations they choose to trust, and b2b crypto across organizations. More expertise and care is expected of the organizations, little is expected of individual users, and some trust is assumed between a user and the organizations they choose.

You don’t have to agree with me on this point to agree with the nomenclature. If you’re interested in protocols where individuals manage their own secret keys and don’t trust intermediaries, you’re interested in p2p crypto. I happen to think that p2p crypto is applicable only to some users and some specific situations.

This is the fourth part of a series on searching on encrypted data. See parts 1, 2 and 3.

In the previous posts we covered two different ways to search on encrypted data. The first was based on property-preserving encryption (in particular, on deterministic encryption), achieved sub-linear search time but had weak security properties. The second was based on functional encryption, achieved linear search time but provided stronger security guarantees.

We’ll now see another approach that achieves the strongest possible levels of security! But first, we need to discuss what we mean by security.

Security

So far, I have discussed the security of the encrypted search solutions informally—mostly providing intuition and describing possible attacks. This is partly because I’d like this blog to remain comprehensible to readers who are not cryptographers but also because formally defining the security properties of encrypted search is a bit messy.

So, which security properties should we expect from an encrypted search solution? What about the following:

1. the encrypted database generated by the scheme should not leak any information about the database of the user;
2. the tokens generated by the user should not leak any information about the underlying search term to the server.

This sounds reasonable but there are several issues. First, this intuition is not precise enough to be meaningful. What I mean is that there are many details that impact security that are not taken into account in this high-level intuition (e.g., what does it mean not to leak, how are the search terms chosen exactly). This is why cryptographers are so pedantic about security definitions—the details really do matter.

Putting aside the issue of formality, another problem with this intuition is that it says nothing about the search results. More precisely, it does not specify whether it is appropriate or not for an encrypted search solution to reveal to the server which encrypted documents match the search term. We usually refer to this information as the client’s access pattern and for concreteness you can think of it as the (matching) encrypted documents’ identifiers or their locations in memory. All we really need as an identifier is some per-document unique string that is independent of the contents of the document and of the keywords associated with it.

So the question is: Is it appropriate to reveal the access pattern? There are two possible answers to this question. On one hand, we could argue that it is fine to reveal the access pattern since the whole point of using encrypted search is so that the server can return the encrypted documents that match the query. And if we expect the server to return those encrypted documents then it clearly has to know which ones to return (though it does not necessarily need to know the contents).

On the other hand, one could argue that, in theory, the access pattern reveals some information to the server. In fact, by observing enough search results the server could use some sophisticated statistical attack to infer something about the client’s queries and data. Note that such attacks are not completely theoretical and in a future post we’ll discuss work that tries to make them practical. Furthermore, the argument that the server needs to know which encrypted documents match the query in order to return the desired documents is not technically true. In fact, we know how to design cryptographic protocols that allow one party to send items to another without knowing which item it is sending (see, e.g., private information retrieval and oblivious transfer).

Similarly, we know how to design systems that allow us to read and write to memory without the memory device knowing which locations are being accessed. The latter are called oblivious RAMs (ORAM) and we could use them to search on encrypted data without revealing the access pattern to the server. The issue, of course, is that using ORAM will slow things down.

So really the answer to our question depends on what kind of tradeoff we are willing to make between efficiency and security. If efficiency is the priority, then revealing the access pattern might not be too much to give up in terms of security for certain applications. On the other hand, if we can tolerate some inefficiency, then it’s always best to be conservative and not reveal anything if possible.

In the rest of this post we’ll explore ORAMs, see how to construct one and how to use it to search on encrypted data.

Oblivious RAM

ORAM was first proposed in a paper by Goldreich and Ostrovsky [GO96] (the link is actually Ostrovsky’s thesis which has the same content as the journal paper) on software protection. That work turned out to be really ahead of its time as several ideas explored in it turned out to be related to more modern topics like cloud storage.

An ORAM scheme consists of:

• A setup algorithm that takes as input a security parameter and a memory (array) of items; it outputs a secret key and an oblivious memory .
• A two-party protocol executed between a client and a server that works as follows. The client runs the protocol with a secret key and an index as input while the server runs the protocol with an oblivious memory as input. At the end of the protocol, the client receives while the server receives , i.e., nothing. We’ll write this sometimes as .

• A two-party protocol executed between a client and a server that works as follows. The client runs the protocol with a key , an index and a value as input and the server runs the protocol with an oblivious memory as input. At the end of the protocol, the client receives nothing (again denoted as ) and the server receives an updated oblivious memory such that the th location now holds the value . We write this as .

Oblivious RAM via FHE

The simplest way to design an ORAM is to use fully-homomorphic encryption (FHE). For an overview of FHE see my previous posts here and here.

Suppose we have an FHE scheme . Then we can easily construct an ORAM as follows [1]:

• : generate a key for the FHE scheme by computing and encrypt as . Output as the oblivious memory .

• : the client encrypts its index as and sends to the server. The server computes

  

  where is a function that takes as input an array and an index and returns the th element of the array. The server returns to the client who decrypts it to recover .

• : the client encrypts its index as and its value as and sends them both to the server. The server computes

  

  where is a function that takes as input an array, an index and a value and returns the same array with the th element updated to .

The security properties of FHE will guarantee that leaks no information about to the server and that the and protocols reveal no information about the index and values either.

The obvious downside of this FHE-based ORAM is efficiency. Let’s forget for a second that FHE is not practical yet and let’s suppose we had a very fast FHE scheme. This ORAM would still be too slow simply because the homomorphic evaluation steps in the and protocols require time, i.e., time linear in the size of the memory. Again, assuming we had a super-fast FHE scheme, this would only be usable for small memories.

Oblivious RAM via Symmetric Encryption

Fortunately, we also know how to design ORAMs using standard encryption schemes and, in particular, using symmetric encryption like AES. ORAM is a very active area of research and we now have many constructions, optimizations and even implementations (e.g., see Emil Stefanov’s implementation). Because research is moving so fast, however, there really isn’t a good overview of the state-of-the-art. Since ORAMs are fairly complicated, I’ll describe here the simplest (non-FHE-based) construction which is due to Goldreich and Ostrovsky [GO96]. This particular ORAM construction is known as the Square-Root solution and it requires just a symmetric encryption scheme , and a pseudo-random function that maps bits to bits.

Setup. To setup the ORAM, the client generates two secret keys and for the symmetric encryption scheme and for the pseudo-random function , respectively. It then augments each item in by appending its address and a random tag to it. We’ll refer to the address embedded with the item as its virtual address. More precisely, it creates a new memory such that for all ,

where denotes concatenation and . It then adds dummy items to , i.e., it creates a new memory such that for all , and such that for all ,

where is some number larger than . It then sorts around according to the tags. Notice that the effect of this sorting will be to permute since the tags are (pseudo-)random. It then encrypts each item in using . In other words, it generates a new memory such that, for all ,

Finally, it appends elements to each of which contains an encryption of under key . Needless to say, all the ciphertexts generated in this process need to be of the same size so the items need to be padded appropriately. The result of this, i.e., the combination of and the encryptions of , is the oblivious memory which is sent to the server. It will be useful for us to distinguish between the two parts of so we’ll refer to the second part (i.e., the encryptions of ) as the cache.

Read & write. Now we’ll see how to read and write to obliviously, i.e., without the server knowing which memory locations we’re accessing. First we have to define two basic operations: and .

The operation takes an index as input and works as follows:

1. the client requests from the server the item at virtual addres in . To do this it first re-generates the item’s tag . It then does an (interactive) binary search to find the item with virtual address . In other words, it asks the server for the item stored at location (let ‘s assume is even) decrypts it and compares its tag with . If is less than the tag of item , then it asks for the item at location ; else it asks for the item at location ; and so on.
2. it decrypts the item with to recover ,
3. it then re-encrypts (using new randomness) and asks the server to store it back where it was found.

The operation takes an index and a value as inputs and works as follows:

1. the client requests from the server the item with (as above);
2. it then encrypts and asks the server to store it back at the location where the previous item (i.e., the one with ) was found.

Notice that from the server’s point of view the two operations look the same. In other words, the server cannot tell whether the client is executing a or a operation since in either case all it sees is a binary search followed by a request to store a new ciphertext at the same location.

Now suppose for a second that only consisted of . If that were the case then would be one-time oblivious in the sense that we could use it to read or write only once by executing either a or a operation. Why is this the case? Remember that we randomly permuted and encrypted our memory before sending it to the server. This means that asking the server for the item at location reveals nothing about that item’s real/virtual address . Furthermore, the binary search we do when looking for the item with virtual address depends only which is random and therefore reveals nothing about .

Of course, this only works once because if we want to access again then we’ll ask the server for the same location which immediately tells it something: namely, that we asked for the same thing twice.

So how do we hide the fact that we’re asking for the same thing twice? This is really the core difficulty in designing ORAMs and this is where the cache will come in.

We start by initializing a counter . To read location we execute the following protocol:

1. We the entire cache. In other words, we execute for all

   

2. If any of the operations above result in the th item (i.e., if we get an item with virtual address ) then we a dummy item by executing . Also, we set to be the item we found in the cache and to be the cache location where we found it.
3. If none of the operations above resulted in the th item, we execute a modified and set to be the result and . The modified version of works like a regular operation, except that we update the item’s virtual address to , where . In other words, we store an encryption of back where we found it. This will be useful for us later when we’ll need to re-structure .
4. We then process the entire cache again but slightly differently than before (we do this so that we can store the item in the cache for future accesses). In particular, for all ,
   • if we execute a operation
   • if we execute a .
5. We increase by .

The first thing to notice is that this is correct in the sense that by executing this operation the client will indeed receive .

The more interesting question, however, is why is this oblivious and, in particular, why is this more than one-time oblivious? To see why this is oblivious it helps to think of things from the server’s perspective and see why its view of the execution is independent of (i.e., not affected by) .

First, no matter what the client is looking for, it always s the entire cache so Step reveals no information about to the server. We then have two possible cases:

1. If the th item is in the cache (at location ), we a dummy item; and the th item at location while we re-process the entire cache (in Step ).

2. If the th item is not in the cache, we the th item and it in the next open location in the cache while we re-process the entire cache.

In either case, the server sees the same thing: a for an item at some location between and and a sequence of operations for all addresses in the cache, i.e., between and . Recall that the server cannot distinguish between and operations. The protocol is similar to the protocol. The only difference is that in Step , we set if the th item is in the cache and in Step we execute and set . Notice, however, that the protocol can introduce inconsistencies between the cache and . More precisely, if the item has been accessed before (say, due to a operation), then a operation will update the cache but not the item in . This is OK, however, as it will be taken care of in the re-structuring step, which we’ll describe below.

So we can now read and write to memory without revealing which location we’re accessing and we can do this more than once! The problem, however, is that we can do it at most times because after that the cache is full so we have to stop.

Re-structuring. So what if we want to do more than reads? In that case we need to re-structure our ORAM. By this, I mean that we have to re-encrypt and re-permute all the items in and reset our counter to .

If the client has enough space to store locally then the easiest thing to do is just to download , decrypt it locally to recover , update it (in case there were any inconsistencies) and setup a new ORAM from scratch.

If, on the other hand, the client does not have enough local storage then the problem becomes harder. Here we’ll assume the client only has storage so it can store, e.g., only two items.

Recall that in order to re-structure , the client needs to re-permute and re-encrypt everything obliviously while using only space. Also, the client needs to do this in a way that updates the elements that are in an inconsistent state due to operations. The key to doing all this will be to figure out a way for the client to sort elements obliviously while using space. Once we can obliviously sort, the rest will follow relatively easily.

To do this, Goldreich and Ostrovsky proposed to use a sorting network like Batcher’s Bitonic network. Think of a sorting network as a circuit composed of comparison gates. The gates take two inputs and and output the pair if and the pair if . Given a set of input values, the sorting network outputs the items in sorted order. Sorting networks have two interesting properties: the comparisons they perform are independent of the input sequence; and each gate in the network is a binary operation (i.e., takes only two inputs). Of course, there is an overhead to sorting obviously so Batcher’s network requires work as opposed to the traditional for sorting.

So to obliviously sort a set of ciphertexts stored at the server, the client will start executing the sorting network and whenever it reaches a comparison gate between the th and th item, it will just request the th and th ciphertexts, decrypt them, compare them, and store them back re-encrypted in the appropriate order. Note that by the first property above, the client’s access pattern reveals nothing to the server; and by the second property the client will never need to store more than two items at the same time.

Now that we can sort obliviously, let’s see how to re-structure the ORAM. We will do it in two phases. In the first phase, we sort all the items in according to their virtual addresses. This is how we will get rid of inconsistencies. Remember that the items in are augmented to have the form for real items and for dummy items. It follows that all items in the cache have the first form since they are either copies or updates of real items put there during and operations.

So we just execute the sorting network and, for each comparison gate, retrieve the appropriate items, decrypt them, compare their virtual addresses and return them re-encrypted in the appropriate order. The result of this process is that will now have the following form:

1. the first items will consist of the most recent versions of the real items, i.e., all the items with virtual addresses other than and ;
2. the next items will consist of dummy items, i.e., all items with virtual address .

3. the final items will consist of the old/inconsistent versions of the real items, i.e., all items with virtual address (remember that in Step of and we executed a modified that updated the item’s virtual address to ).

In the second phase, we randomly permute and re-encrypt the first items of . We first choose a new key for . We then access each item from location to and update their tags to . Once we’ve updated the tags, we sort all the items according to their tags. The result will be a new random permutation of items. Note that we don’t technically have to do this in two passes; but it’s easier to explain this way.

At this point, we’re done! is as good as new and we can start accessing it again safely.

Efficiency. So what is the efficiency of the Square-Root solution? Setup is : to construct the real, dummy and cache items and to permute everything through sorting.

Each access operation (i.e., or ) is : total get/put operations to get the cache twice and for each get/put operation due to binary search.

Restructuring is : to sort by virtual address and to sort by tag. Restructuring, however, only occurs once every accesses. Because of this, we usually average the cost of re-structuring over the number read/write operations supported to give an amortized access cost. In our case, the amortized access cost is then

which is .

ORAM-Based Encrypted Search

So now that we know how to build an ORAM, we’ll see how to use it for encrypted search. There are two possible ways to do this.

A naive approach. The first is for the client to just dump all the documents in an array , setup an ORAM and send to the server. To search, the client can just simulate a sequential search algorithm via the protocol; that is, replace every read operation of the search algorithm with an execution of the protocol. To update the documents the client can similarly simulate an update algorithm using the protocol.

This will obviously be slow. Let’ s assume all the documents have bit-length and that has a block size of bits. The document collection will then fit in (approximately) blocks. The sequential scan algorithm is itself , but on top of that we’ll have to execute an entire protocol for every address of memory read.

Remember that if we’re using the Square-Root solution as our ORAM then the protocol requires amortized work. So in total, search would be which would not scale. Now imagine for a second if we were using the FHE-based ORAM described above which requires work for each and . In this scenario, a single search would take time!

A better approach. [2] A better idea is for the client to build two arrays and [3]. In it will store a data structure that supports fast searches on the document collection (e.g., an inverted index) and in it will store the documents themselves. It then builds and sends and to the server. To search, the client simulates a query to the data structure in via the protocol (i.e., it replaces each read operation in the data structure’s query algorithm with an execution of ) [4]. From this, the client will recover the identifiers of the documents that contain the keyword and with this information it can just read those documents from .

Now suppose there are documents that contain the keyword and that we’re using an optimal-time data structure (i.e., a structure with a query algorithm that runs in time like an inverted index). Also, assume that the data structure fits in blocks of bits and that the data collection fits in blocks.

Again, if we were using the Square-Root solution for our ORAMs, then the first step would take time and the second step will take

In practice, the size of a fast data structure for keyword search can be large. A very conservative estimate for an inverted index, for example, would be that it is roughly the size of the data collection [5]. Setting , the total search time would be

which is (since ) compared to the previous approach’ s .

In cases where the search term appears in documents, this can be a substantial improvement.

Is This Practical?

If one were to only look at the asymptotics, one might conclude that the two-RAM solution described above might be reasonably efficient. After all it would take at least time just to retrieve the matching files from (unencrypted) memory so the two-RAM solution adds just a multiplicative factor over the minimum retrieval time.

Also there are much more efficient ORAM constructions than the Square-Root solution. In fact, in their paper, Goldreich and Ostrovsky also proposed the Hierarchichal solution which achieves amortized access cost. Goodrich and Mitzenmacher [GM11] gave a solution with amortized access cost and, recently, Kushilevitz, Lu and Ostrovsky [KLO12] a solution with amortized cost (and there are even more recent papers that improve on this under certain conditions). There are also works that tradeoff client storage for access efficiency. For example, Williams, Sion and Carbunar [WSC08] propose a solution with amortized access cost and client storage while Stefanov, Shi and Song [SSS12] propose a solution with amortized overhead for clients that have local storage, where the underlying constant is very small. There is also a line of work that tries to de-amortize ORAM in the sense that it splits the re-structuring operation so that it happens progressively over each access. This was first considered by Ostrovsky and Shoup in [OS97] and was further studied by Goodrich, Mitzenmacher, Ohrimenko, Tamassia [GMOT11] and by Shi, Chan, Stefanov and Li [SSSL11].

All in all this may not seem that bad and, intuitively, the two-RAM solution might actually be reasonably practical for small to moderate-scale data collections—especially considering all the recent improvements in efficiency that have been proposed. For large- or massive-scale collections, however, I’d be surprised [6].

Conclusions

In this post we went over the ORAM-based solution to encrypted search which provides the most secure solution to our problem since it hides everything—even the access pattern!

In the next post we’ll cover an approach that tries to strike a balance between efficiency and security. In particular, this solution is as efficient as the deterministic-encryption-based solution while being only slightly less secure than the ORAM-based solution.

Notes

[1] I haven’t seen this construction written down anywhere. It’s fairly obvious, however, so I suspect it’s been mentioned somewhere. If anyone knows of a reference, please let me know. Also, as Jon Katz points out in the comments, this approach requires the server to compute whereas traditional ORAM constructions do not. Some recent works have started to distinguish these variants of ORAM (see comments for more).

[2] Like the FHE-based ORAM, I have not seen this construction written down anywhere so if anyone knows of a reference, please let me know.

[3] Of course, the following could be done using a single RAM, but splitting into two makes things easier to explain.

[4] Note that this will reveal to the server some information. If we’re using an inverted index as the underlying search structure, it will reveal the number of documents that contain the keyword (see the comments for a brief discussion). The natural way to address this, of course, is to use an inverted index with lists that are padded to the maximum size.

[5] In practice, this would not be the case and, in addition, we could make use of index compression techniques.

[6] I won’t attempt to draw exact lines between what’s small-, moderate- and large-scale since I think that’s a question best answered by experimental results.

There’s a new study on what people would pay for privacy in apps. As reported by Techflash:

A study by two University of Colorado Boulder economists, Scott Savage and Donald Waldman, found the average user would pay varying amounts for different kinds of privacy: $4.05 to conceal contact lists, $2.28 to keep their browser history private, $2.12 to eliminate advertising on apps, $1.19 to conceal personal locations, $1.75 to conceal the phone’s ID number and $3.58 to conceal the contents of text messages.

Those numbers seem small, but they’re in the context of app pricing, which is generally a few bucks. If those numbers combine linearly, people being willing to pay up to $10 more for a private version is a very high valuation. (Of course, the numbers will combine in ways that are not strictly rational. Consumers satisfice.

A quick skim of the article leads me to think that they didn’t estimate app maker benefit from these privacy changes. How much does a consumer contact list go for? (And how does that compare to the fines for improperly revealing it?) How much does an app maker make per person whose eyeballs they sell to show ads?

New Cyber Security Committee Will Identify and Share Global Information Security Best Practices in the Protection of Market Infrastructures

The World Federation of Exchanges (WFE), a 62-member trade association for operators of regulated financial exchanges, has launched a new cyber security committee designed to help members from the exchange industry protect against cyber threats targeting the global capital markets.

According to the WFW, the Cyber Security Working Group will bring together representatives from a number of exchanges and clearinghouses across the globe, to collaborate on best practices in cybersecurity.

The group will be chaired by NASDAQ OMX CISO Mark Graff and vice-chaired by Jerry Perullo, Vice President, Information Security at Intercontinental Exchange (ICE).

Founding committee members include:

• Australian Securities Exchange

• BM&FBOVESPA

• CME Group

• The Depository Trust & Clearing Corporation (DTCC)

• Deutsche Boerse

• IntercontinentalExchange (ICE)

• International Securities Exchange (ISE)

• NASDAQ OMX

• NYSE Euronext \ ICE

• Saudi Stock Exchange

• Singapore Exchange

• SIX Swiss Exchange

• Toronto Stock Exchange

According to the WFE, the Cyber Security Committee will focus on the following principles:

• Establishing a communication framework among participants based on mutual trust;

• Facilitating information sharing, including threat intelligence, attack trends, and useful policies, standards and technologies;

• Enhancing dialogue with policy makers, regulators and government organizations on cyber threats for fair, transparent and efficient markets;

• Supporting improved defenses from both external and internal cyber-based threats against the markets.

“I’m proud to be working with an array of some of the brightest information security officers who in the exchange industry around the world,” said Graff. “We are tasked with a significant goal: to build universal best practices and partner with third-parties to combat systemic cyber abuse to ensure the resiliency and strength of our capital markets. I look forward to addressing this head on with the founding committee and new members alike.”

A "significant" number of stock exchanges have been hit by cyber-attacks over the past year, according to a survey from the International Organization of Securities Commissions and WTE, released in July 2013.

The survey showed that about 53 percent of exchange had been hit by a cyber-attack. Of the 46 exchanges around the world included in the survey, nearly a quarter of the exchanges who had been attacked were in the Asia-Pacific region.

“While there is uncertainty around the size of the cyber-crime threat in securities markets, there are clear signs that it is a growing threat to the financial sector, with potential for large costs,” the study's co-authors, Iosco's Rohini Tendulkar and WFE's Gregoire Naacke, noted at the time.

“The creation of this committee group is a significant milestone for the global exchange community,” said Ravi Narain, Chairman of the Working Committee. “Cyber security is a crucial issue to global markets, and paramount for maintaining market integrity and resiliency. We are pleased with the positive collaboration in this committee, which will transcend competitors and regions in order to tackle key issues and present best practices, and we believe that the formation will universally benefit the financial markets of the world.”

Related Reading: Cyber Attacks Against Stock Exchanges Threaten Financial Markets

Research Vice President at NSS Labs Stefan Frei joins the podcast to discuss his idea -- and the economics -- to support an an International Vulnerability Purchase Program.

“Few things in life are as solid as they seem.”

When she was five years old, Dame Stephanie “Steve” Shirley, born Vera Buchthal, fled Nazi Germany as a child refugee, escaping certain death and plunging into a life that would show her a quieter yet oppressively persistent kind of discrimination and injustice. A girl with a strong passion for mathematics, she had to transfer to the boys’ school to get a proper education, where she first became aware of her gender’s second-class status — but she, a voracious reader, took refuge in the school library and devoured countless books spanning every imaginable field. As a young woman drawn to the computing industry, she saw that signing her name opened no doors, so she adopted the nickname Steve and began signing as “Steve Shirley.” Suddenly, doors swung open.

Steve Shirley went on to become the world’s first freelance programmer and founded the software company F.I. Group in 1962, one of the UK’s earliest startups. It was a revolutionary company, writing software only — an outrageous proposition at the time. It was managed and operated by highly skilled female engineers (“We hired men. If they were good enough.”), who worked from home — also unthinkable amidst the era’s gender biases and social norms. And yet they forged forward, forever changing the course of entrepreneurship and women in technology. When F.I. was eventually floated on the London Stock Exchange in 1996, it earned hundred of millions of pounds. Today, she is bringing that same zest for change and betterment to her work in philanthropy.

This fantastic short film, produced by Google as part of a series highlighting women’s involvement in the early days of computing, tells Dame Steve Shirley’s remarkable and heartening story:

The fact that I almost died in the Holocaust means that I’m very motivated to make sure that each day is worth living, that my life was worth saving… I had built a determination that I was not going to let other people define me, to break through, to build something new, to not be put off by the conventions of the day.

In her memoir, Let It Go (public library), Dame Steve further elaborates on this disposition, tracing how her childhood experience of being sent away into safety by her German parents and raised by kindly strangers in the UK shaped her outlook on life, work, and philanthropy:

I have known failure and heartbreak as well as success, but I have never quite lost sight of two life-defining ideas – both of which I can trace back to my arrival in England all those years ago as a terrified, weeping child refugee.

The first is the conviction that even in the blackest moments of despair there is hope, if one can find the courage to pursue it. Sometimes the worst is less overwhelmingly awful than we fear; sometimes the right attitude can create good even from life’s most terrible situations.

My second big idea is the matching conviction that, even though I ostensibly lost everything when my parents were forced to send me away, I was not just the victim of bigotry and cruelty. I was also the fortunate beneficiary of the unearned generosity of many people: the Jewish and Christian activists who set up the Kindertransport, the Quakers who kept the project going when it ran out of money, the ordinary people who chipped in with the various tedious administrative tasks that allowed the project to function, the Catholic nuns who helped to educate me, and the quiet, middle-aged, nominally Anglican couple who took me in.

Without my being fully aware of what was going on or why, a large number of good-natured strangers took it upon themselves to save my life. It took me some years to digest this fact and its implications. But, once I had, a simple resolution took root deep in my heart: I had to make sure that mine was a life that had been worth saving.

I may not always have succeeded in this aim. But I have at least learnt lessons along the way: about how to make things happen, how to deal with setbacks and how to turn the most improbable dreams into realities.

She reflects on her separation from her birth parents:

Looking back today, from the other end of a life that has been exceptionally rich in nearly every sense, I can see that most of my subsequent achievements can be traced back to that unnatural separation. It marked the beginning of a narrative far more interesting than the one that had originally been scripted for me. But it also taught me, with the ending of my first life, a profound lesson: that few things in life are as solid as they seem; that tomorrow will not always resemble today; and that wholesale change, though often terrifying, is not necessarily synonymous with catastrophe.

Let It Go goes on to trace just how this extraordinary woman turned “improbable dreams into realities” — a story of bravery, determination, and triumphant ingenuity against even the most inauspicious of odds.

Brain Pickings has a free weekly newsletter. It comes out on Sundays and offers the week’s best articles. Here’s what to expect. Like? Sign up.

Brain Pickings takes 450+ hours a month to curate and edit across the different platforms, and remains banner-free. If it brings you any joy and inspiration, please consider a modest donation – it lets me know I'm doing something right.

Security experts have long opined that one way to make software more secure is to hold software makers liable for vulnerabilities in their products.  This idea is often dismissed as unrealistic and one that would stifle innovation in an industry that has been a major driver of commercial growth and productivity over the years. But a new study released this week presents perhaps the clearest economic case yet for compelling companies to pay for information about security vulnerabilities in their products.

Before I delve into this modest proposal, let’s postulate a few assumptions that hopefully aren’t terribly divisive:

• Modern societies are becoming increasingly dependent on software and computer programs.
• After decades of designing software, human beings still build imperfect, buggy, and insecure programs.
• Estimates of the global damage from cybercrime ranges from the low billions to hundreds of billions of dollars annually.
• The market for finding, stockpiling and hoarding (keeping secret) software flaws is expanding rapidly.
• Vendor-driven “bug bounty” programs which reward researchers for reporting and coordinating the patching of flaws are expanding, but currently do not offer anywhere near the prices offered in the underground or by private buyers.
• Software security is a “negative externality”: like environmental pollution, vulnerabilities in software impose costs on users and on society as a whole, while software vendors internalize profits and externalize costs. Thus, absent any demand from their shareholders or customers, profit-driven businesses tend not to invest in eliminating negative externalities.

Earlier this month, I published a piece called How Many Zero-Days Hit You Today, which examined a study by vulnerability researcher Stefan Frei about the bustling market for “zero-day” flaws — security holes in software that not even the makers of those products know about. These vulnerabilities — particularly zero-days found in widely-used software like Flash and Java — are extremely valuable because attackers can use them to slip past security defenses unnoticed.

Frei’s analysis conservatively estimated that private companies which purchase software vulnerabilities for use by nation states and other practitioners of cyber espionage provide access to at least 85 zero-day exploits on any given day of the year. That estimate doesn’t even consider the number of zero-day bugs that may be sold or traded each day in the cybercrime underground.

At the end of that post, I asked readers whether it was possible and/or desirable to create a truly global, independent bug bounty program that would help level the playing field in favor of the defenders and independent security researchers. Frei’s latest paper outlines one possible answer.

BUYING ALL BUGS AT ABOVE BLACK-MARKET PRICES

Frei proposes creating a multi-tiered, “international vulnerability purchase program” (IVPP), in which the major software vendors would be induced to purchase all of the available and known vulnerabilities at prices well above what even the black market is willing to pay for them. But more on that in a bit.

The director of research for Austin, Texas-based NSS Labs, Frei examined all of the software vulnerabilities reported in 2012, and found that the top 10 software makers were responsible for more than 30 percent of all flaws fixed. Frei estimates that if these vendors were to have purchased information on all of those flaws at a steep price of $150,000 per vulnerability — an amount that is well above what cybercriminals or vulnerability brokers typically offer for such bugs — this would still come to less than one percent of the annual revenues for these software firms.

Frei points out that the cost of purchasing all vulnerabilities for all products would be considerably lower than the savings that would occur as a result of the expected reduction in losses occurring as a result of cyber crime — even under the conservative estimate that these losses would be reduced by only 10 percent.

In the above chart, for example, we can see Oracle — the software vendor responsible for Java and a whole heap of database software code that is found in thousands of organizations — fixed more than 427 vulnerabilities last year. It also brought in more than $37 billion in revenues that year. If Oracle were to pay researchers top dollar ($150,000) for each vulnerability, that would still come to less than two-tenths of one percent of the company’s annual revenues (USD $67 million).

Frei posits that if vendors were required to internalize the cost of such a program, they would likely be far more motivated to review and/or enhance the security of their software development processes.

Likewise, Frei said, such a lucrative bug bounty system would virtually ensure that every release of commercial software products would be scrutinized by legions of security experts.

“In the short term, it would hit the vendors very badly,” Frei said in a phone interview with KrebsOnSecurity. “But in the long term, this would produce much more secure software.”

Next year’s Workshop on the Economics of Information Security (WEIS 2014) will be at Penn State on June 23–24. The call for papers is now out; papers are due by the end of February.

It will be fascinating to see what effects the Snowden revelations will have on the community’s thinking about security standards and regulation, the incentives for information sharing and cooperation, and the economics of privacy, to mention only three of the workshop’s usual topics. Now that we’ve had six months to think things through, how do you think the security game has changed, and why? Do we need minor changes to our models, or major ones? Are we in the same policy world, or a quite different one? WEIS could be particularly interesting next year. Get writing!

Everyone knows that it is practically impossible to write secure software. In spite of massive security investments by the software industry, we have come to expect the frequent publications of large quantities of new vulnerabilities. Yet we tend to ignore the fact that these vulnerabilities in fact are in existence long before publication but are known only to privileged groups such as cyber criminals, brokers, and government agencies.

On any given day, nation-states and criminal hackers have access to an entire arsenal of zero-day vulnerabilities  – undocumented and unpatched software flaws that can be used to silently slip past most organizations’ digital defenses, new research suggests.  That sobering conclusion comes amid mounting evidence that thieves and cyberspies are ramping up spending to acquire and stockpile these digital armaments.

Security experts have long suspected that governments and cybercriminals alike are stockpiling zero-day bugs: After all, the thinking goes, if the goal is to exploit these weaknesses in future offensive online attacks, you’d better have more than a few tricks up your sleeve because it’s never clear whether or when those bugs will be independently discovered by researchers or fixed by the vendor. Those suspicions were confirmed very publicly in 2010 with the discovery of Stuxnet, a weapon apparently designed to delay Iran’s nuclear ambitions and one that relied upon at least four zero-day vulnerabilities.

Documents recently leaked by National Security Agency whistleblower Edward Snowden indicate that the NSA spent more than $25 million this year alone to acquire software vulnerabilities from vendors. But just how many software exploits does that buy, and what does that say about the number of zero-day flaws in private circulation on any given day?

These are some of the questions posed by Stefan Frei, research director for Austin, Texas-based NSS Labs. Frei pored over reports from and about some of those private vendors — including boutique exploit providers like Endgame Systems, Exodus Intelligence, Netragard, ReVuln and VUPEN – and concluded that jointly these firms alone have the capacity to sell more than 100 zero-day exploits per year.

According to Frei, if we accept that the average zero-day exploit persists for about 312 days before it is detected (an estimate made by researchers at Symantec Research Labs), this means that these firms probably provide access to at least 85 zero-day exploits on any given day of the year. These companies all say they reserve the right to restrict which organizations, individuals and nation states may purchase their products, but they all expressly do not share information about exploits and flaws with the affected software vendors.

KNOWN UNKNOWNS

That approach stands apart from the likes of HP TippingPoint‘s Zero-Day Initiative (ZDI) and Verisign‘s iDefense Vulnerability Contributor Program (VCP), which pay researchers in exchange for the rights to their vulnerability research. Both ZDI and iDefense also manage the communication with the affected vendors, ship stopgap protection for the vulnerabilities to their customers, and otherwise keep mum on the flaws until the vendor ships an update to fix the bugs.

Frei also took stock of the software vulnerabilities collected by these two companies, and found that between 2010 and 2012, the ZDI and VCP programs together published 1,026 flaws, of which 425 (44 percent) targeted flaws in Microsoft, Apple, Oracle, Sun and Adobe products. The average time from purchase to publication was 187 days.

“On any given day during these three years, the VCP and ZDI programs possessed 58 unpublished vulnerabilities affecting five vendors, or 152 vulnerabilities total,” Frei wrote in a research paper released today.

Frei notes that the VCP and ZDI programs use the information they purchase only for the purpose of building better protection for their customers, and since they share the information with the software vendors in order to develop and release patches, the overall risk is comparatively low. Also, the vulnerabilities collected and reported by VCP and ZDI are not technically zero-days, since one important quality of a zero-day is that it is used in-the-wild to attack targets before the responsible vendor can ship a patch to fix the problem.

In any case, Frei says his analysis clearly demonstrates that critical vulnerability information is available in significant quantities for private groups, for extended periods and at a relatively low cost.

“So everybody knows there are zero days, but when we talk to C-Level executives, very often we find that these guys don’t have a clue, because they tell us, ‘Yeah, but we’ve never been compromised’,” Frei said in an interview.  ”And we always ask them, ‘How do you know?’”

Frei said that in light of the present zero-day reality, he has three pieces of advice for C-Level executives:

• Assume you are compromised, and that you will get compromised again.
• Prevention is limited; invest in breach detection so that you can quickly find and act on any compromises.
• Make sure you have a process for properly responding to compromises when they do happen.

ANALYSIS

Although’s Frei’s study is a very rough approximation of the zero-day scene today, it is almost certainly a conservative estimate: It makes no attempt to divine the number of zero-day vulnerabilities developed by commercial security consultancies, which employ teams of high-skilled reverse engineers who can be hired to discover flaws in software products.

Nor does it examine the zero-days that are purchased and traded in the cybercriminal underground, where vulnerability brokers and exploit kit developers have been known to pay tens of thousands of dollars for zero-day exploits in widely-used software. I’ll have some of my own research to present on this latter category in the coming week. Stay tuned. Update, Dec. 6, 1:30 p.m. ET: Check out this story on the arrest of the man thought to be behind the Blackhole Exploit Kit. He allegedly worked with a partner who had a $450,000 budget for buying browser exploits.

Original story:

But Frei’s research got me to thinking again about an idea for a more open and collaborative approach to discovering software vulnerabilities that has remained stubbornly stuck in my craw for ages. Certainly, many companies have chosen to offer “bug bounty” programs — rewards for researchers who report zero-day discoveries. To my mind, this is good and as it should be, but most of the companies offering these bounties — Google, Mozilla, and Facebook are among the more notable —  operate in the cloud and are not responsible for the desktop software products most often targeted by high-profile zero-days.

After long resisting the idea of bug bounties, Microsoft also quite recently began a program to pay researchers who discover novel ways of defeating its security defenses. But instead of waiting for the rest of the industry to respond in kind and reinventing the idea of bug bounties one vendor at a time, is there a role for a more global and vendor-independent service or process for incentivizing the discovery, reporting and fixing of zero-day flaws?

Most of the ideas I’ve heard so far involve funding such a system by imposing fines on software vendors, an idea which seems cathartic and possibly justified, but probably counterproductive. I’m sincerely convinced that a truly global and remunerative bug bounty system is possible and maybe even inevitable as more of our lives, health and wealth become wrapped up in technology. But there is one sticking point that I simply cannot get past: How to avoid having the thing backdoored or otherwise subverted by one or more nation-state actors?

I welcome a discussion on this topic. Please sound off in the comments below.

DARPA debuts five different puzzle games to test whether players can spot mathematical flaws in open-source code used by the Defense Department

Private brokers sell zero-day bugs for anywhere between $40,000 and $160,000 -- and in some cases as much as $1 million, a new study says

This story should get more publicity than it has.

About a third of large companies have a cyberinsurance policy, but the industry still has issues measuring risks and gauging threats