Conceptualizing the Mandate for the Bureau of Cyber Statistics

The U.S. Capitol building at dusk. (Barbar Facemire, https://pixy.org/3293/; CC0 Public Domain, https://creativecommons.org/publicdomain/zero/1.0/deed.en) As I noted in an earlier Lawfare post (co-authored with Chas Kissick), the Cyberspace  Solarium Commission has recommended the establishment of a Bureau of Cyber Statistics (BCS). In our earlier post, Kissick and I reflected on several questions that relate to the bureau’s organizational structure, in an effort to advance the discussion on the structure and direction that the bureau should take.  This post moves to the second part of the discussion: the substance of the bureau’s mandate. Answering questions about the substance of the bureau’s work will require wide-ranging consultation with the government, the private sector and non-governmental organizations during the coming months. Through that sort of consultation, the outlines of the BCS can be developed.  In an effort to advance those discussions, in this post I identify some of the most salient substantive questions about the BCS. For instance, as Congress contemplates implementing legislation in this session, it will be asked to define the institution: What shall it measure? Why shall it measure certain things? What authorities should it have? And so on. As with the earlier post , the questions identified here are intended to be neither exhaustive nor definitive .  Mission I’ll start with the broadest question: What is the bureau’s mission? The Cyberspace Solarium Commission envisions that the bureau would be “the government statistical agency that collects, processes, analyzes, and disseminates essential statistical data on cybersecurity, cyber incidents, and the cyber ecosystem to the American public, Congress, other federal agencies, state and local governments, and the private sector.” Taking the commission’s mandate as descriptive, its tasking identifies three types of data (security statistics, incident data and ecosystem health data) that could be of value to five different possible consumers. Given the reality that different consumers have different needs, the BCS might in theory create as many as 15 different categories or groupings of data streams. Some of those data streams are likely to overlap. But even if that is true, it would be overly ambitious for a new bureau to start with the objective of delivering all of this data immediately. It could also be too broad a mandate for the BCS, even when it reaches a mature level of development. Thus, the first major question that will face Congress is broadly to define what it expects the BCS to do and to prioritize among these various areas of interest. Should the BCS initially focus on the collection of enterprise security data, for example, or on strategic- level network hygiene data? Should it collect data that would be of principal value to the federal government, or focus initially on data of value to the private sector? There are, naturally, many permutations to this inquiry. How one conceives of the bureau’s mission will influence the more detailed subsidiary questions identified below, but it is critically important to ask these questions at the outset: What do we want from the BCS? Who are the primary customers, and what do they want?  For my own part, I suggest that—whatever the details eventually decided upon—the bureau’s overall mission should be to define and collect metrics that can inform public- and private- sector decision-makers about risk mitigation and resource allocation. In practice, that means metrics that are transparent, scalable, auditable, usable and widely agreed upon. At this high level of generality, the setting of strategic objectives is relatively easy—the devil is always in the details. But in general the BCS should have two broad mission sets:       To monitor and evaluate the effectiveness of government cybersecurity programs in driving down cyber risk.       To evaluate the effectiveness of widely held best practices (such as preventive measures and security controls) both within the government and in the private sector. Who Is the Target Audience? It is possible to imagine the utility of cybersecurity data to both private or public actors. The development of a transparent and auditable system of metrics is intended to unite decision- makers across various enterprise and use case scenarios. Given the breadth of potential consumers of data and the variegated nature of their demands, the bureau will initially have to determine whether the first metrics it chooses to develop are of greater value to government consumers or to the private sector. The argument for prioritizing government-focused metrics is simple: Doing so eliminates many of the external communications and influence questions that are inherent in government-private interactions. The BCS will reside within the government. With the goal of easing the bureau’s implementation into the government, its primary focus should have government consumers in mind.  This initial focus on government activity is likely to pay early dividends. It would not be an “inward focus” simply to measure the effectiveness of government enterprise cybersecurity. Rather, if the bureau were to initially develop government-focused metrics, its consumers could anticipate a more precise and granular understanding of how successful (or unsuccessful) certain programs are. The government would have a better measurement, for example, of the effectiveness of the work that the Cybersecurity and Infrastructure Security Agency (CISA) does to improve cybersecurity of critical infrastructure, of the National Institute of Standards and Technology (NIST) framework’s impact on cyber risk, and of the value of sector risk management agencies’ programming to manage risk in their sectors. In the long run, the private sector will likely have a greater need for agreed-upon metrics. A vast majority of America’s critical cyber infrastructure is in the hands of the private sector. Private- sector mechanisms for improving cybersecurity (like a liability regime) will depend heavily on good metrics. While some observers may argue that the BCS should focus on the private sector due to greater demand for those statistics, the primary focus on government consumers seems a wiser course. However, the enabling legislation should be explicit in noting that the initial government focus is a stepping stone to metrics for a wider audience, not an end in itself. What Metrics? The next, more detailed subsidiary question is precisely what can or should be measured. Congress, in authorizing the BCS, will want to define the scope of its responsibility and its methods. Much about this is indefinite. There are three broad categories, with some overlap, into which the metrics could fall: metrics that are epidemiological in nature, assessing the hygiene of the cyber ecosystem at a high level of generality; metrics that are more detailed, broadly applicable measures of enterprise security; or metrics that collect incident data. While these categories may appear similar, they are conceptually and practically distinct. Which are the right metrics to develop and in what order? Several factors will play into that decision. Here are a few considerations: First, BCS should have the authority to collect and aggregate already-existing public data from whatever sources currently collect them—whether by lawful free access or, if necessary, purchase. Collecting and cataloging preexisting databases into a single integrated source would be an easy and practical first step. For example, various databases across the United States already collect incident data, such as Verizon’s annual report on the topic. The federal government may currently hold data through other existing programs. Even without creating any new reporting requirements, Congress would make great strides by authorizing the BCS to identify generally available cybersecurity data and aggregate and standardize those reports into a single source. Sources of data for the BCS will likely fall into one of five broad categories: publicly available data, survey data, government-held data, purchasable data and shared private data. A second factor will depend on the nature of data that is not already created and collected. Some data may be readily observable because it exists, even if not collected in a usable form. In other words, various systems might already produce data and analytics, but entities have yet to affirmatively capture this data. Other types of data probably do not exist but might be created through surveys, a common collection vector used by other federal statistical agencies. One benefit of a survey system is that it would probably not pose many novel legal or political issues in implementation.       Other types of data might require new enterprise implementation to create. There will be a spectrum of feasibly collectable data, from data that falls into the category of “already exists, we just have to find it” to “we have to have people change what they do
Conceptualizing the Mandate for the Bureau of Cyber Statistics posted first on http://realempcol.tumblr.com/rss