Next week, many of us here will be heading down to Las Vegas for Black Hat. The MSRC, and other
teams in Microsoft, have been attending Black Hat for years. In fact, we've been sponsoring the show for the last eight
years-the last five as a platinum sponsor. Some might ask why? It's funny, I can actually remember back in my days as an
officer protecting networks in the U.S. Air Force, questioning why Microsoft had such a presence at the show. As much as I'd
like to say it's because of the weather (after all, most of us are over here in the rainy Northwest), or because it's the
largest security conference out there (it's not), or even better, because we so look forward to getting our next Pwnie
Award-the truth is it's none of the above. Well, maybe just a bit on the Pwnie. But the reality is that to us, Black Hat has
always been a reflection of, and driven by, the community-likeminded people from all walks of life and professions with a
shared interest in advancing the state of security. They come together to share ideas, advance thinking, network and
collaborate, and ultimately learn from one another. We feel connected to that and always look forward to being a part of
So with the show fast approaching, I've taken some time to reflect on where the Microsoft Security Response
Center is currently and where we see ourselves going with respect to security. Specifically, I've been thinking a lot about
three areas: 1) our work to address vulnerabilities in our software, 2) our work with the security community and 3) our
philosophy on vulnerability disclosure. Given the fact that each of these topics have recently garnered interest and fueled
discussion in the community and media, I thought I'd share my thoughts.
Vulnerabilities and Time to Fix
Some will say that we take too long to fix our vulnerabilities. But it isn't all about time-to-fix: Our chief priority with
respect to security updates is to minimize disruption to our customers and to help protect them from online criminal
attackers. These customers own and operate a diverse ecosystem of nearly a billion systems worldwide. It's humbling to think
about the responsibility this entails and yet we embrace the challenge. Even in the face of that, our overall track record
shows the window of vulnerability is being reduced and we have additional plans to improve.
The Microsoft Security
Response Center (MSRC) receives more than 100,000 e-mail messages per year at email@example.com - that's nearly 275 per
day or 11 per hour. This is filtered down to approximately 1,000 legitimate investigations per year. Once a vulnerability has
been confirmed, a comprehensive examination is undertaken to ensure that the reported vulnerability is addressed, other
vulnerabilities that might exist in related code are identified and addressed, and no new vulnerabilities or bugs are
introduced during this process.
But why don't we commit to fixed timelines? Because it is important to consider
the overall customer risk when focusing on updating software for security issues. Most security updates released by the MSRC
will be rapidly deployed to hundreds of millions of systems worldwide helping to protect customers from attacks in a very
short timeframe. And the software being updated is being used by hundreds of thousands of applications on all sorts of
hardware in all sorts of scenarios. So it is imperative that the update has been rigorously engineered and tested in order to
avoid creating any type of disruption to these systems. During this time, the MSRC monitors for signs that the vulnerability,
or variants, are being used in active attacks. The MSRC does this by using comprehensive telemetry systems as well as data
and information provided by customers and partners around the world, and the rest of the industry. This approach helps
Microsoft balance between the potential urgency of releasing an update for a particular vulnerability and ensuring high
confidence that the update will address the vulnerability, all of its variants and maintain the functionality and stability
that customers expect from the affected products.
Many times the issue that the finder reported is an indication
of other similar vulnerabilities in that area of code. And the original issue may not be the most complicated, or even the
most likely to get used in attacks. Microsoft tries to address vulnerabilities and all of their variants in as few updates as
possible because they cost enterprise customers time, effort and money to re-assess and deploy multiple updates for issues
that could potentially be addressed in a single update. The time it takes to complete a comprehensive examination helps to
ensure the number of security updates Microsoft releases and needs to re-release is kept to a minimum, thus reducing the
costs and potential disruption to enterprise customers' operations. Due to the increase in quality that Microsoft has
achieved over the last five years, some enterprise customers deploy security updates with little or no testing, and hundreds
of millions of consumers continue to use the Automatic Update client on their systems to ensure that they stay protected
For the majority of issues, we are able to release high quality and comprehensive security updates
to customers well before any indication of attacks, and well before they are disclosed publicly. However, there are
exceptions. In some cases attacks result, and when that happens, we have to compress testing to release updates quickly.
Also, when there are attacks, we release workarounds in days that can block these attacks even without the updates. Usually
these take the form of a "FixIt" that can protect customers with one click or be easily deployed throughout the
However, there are cases that take much longer. In fact, last year at Black Hat there was a security
event dealing with a vulnerability in a library called "ATL" or "Active Template Library." That issue affected not only
multiple Microsoft product versions, but also several 3rd party products and services. It took over a year to coordinate that
release, and in the end, even the finders themselves understood and commented that with the complexity involved, taking over
a year wasn't unreasonable. When seemingly simple security issues, such as a memory corruption bug, affect multiple different
products, the coordination and calibration can drive longer timelines so no product, or customers of those products are left
behind. And there have also been cases that are such deep architectural changes that they can take multiple years to fully
resolve or may not be able to be resolved in some of our older products. Usually these issues result from new threats
emerging that product designs or assumptions couldn't anticipate. Changing those assumptions for products that have been in
market for several years does take time and coordination so customers and applications can work effectively with them.
Focusing on resolving security issues has and will always be a priority for us. And work to improve our processes
will continue, but we must always strike a balance between timeliness and quality.
Working with the Security
The topic of how well Microsoft works with the security community is important to me personally, and to
my team. Years ago, this was a very valid concern. I can remember being on the outside of Microsoft and watching researcher
discussions noting how Microsoft wouldn't engage or was unresponsive. We've made dramatic changes on this front since the
inception of Trustworthy Computing. At Microsoft we recognize, and appreciate, the unique value that security researchers
play in identifying issues and helping the entire computing ecosystem improve from a security perspective. We also thank many
in the community for their collaborative work over the years, and for nearly a decade we have demonstrated our commitment to
working with them in an honest and transparent manner. We may not always agree on the severity and the amount of time it
should take to develop and test an update that has to work with hundreds of millions of computers, but we do believe we're
fair and open when working with researchers. It's not in our interest or the interest of our customers to behave any
Throughout the years we've seen researchers saying that if vendors really valued their work, we'd
compensate them directly for the vulnerabilities they discover. That's a trend that's continued in recent weeks. We
absolutely value the researcher ecosystem, and show that in a variety of ways. The most well-known is the fact that we
acknowledge the researcher's work in our bulletins when a researcher has coordinated the release of vulnerability details
with the release of a security update. And that's just the tip of the iceberg. We also work to make sure we can support the
community's development by sponsoring and supporting nearly 50 security conferences in over 20 countries each year.
Probably the community effort that started more of the deeper relationships we've built with researchers is our own
little "hacker" conference we host at Redmond each year, called "BlueHat Security Briefings." Launched in 2004, this
conference is aimed at bringing Microsoft security professionals and external security researchers together in a relaxed
environment to promote the sharing of ideas, social networking and ultimately improving the security of Microsoft products.
Key to the success of BlueHat and its benefit to our customers is the direct question-and-answer access that researchers get
with the specific owners of the technology they're researching. In many cases, some of our direct competitors have sat on our
stage at Microsoft and talked about problems in our products, directly to the folks that develop and manage them. And they've
been able to get feedback on their research from the same folks as well.
The Shift to Coordinated Vulnerability
If there's one area that has had had staying power in terms of driving polarized debate in the broader
security community-as manifested in mainstream and social media this past month-it's in how to disclose vulnerability
details. Ideally, updates for those vulnerabilities are available for all customers before details are broadly available.
This allows us to protect the end-users because they just get the updates automatically, and large Enterprises can analyze,
prioritize and deploy updates to hundreds of thousands of systems quickly. When communication breakdowns and disagreements
happen, resulting in vulnerability details disclosed by researchers before we release an update, those details are then used
by criminals to attack our customers. The worst situation is when vulnerabilities aren't disclosed to the vendor at all,
because then there's very little hope of broad protections ever getting released for all customers.
this range of situations, we also see a range of philosophies. Of course, Microsoft always supported the position that the
best way to disclose issues is in a coordinated fashion, where details of the vulnerability are released in conjunction with
an update that is broadly available for customers. This is known as "Responsible Disclosure." The term itself can be
subjective because if either party doesn't abide by those terms, it is implied that they themselves are "irresponsible."
Debate on this very issue of responsibility is understandable; however, it is important to remember that in the end we are
dealing with customer safety issues - and we should all take that seriously. It is unfortunate these debates can make us lose
focus on what is really important - protecting people using the Internet from harm.
Today, Matt Thomlinson, the
general manager of Security at Trustworthy Computing, introduced a new disclosure philosophy Microsoft is adopting called
Coordinated Vulnerability Disclosure http://blogs.technet.com/b/msrc/arch...isclosure.aspx . Katie Moussouris, senior
security strategist on the MSRC Ecosystem Strategy team, provides more information and insight on the necessity of this shift
in disclosure philosophy and practice on the MSRC Ecosystem Strategy Team Blog
http://blogs.technet.com/b/ecostrat/...the-force.aspx. You'll see from her post, we're not alone in acknowledging it is time
for a change. Other vendors and researchers from the broader community of defenders are supportive and will be instrumental
in making this shift a reality. So read the post, provide your feedback and then join us in making this an industry wide
Now back to the catalyst for this post-Black Hat. We're just a few days from the event itself and we'll
likely see more themes develop once it kicks-off. But I hope the thoughts I've shared here provide some insights into our
point of view on recent discussions in the community.
The realities of today's threat landscape point to a world
that has shifted from a variety of participants with various motives to one of two sides-those who intend to harm or commit
crime and those who intend to prevent harm and fight crime. As an industry and community, philosophical differences or
competition aside, we should be in this together. Our own welfare as individuals and a collective community is at stake with
unseen criminals who show no indication of backing down. It's our hope that this effort to shift to a shared responsibility
of coordination and collaboration is something that is carried beyond Black Hat as we progress and evolve as a global
community of defenders.
Hope to see you at Black Hat!