Fixing problems with cloud procurement
- By Michael Garland
The government's cloud computing push is gathering steam. The Trump administration is drafting a major new cloud policy (the first since 2010), and the Department of Agriculture began a pilot program with the newly created Cloud Center of Excellence, designed to drive government cloud adoption.
What's feuling this momentum? It's simple: When properly and strategically deployed, cloud saves buckets of time and money.
The cloud model is based on renting, on-demand, various computational resources from cloud providers, paying only for what is actually consumed, after the fact, on a monthly basis. Not only does this mean the cost of consumption is closely aligned to actual demand, but the provisioning of needed services can be almost instantaneous. Instead of ordering and installing a server, a virtual instance of a computer can be spun up by a cloud services provider in a matter of minutes, in a fully automated way, with zero human intervention. With a CSP account and a connection, you can self-serve virtually everything needed to build and deploy an application.
But wait, there's more. As the cloud providers have matured, the types of services they provide have significantly reduced the burden on software developers. Cloud service providers now offer many of the underlying algorithms and routines that developers once were forced to code or scavenge for themselves. Cloud-native software developers no longer focus on things like tuning databases, building storage solutions or managing capacity. Unglamorous but critical tasks like keeping software versions up to date with the latest patches also can be done by the cloud company.
The cloud value proposition is obvious. It's not an overstatement to say the old "on-premise" data-center model, for most applications, will soon go the way of the VHS video recorder. The advent of cloud computing has done for software developers what the medieval inn did for early European travelers – it has relieved them of the obligation to pack and drag along all their own stuff. This is a stunning generational shift and best represented by an amazing data-point: Netflix now accounts for approximately 40 percent of all U.S. evening internet traffic. Every pixel of Netflix content that finds its way to a phone, tablet or TV is delivered by Amazon Web Services, which has turned itself into a $20-billion-a-year global cloud colossus in just 12 short years.
While the future of IT isn't completely written, we do have the basic vocabulary -- and it starts with the word cloud. From a government perspective, however, do we know how best to buy cloud services and truly gain its economic advantage?
From an acquisition perspective, cloud offers a variety of unique problems for contracting officers. Firstly, a quick keyword search of "cloud" against the Federal Acquisition Regulation shows zero hits. The word simply doesn't appear.
Expanding the search to include all agency FAR supplements produces six references in the Defense Federal Acquisition Regulation Supplement, and one reference in the Defense Information Systems Agency's addendum. These sparse references are all focused on security and advise contracting officers to be wary of security concerns.
This lack of references in the FAR would be less problematic if cloud snugly fit into other procurement categories. Unfortunately, it does not.
Cloud is a particularly cagey item -- sometimes it feels like a product, and sometimes it feels like a service. And IT companies, looking to cash in on the branding cachet, are increasingly calling everything cloud, whether it fits the National Institute of Standards and Technology's technical definition or not. From an acquisition perspective, cloud is a bit of a mess.
Here are just some of the most obvious acquisition problems with cloud:
1. The value of cloud is based on paying only for what is consumed, in arrears, after the fact. Contracting officers are schooled on the "bona-fide need" rule, and the "Anti-Deficiency Act," which requires them to be able to define a fairly precise amount of money to obligate to a contract. Too much money obligated creates the problem of de-obligation at the end of the year, and returning money to the Treasury. Too little money obligated could force the agency into a deficit, which is a violation of the Anti-Deficiency Act.
Yet when a mission critical application is running in the cloud, what's an agency to do, if it had accidently underfunded it? Spin down the servers? Likewise, over-obligating funding as a precaution negates the value of cloud, and creates a bureaucratic nightmare for later recovery of said funds. And what happens when the government temporarily shuts down? Should the cloud application be shut off, the same way a services contractor can be sent home? Perhaps there should be way to color cloud money, to negate some of these tricky funding issues.
2. Government contracting requires well-crafted scopes of work and precise definitions of the items to be procured. This level of precision is hard with cloud. Providers have a huge catalogue of rapidly changing services. AWS for instance, added over 200 services last year, and once they were added, they were instantly available to all contract holders. Rapidly changing and updating services is part of the cloud value proposition.
To make matters more complicated, the pricing for these various items also can change on the fly, and there is no one size fits-all consumption metric. Some service pricing is based exclusively on time, while others are based on volume or quantity or a combination of all of the above. To cope with this, some government cloud acquisitions are being done by what's colloquially called "the gift card model." The contracting officer effectively loads up a gift card with a certain amount of funding that can be used for items the cloud service provider sells. This is a consumer-friendly approach, but without appropriate controls it could be a recipe for abuse. By analogy, with a Starbucks Gift Card, you can purchase anything from a single croissant to a hundred insulated travel mugs. The potential for an equivalent wide-ranging scope of cloud services needs to be controlled. Plus, the gift card concept still doesn't solve the underlying "bona-fide need" rule or the "anti-deficiency act" conundrum.
3. In general, agencies are rightfully obsessed with data security, but guidance on the proper terms and conditions to include in a federal contract do not exist. While there is ample opportunity for a contracting officer to accidently omit security requirements, there is even more opportunity for abuse by over-prescribing it.
I am personally aware of a government cloud contract that is approximately 400 pages with a full 300 pages dedicated to security. Isn't most of this security concern, already alleviated when an application has been properly vetted by the Federal Risk and Authorization Management Program (FedRAMP)? Unnecessarily prescribing security artificially inflates the price of the service, as a conscientious vendor will factor in added risk to the price.
Alternatively, overzealous security requirements may be so onerous that only less-than-scrupulous vendors may agree to them, without much intent to even understand them. With the well-thought-out security controls inherent in FedRAMP, there is likely an elegant way to create standard security terms and conditions designed to protect the government, where FedRAMP isn't protective enough. Perhaps this can be done with a collection of a few well-crafted clauses.
4. The fact that cloud feels a like a product and also like a service creates unique problems. It's particularly confusing, because consumers previously purchased all of their IT capability as individual hardware and software products. The cloud model takes traditional IT products, fractionalizes them and rents them as a service. When we characterize cloud as a service, as we currently do, it is unlikely that we can ever procure it from a small business, because the requirement of massive scalability makes it impossible for a small business to play in this space. Indeed, all the IT industry players in the infrastructure- or platform-as-a-service space are huge. It's also impossible to gain small business credit by using a small business reseller, because the requirement to provide at least 51 percent of the services while selling AWS or Microsoft Azures would not be feasible.
On the flip side, if we characterized cloud as a product, we could potentially gain the credit of small business resellers, under the non-manufacturer rule, by correctly determining there are no small businesses that make cloud. But there's another issue that I almost hesitate to address. If we characterize cloud as a product, we'd most certainly have to determine if it were Trade Agreement Act (TAA) complaint, meaning an analysis of where it was manufactured or significantly transformed would be an issue.
If cloud was a product and used hardware manufactured in China, where much of IT hardware is made, it would not be TAA compliant. By transforming non-TAA compliant hardware and software into a rental service, we obviate the traditional TAA analysis. Services and products are evaluated differently. But what if our cloud provider is using Huawei router devices manufactured in China, thought by some to include spyware? What if Russian-made Kaspersky anti-virus software was on every machine? Do we care about the nature of the products we deploy only when we possess them, and our concerns evaporate if we rent them as a service?
Despite these concerns, the government is reasonably proficient in purchasing large-scale cloud implementations like the CIA's Intel Cloud (provided by AWS) and the Defense Department's planned Joint Enterprise Defense Infrastructure cloud program. But these are huge programs that bring together the best and brightest in government procurement (lawyers and contracting officers) who work together for many months to create a large-scale, FAR Part 8, customized solicitation. This won't work for every agency. What about the contracting officer who has been asked to procure a simple test-and-development environment, with a budget of $100,000, a desire to use the simplified acquisition procedures, and a preference for a small business? Where does this contracting officer turn for advice?
An overlooked opportunity?
There is a woefully under-utilized place in the FAR, where cloud guidance would nicely fit. Although the U.S. government spends north of $80 billion a year on IT, FAR Part 39, aptly named "Acquisition of Information Technology," remains mostly a dusty remnant of Clinger-Cohen from 1996, with little to no useful regulations for contemporary IT. Its primary focus can best be summed up as 1) a mandate to manage IT project risk by breaking up IT system development into small modular parts, and 2) assurance that IT systems meet the rules of 508 compliance so those with disabilities are not disadvantaged by non-complying IT applications. As recently as 2012, FAR Part 39 still included a provision for dealing with the Year 2000 problem.
With the government's dramatic and necessary push into cloud computing, the time is ripe to convene a panel of smart experts who possess technical, industry and federal procurement knowledge in order to lay down some basic guidance for contracting officers to make procurement of cloud as efficient and agile as the promise of its consumption.
This call to action isn't to build an overly prescriptive and burdensome new set of procurement rules. The right outcome would be to create a series of useful FAR provisions that would clarify ambiguities and give contracting officers a level of confidence in purchasing this new type of IT item, that defies easy categorization. Without some help and guidance for contracting officers, all we have done is reposition the cloud acquisition bottleneck from the point of data-center provisioning, back to the point of initial procurement.
To take full advantage of cloud, let's assess the entire universe of cloud procurement ambiguities, reduce those issues to manageable and clear guidance, and thereby significantly improve the government's chances of realizing the full benefits of cloud.
Michael Garland is the founder of Garland, LLC, a consulting firm that advises clients on issues related to federal procurement law and the business of IT.