big data in cloud

Shadow cloud tempts researchers with fast deployments, no bureaucracy

Providers such as Amazon, Google and Microsoft are luring college and university researchers to bypass IT for cloud-based high-performance computing resources.

Although enterprise IT managers have long wrestled with shadow IT, the practice is evolving as users move their work into the cloud. Besides access to software and services not officially supported by the IT department, the cloud's ability to provide virtually unlimited compute and storage resources is luring researchers and reigniting a long smoldering argument about managing unsanctioned tech.

Employees at NASA, which has been among the agencies most aggressively pursuing federal "cloud first" initiatives, are using cloud services without going through tech officials, according to NASA's internal watchdog

A Feb. 7 Office of Inspector General  report found that not all of the agency's cloud systems were registered through the approval process for the Federal Risk and Authorization Management Program, and that several services lacked authorizations to operate "and were not covered by an IT system security plan."

Besides services like Dropbox and Evernote that appeal to convenience, cloud "offers a kind of instant gratification and nimbleness that is very good for research," said Erik Deumens, research computing director at the University of Florida, a large public research university in Gainesville.

Additionally, cloud providers are marketing their offerings directly to government and academic researchers. Amazon's AWS Research Cloud Program promises to help them to "focus on science, not servers." Microsoft's Azure 4 Research program claims that its cloud platform "can help with almost any research computing task." Google sees itself as a supporter and participant in the research community, and promotes programs that provide funding for research enabled by the Google Cloud Platform.

These are effective pitches, and they're getting researchers' attention for obvious reasons: fast deployments and no bureaucracy, said Patrick Mungovan, VP of Oracle's Higher Education, Research and Academic Medical Center Technology Sales group.

"If you look at what cloud does, it can be an incredible enabler for university researchers," he said. "The types of research we're seeing these days cross a variety of disciplines, and a lot of them utilize sensor data and the internet of things, which generate a staggering amount of data. The cloud is really the only thing that provides the bursting ability that allows researchers to take on massive amounts of information and stand it up or stand it down, depending on what they want to do with it."

It's also likely that the researchers who go directly to a cloud services provider are not exercising the due diligence a circumspect CIO would pursue for what are essentially software-as-a-service agreements, said Edward Chapel, senior VP at, a nonprofit technology consortium of academic and research institutions in New Jersey. "The researchers are not IT administrators," he explained. "They just want to get their work done, and the cloud helps them to avoid the often steep climb they face going through the local technology organization."

The ability to stand up a cloud environment in minutes without IT department oversight does come with risks. According to IT industry analysts at Gartner, by 2020 more than a third of successful attacks on organizations will be accomplished through their shadow IT resources. But it's important to keep in mind that Gartner's prediction isn't a knock against cloud computing, per se, but a reminder of the risks posed by IT assets that are essentially invisible to the IT department.

"The thing that people forget," Deumens said, "is that once you get a virtual machine from Amazon, you own it and you're responsible for its configuration and its system administration. That's okay if all you have are simple problems, but what happens when bigger problems arise, when security is not done properly, when patches aren't applied, or when there are new mandates on properly managing restricted data? Some of this stuff is really hard, and that's when shadow IT becomes a real risk."

Gartner's prediction suggests that shadow IT will be with us for the foreseeable future, which makes old strategies for rooting out and eliminating unmanaged technologies seem like futile exercises. In fact, Gartner recommends establishing a culture of "acceptance and protection versus detection and punishment" to organizations looking for solutions to their shadow IT challenges.

"Don't think about limiting access, but filtering access and establishing a base-level control gate," Mungovan suggested.

The University of Florida is going even further with an emphasis on dramatically increased access for its researchers through a model Deumens calls "research as a service." He said the school has come a long way in the management and orchestration of its shadow IT by drawing researchers back behind the firewall, where they find faster deployments and computing resources enhanced by public cloud capabilities.

"It's definitely more carrot than stick," he noted. "We created an environment that is dedicated to research computing that exists for the most part on our own infrastructure. People get an allocation in the form of the number of cores and number of terabytes of storage, which gets allocated to them within one to two business days. We tried to make it a deal they couldn't refuse, and it really seems to be working."

Although the bulk of the university's research-as-a-service offering is provided locally, the school adds elasticity to the service through the cloud, Deumens said. It's one of the secrets of the strategy's success, for both the university and the researchers.

"We first make sure that everyone is using our resource," he said, "then we know exactly what the workloads are and how we can support them with our small staff. Once we have them all supported on our local, private cloud infrastructure, then we can judge whether we should buy more cores and expand the hardware of our local infrastructure, or burst out this workload into the cloud and make sure that it's cost effective."

Elias Eldayrie, the university's CIO, is the driving force behind Florida's research-as-a-service strategy, Deumens said. "To make this work, Elias had the entire upper administration aligned around the idea. He created a partnership with the provost and VP for research to establish this centralized infrastructure and to get people to buy into and use it."

That kind of interdepartmental communication is essential to the success of any strategy for coping with shadow IT, Deumens insisted.

About the Author

John K. Waters is a freelance journalist and author based in Palo Alto, CA.


Charter Sponsors