I spent last week in Las Vegas attending VMworld 2018. Over the years I’ve been to many IT conferences both as an individual attendee and while representing a vendor. This time around, I had on my vendor hat doing booth duty. Which means I stood in the expo hall at my employer’s booth with my fellow engineers and spoke to the passersby who wanted to see a demo of our product and talk about our technology.
My employer was very kind this year in giving booth staff 4 hour shifts instead of a full 8-hour day (thanks!). I worked 3 days straight for 4 hours a day for a total of 12 hours (give or take). Each hour I probably spoke to about 4-5 people at length giving a full demo of our product and talking about their current storage technology. For the sake of argument, let’s say I spoke to a total about 50 people in total during my 12 hours working at the conference.
I had a fun time working at the conference and all the conversations I had at the booth with customers were great. But after talking about their current technology and their future business goals for data storage, one thing struck me as honestly surprising. These customers were not using and were generally not interested in public cloud storage.
I currently work in the field of data protection and data management. I’ve been specializing in data storage for most of my career. I’ve seen firsthand the explosion in the sheer amount of data stored in the enterprise datacenter over the past 15+ years. We generate more data today than we ever had in the past. And one of the greatest challenges to solve for is how to efficiently store this data without spending an inordinate amount of time and money on the problem.
Just because we have lots of data doesn’t mean it’s all important. Most data is probably not that important. But it’s often more difficult to classify data as important than it is to just keep it around. It also difficult for individuals and their managers to decide if things can be deleted, will the business need this data later? Isn’t it easier (and safer) to just keep everything? So what companies often do is just keep everything and try to find a cheap bulk platform for this data of questionable value.
Bulk data storage platforms have been around for many years and are most often located in the same datacenter as the networking, servers, and the storage platforms for more important data. Tape is the most common storage medium for bulk data historically but spinning disk arrays are also popular for storing petabytes of archive data. Software and data protection applications have the ability to move data around and usually have the ability to take older data that is less frequently accessed and stick it on bulk storage that is somewhere in the datacenter on a tape or slow spinning disk drive. This was true 10 years ago and is still true today.
And I get it. You get a solution that has worked for many years and it seems like a good enough solution. But onsite bulk storage systems might gradually become expensive to maintain for the limited value they provide. And their vendors may deliver very little innovation over the years and keep the architecture in business as usual mode. But then again, the system is what everyone is used to, what everyone is comfortable with, and it mostly works.
Public cloud, no thanks!
Public cloud for bulk storage? No thanks! This was the overwhelming feeling I got when speaking with VMworld attendees. These were customers, not other vendors, who were working every day with data protection, data management, and data storage. I would simply ask the folks I spoke with where they were keeping their bulk data and if their plans included moving this data to the big public cloud vendors – AWS, Microsoft, or Google. And among the ~50 people I spoke with, the answer was a resounding “no”.
I’ll be the first to admit my sampling methods here are flawed and this is in no way representative of all customers at the conference. Maybe only big iron customers wander into the vendor hall. Or maybe only old-school sysadmins ask a vendor for a demo. I doubt either of these were the case, but I will say this was an unscientific poll performed by me and I’m using some casual conversations I had to make a point.
Traditional enterprise customers that buy hardware and software from their favorite tried-and-true legacy vendors seem to have little interest in exploring the options and benefits offered by the major public cloud vendors. Maybe this was reflective of their conservative employers that are risk and change adverse. Or maybe this was reflective of personal views of these customers. Either way, this seems odd to me and flies in the face of all current IT perceived trends and cost savings promised by the public cloud.
Operators and planners
I’ll break down the types of customers I spoke with into two categories. This is admittedly oversimplified but will make my point easier. End users of technology I spoke to at the conference were usually either technology operators or technology planners.
Technology operators are deep into the details of hardware and software solutions and run the day-to-day operations of IT shops. They press the buttons, turns the nobs, and fix things when they break. Operators become experts at the things they manage and know what vendor’s solutions works well and what might not work so well. Operators are often seen as domain experts in their field and their advice is valued by non-experts.
Technology planners are often looking at the big picture in terms of architectures, project timelines, and cost. Planners are less interested in the fine details and will go find an operator if that is necessary. A planner could be an architect who is building a larger solution of several point products. Or a planner might be a manager working directly with another set of planners on providing a solution that meets a set of business requirements. Planners are trying to solve a business problem and keep the solution at a certain cost.
When I asked technology operators about public cloud storage, these people looked at me quizzically and said they didn’t currently use it. Or they answered with an emphatic NO, as if AWS were going to take their job away and be a horrible end to an otherwise great career.
Technology planners had a bit more nuanced answer, they simply said that the business wouldn’t allow it due to some form of bureaucracy, lack of cost savings, or legal rules. They had looked at it before and it was a dead end – the project was never implemented, there was a perceived data sovereignty problem, or the savings just couldn’t be justified.
This is easy!
Putting bulk backup and archive data in the public cloud is pretty easy. Some might say it’s even a boring use case. Because both the onsite data storage vendors and the public cloud vendors have already solved this problem. Some may offer better integrations, some may be better solutions for certain use cases, but in general this problem has already been solved by the industry.
Is public cloud really cheaper? For bulk data storage, it is most likely cheaper than your current onsite storage product but your mileage may vary. Bulk storage prices are a race to the bottom in this market, if you are willing to pay by the month you can probably get a better deal and will continue to get a better deal as commodity storage prices drop. You can even tier data within the public cloud for even cheaper prices.
Is my data secure in the public cloud? Yes, if you encrypt everything at rest and in-flight and work with your vendors to make sure you get all the implementation details correct (hint – your storage operators can help here). Now if you have laws against your data movement or other data sovereignty issues you may be out of luck. But I wouldn’t get hung up on security, these are industry standard solutions that deploy agreed upon encryption algorithms.
This isn’t a commercial for my employer. And I get it, cloud may not be the right fit for every use case or every business. But your business may benefit greatly from storing bulk data on the cheapest possible platform which is the public cloud. If you aren’t using public cloud as a component in your current storage architecture, you are going to get left behind. Why get bogged down with millions of dollars with of complex storage gear onsite to store petabytes of data? It is only a matter of time before your business competitors are certainly going to take advantage of cloud prices and efficiency.
Call to action
Technology operators – start to learn about public cloud storage. You don’t need to be a cloud expert but take a look at object storage, learn about the tiers from AWS, Azure, and GCP, learn about the pricing, and compare those tools to the ones you have onsite in your datacenter. Learn about ingress, egress, puts, and gets. Learn how similar Glacier is to tape and how it differs from standard S3. Most importantly, you will be more valuable to your business if you have more tools available to you and are able to use them when it makes sense.
Technology planners, work with enlightened technology operators who are able to objectively build solutions with you that may not include the point products you have used for 15+ years. Revisit the possibility of storing data in different places and how you can save time and money. Find a vendor or trusted partner to walk you through a TCO and evaluate something new that might take advantage of public cloud.
I hear business leaders who are served by the IT department say “we aren’t in the datacenter business” but I don’t see much action taken in seriously looking at the public cloud. Bulk storage is the easiest thing to put in the cloud so take a fresh look at the solutions available!
Thanks for reading!