Today, firms must learn to manage increasing data sizes, letting go of dark and redundant data or risk inefficiencies and high costs, enterprise data platform firm Veritas tells AMEinfo.
As organizations worldwide continue to find themselves part of a wave of digital transformation, they find that their data needs are growing. With increasing data comes a need for an increase in storage capacity. Over time, it can be quite expensive to manage the accumulating data, and organizations soon find themselves faced with dark (unclassified) data, as well as Redundant, Obsolete, or Trivial (ROT) data, the products of months and years of accumulated data that eventually outlives its usefulness.
“Not all data is born equal,” Johnny Karam, Vice President of Emerging Markets at Veritas, tells AMEinfo, commenting on the usefulness of differing types of data.
Veritas recently released the 3rd edition of its “Middle East Databerg Report,” which revealed that UAE businesses surveyed are failing to manage their dark and ROT data, which is slowing cloud adoption. Sitting at 88% of total company data, dark and ROT data has significantly contributed to respondents failing to reach their own targets for cloud migration. While last year’s prediction of cloud storage uptake was 55%, the actual uptake this year was just 45%.
We asked Karam about this and more, hoping to learn how companies can manage this data, and ease their transition to the cloud.
In its latest study, Veritas revealed that UAE cloud migration was slowing down due to firms failing to manage their dark (unclassified) and Redundant, Obsolete or Trivial (ROT) data. What other negative outcomes can we anticipate from this malpractice?
Data is growing at phenomenal rates. What fueled this?
Here are the problems that will arise if we don’t manage data properly.
We cannot keep on using more storage to address the problem of data. It’s not a sustainable solution. 1. From a cost perspective: Every year you need to buy more storage. 2. From a space perspective: Where do you place all of the storage units in your data center? You’re going to run out of space sooner or later. Your data center bill will go up, your electricity bill will go up, your cooling bill will go up. More storage means more team members will be needed to manage it. Therefore, it is not a sustainable solution for organizations to not manage the data properly – especially with dark and ROT data growing at phenomenal rates. We can no longer fix it by throwing more storage at it.
If we don’t automate, classify and manage that data, it will continue to be a problem for most organizations.
What are the most ideal solutions to the mounting dark and ROT data issue?
Only 12% of the data stored by organizations is clean – the rest is just ROT and unclassified data. Before organizations go and buy additional storage, they need to consider the following.
The first step is to get visibility on their data estate, which could be sitting on physical servers, virtual servers, or on the cloud you’re renting. They need to check their storage units and actually look at what’s stored in them. This is the first step we undertake with most of our customers – to seek visibility on data.
When we gain visibility, we sit down with the customer and explore how we can classify the data – what’s important, and what’s not?
Not all data is born equal. If you are a bank, data that involves account numbers and customer information is clearly more important than a marketing video that you created 5 years ago that you’re no longer using. What happens is we set a policy with the customer to decide how to classify each type of data as per the requirements of their organization. For example, they could say that any marketing video that exceeds one year of age can be relegated to the cheapest storage available. As per the same policy, customer data would be given the highest priority on the best quality storage, backed up 3 times a day and replicated across three different locations.
So, once they decide on the policy, this is where we come in. Now, we implement automation solutions. This is not about doing it once – you need to ensure this is applied all the time. Therefore, once we implement the policy and automate it, the minute that video turns one year old, it’s automatically shifted to the cheapest storage or deleted, as per the chosen policy.
Finally, you have the clearing of storage.
Is a lack of qualified personnel or cybersecurity knowledge the reason behind this mismanagement of data? Whose responsibility is it to make sure that all data is stored, backed up and disposed of properly?
It is the responsibility of the organization. Think about it this way: It’s not just about managing your data, growth and cost associated with it. There is a compliance issue to consider as well.
Karam explains how there are regulators to keep in mind, who might at any moment request all information regarding a specific person, if you’re a bank, for example. The organization needs to retrieve this data whenever it is demanded, be it in a combination of databases, emails, documents, etc.
How do you ensure you can retrieve that information? That’s another reason why firms need to have visibility of their data.
What would you say the cause is from an organizational perspective?
Surely enough, there needs to be a mindset shift that needs to happen within organizations – from the top down. The first step towards this shift needs to be the realization that you can’t solve data management problems by just ‘throwing more storage’ at it – it’s not sustainable.
The second step is education and awareness – organizations need to realize that there are different ways of managing data. It’s no longer just about tools, but strategy. The first question I ask many CIOs and CEOs is ‘what is your data management strategy?’ More likely than not, they often have a general idea about their strategy, but it’s not written down.
He also explains how a proper data management strategy is in fact a competitive advantage, explaining how a company like Uber has a competitive advantage with the user information it collects and manages, which could be used to improve customer UX with things like suggested destinations, routes, etc. Companies without a proper data management strategy will suffer from a lack of competitiveness in this day and age.
What is the full potential of the cloud when used properly in a prospering country like the UAE? What do private and public sector firms stand to gain from this?
There are multiple advantages to using the cloud. First, the large cloud providers have the benefit of economies of scale. Because they are managing a huge amount of servers and massive storage capacities, the cost per Terabyte for them would be cheaper and more cost-effective than if a company where to purchase a handful of storage units and servers on its own.
Secondly, from a security perspective, organizations will benefit from all of the tools and security systems in place that these big cloud providers have invested in.
Thirdly, because these cloud providers are managing a lot of data, they start observing trends that organizations using their services can benefit from. These are some of the advantages of working with cloud providers.
One of the biggest questions organizations face today, even if they decide on moving to the cloud, is which data do we move? Where do we start – do we move everything? This is where a company like Veritas comes in. We help organizations identify which data is critical and help them automate their data management.
Generally, do you believe the UAE is in a good place from a data management perspective? Your 2019 Middle East Databerg study identified that confidence in the cloud was waning.
The way I would look at this is that cloud adoption has not been as fast as anticipated. One of the key reasons for this, as explained earlier, is that these organizations don’t know where to start. They need a data management strategy, and someone to help them gain visibility on their data, understand compliance and a lot more. With data capacities increasing faster and faster, it’s becoming more difficult to manage without clear visibility.