Everything data will soon live in the cloud. That may be a bold statement, however there is no question that we have moved into a new era of information. An era where data is dispersed across the globe and demand for data has accelerated at a lightning pace. Sorry on-premise server farms, only the agility and flexibility of cloud technologies and architectures will accommodate that speed.
Enter RevGen’s Agile Cloud Foundation.
The concept of the Agile Cloud Foundation outlines the core components of cloud architectures. These are critical for scaling your organization to meet the ever-growing, data-driven demands of your customers and stakeholders while also remaining nimble.
In this article, we will share several “rules of the cloud” that the Agile Cloud Foundation addresses, both as opportunities to exploit and threats to mitigate. Our goal is to provide the key considerations and additional information to help you realize the promises of the cloud – the promises of scale, agility, and flexibility – while also avoiding the risks of an unbounded technology.
The core components outlined here as part of the Agile Cloud Foundation are not new in concept nor in cloud computing. However, their unique considerations for effectively leveraging the cloud while remaining agile is what makes these so important. One has to journey into the cloud with eyes wide open to avoid falling prey to misconceptions or be paralyzed by its limitless potential.
The cloud components key to agility, flexibility, and scale are:
- Cloud Management & Monitoring (CM&M)
- Data Storage
- Database & analytics Services
- Applications (specifically SaaS)
- Virtualized Compute
- Virtual Private Cloud (VPC)
Before we delve further into the core components of the Agile Cloud Foundation, let’s first set some context with RevGen’s three rules of the cloud.
RevGen’s Rules of the Cloud: Opportunity vs. Threat
1. The cloud is flexible and expandable.
In principle, every CPU, disk, and memory segment is virtualized, making cost the only actual ceiling in this virtual state. Understanding and working within these cost ceilings is the first challenge, as pricing can be a crucial growth limiter even to a scalable cloud framework.
2. The movement of data in the cloud regulates the speed of change.
The slowest, most oblique connection between a legacy system and the cloud is the cap on data movement, narrowing what could be a massive pipeline down to the width of a drinking straw.
Our second cloud challenge is to create networks (intra-cloud, inter-cloud, and between cloud and on-premise) to move data safely and at the greatest speed possible.
3. The cloud’s unbounded freedom.
Yes, the cloud presents seemingly endless possibilities, however, without careful implementation of your Agile Cloud Foundation, lack of governance – of data, resources, and users – will be an inhibitor to success.
Without effective cloud management, you lack the ability to observe everything within the cloud. See the forest, not just the trees.
This freedom means there is a high risk of losing control over your data, databases, and applications resulting in soaring costs (see Rule # 1) and security and privacy risks for your customer’s personal data, your organization’s intellectual property, and more.
Core Components of the Agile Cloud Foundation
Now that we’ve discussed the Rules of the Cloud, let’s explore how the core components of the Agile Cloud Foundation help seize the opportunity for scaled flexibility and mitigate the threats of cost overruns, data latency, and governance risk.
Cloud Management & Monitoring (CM&M)
Cloud Management and Monitoring are platform-level functions that orchestrate all significant areas of the Cloud. At the very minimum, CM&M functionality includes the following:
- Provisioning & Orchestration
- Security & Governance
- Monitoring & Logging
- Cost Management & Optimization
- Migration, Backup, Higher Availability, and Disaster Recovery
- Identity & Access Management
Much care and attention is needed at the onset of your cloud journey in each of the aforementioned functions. By enabling CM&M early, you avoid the risks outlined in our Rules of the Cloud – specifically risks of cost overruns and lack of data governance. It is much harder to rein in an out-of-control cloud environment after the fact than it is implement these functions correctly at the onset.
Each of these functions warrant their own white paper, but the key point here is simply that none of these critical ingredients can be neglected. Implement these CM&M components effectively, and you will have a solid grounding for realizing your digital, data, and analytics objectives in the cloud.
Leading cloud CM&M technologies include AWS CloudWatch, AWS CloudFormation, and AWS CloudTrail. Google and Azure offer similar technologies, such as Azure Cloud Resource Manager.
There are also commercial off-the-shelf tools that perform similar tasks to the native cloud management tools. Most notable are OpenStack, Apache CloudStack, Scalr, and Terraform.
Cloud data storage is based on virtualized infrastructure and, if implemented correctly, is highly flexible in terms of interfaces, near-instant elasticity and scalability, multi-tenancy, and metered resources.
The multiple data storage types available provide additional flexibility for varying storage needs. Storage types include object (or blob) storage, file storage, and block storage, which is often used for databases.
One size does not fit all. For instance, if you are primarily looking to store various shapes, sizes, and formats of data (i.e., unstructured data) to support analytics or downstream data warehousing, you should strongly consider object or file storage for this flexibility. If your primary concern is enabling storage and backups to enable virtualized environments (e.g., VMs), you should strongly consider block storage.
Picking the right storage types for your use cases is critical for the cloud’s ability to support your organizational needs.
Leading cloud storage technologies include Amazon Simple Storage Service (S3), Azure Blob Storage, and Google Cloud Persistent Disk Storage.
Database & Analytics Services
A cloud database is a database that operates within a virtual private cloud or can be centralized in a public cloud using multi-tenancy. There are two standard deployment models. In the first, users run databases on the cloud independently using a virtual machine image. Otherwise they can purchase access to a database service maintained by a cloud database provider (i.e., Database as a Service).
Database as a Service (DBaaS) takes care of all scalability and high availability needs and makes the underlying software stack transparent. Managing software releases and updates is the responsibility of the cloud provider, alleviating the administrative burden for the enterprise.
In addition, integrated AI and analytics services categorized as Analytics as a Service (AaaS) provide the ability to not only source and cleanse data, but also provide configurable analytics models to accelerate time-to-value from data science.
If low administration with high flexibility and speed to value are your goals, you should strongly consider DBaaS and AaaS for your needs.
Leading cloud database and analytics technologies include Snowflake, Databricks, Azure Synapse Analytics, Google Big Query, and AWS Redshift.
Computing in the cloud is executed as an on-demand service for running cloud-based applications. It provides computing resources such as multi-core processors as single units, in clusters, or in pools.
The latest class of cloud virtualized compute services, called managed container orchestration services (think Kubernetes or Fargate), allow for the dynamic allocation of compute services operating on very specific application tasks for very specific time periods.
Is the cost of always-on cloud resources a concern of yours? Well, a fundamental aspect of cloud compute services is paying for a specific process to execute its program. Therefore, the cost of the process execution should only occur while it is running.
Why pay for compute processing when it isn’t doing anything? If configured correctly (which is key), virtualized compute will allow you to control costs and provide scaling as needed.
Leading virtualized compute technologies include AWS Elastic Compute Cloud (EC2), AWS Elastic Container Services (ECS), Amazon Elastic Kubernetes Service (EKS), Azure Service Fabric, and Azure Container Services.
Vendor compute products are also available across all Cloud platforms, most notable are Databricks and VMWare.
Virtual Private Cloud (VPC)
A Virtual Private Cloud is a secure, isolated private cloud hosted within a public cloud. VPC customers can run code, store data, host websites, and do anything else they could do in an ordinary private cloud, but the private cloud is hosted remotely by a public cloud provider. The method for accessing an organization’s VPC is typically done through a Virtual Private Network (VPN).
This approach combines the security and peace of mind of a private cloud or on-premise data center with the flexibility and scalability of the public cloud. This also provides the same benefits of compliance with regulations you receive from a public cloud.
Note that there are built-in VPCs with many cloud services to leverage – it is important to research whether you are getting this level of security with your chosen cloud service or if you need a standalone VPC service.
Leading VPC technologies are inherent to many of the top cloud services (e.g., AWS Lambda) but also come as stand-alone services through the leading cloud platforms including AWS, Azure, Google Cloud, and Alibaba.
In this article, we have examined the core cloud services within the Agile Cloud Foundation and the key considerations that are critical for a successful journey to the cloud – from using cloud management tools, to provisioning SaaS applications along with compute, storage, and database resources, all governed by cloud monitoring tools.
A fully deployed Agile Cloud Foundation integrates services that flex intelligently. If implemented correctly and effectively, your systems in the cloud will not only be secured but also actively monitored and will protect your most valuable business asset – your data.
We hope that sharing some of these perspectives will help you in your cloud journey. Specifically, we hope you realize the promises of the cloud: scale, agility, and flexibility.
Undertake your cloud journey with these key considerations top of mind and the Agile Cloud Foundation will enable its success. We hope the cloud will be your impetus to meeting the ever-evolving demands of today’s tech-savvy customers and data-hungry stakeholders.
Not sure where to begin your cloud journey? Contact us to speak to one of RevGen’s experts.
Al Shain is a Principal Architect for RevGen Partners who is passionate about unlocking the potential of data.