{"id":2157,"date":"2012-06-15T13:15:20","date_gmt":"2012-06-15T13:15:20","guid":{"rendered":"http:\/\/cloudnewsdaily.com\/?p=10661"},"modified":"2012-06-15T13:15:20","modified_gmt":"2012-06-15T13:15:20","slug":"bursting-into-the-clouds-experimenting-with-cloud-bursting","status":"publish","type":"post","link":"https:\/\/icloud.pe\/blog\/bursting-into-the-clouds-experimenting-with-cloud-bursting\/","title":{"rendered":"Bursting into the Clouds \u2013 Experimenting with Cloud Bursting"},"content":{"rendered":"<\/p>\n

Guest Post by Dotan Horovits, Senior Solutions Architect at GigaSpaces<\/a><\/strong><\/p>\n

\n\t\"Dotan<\/p>\n

Dotan Horovits is one of the primary architects at GigaSpaces<\/p>\n<\/div>\n

Who needs Cloud Bursting?<\/strong><\/p>\n

We see many organizations examining Cloud as replacement for their existing in-house IT. But we see interest in cloud even among organizations that have no plan of replacing their traditional data center.<\/p>\n

One prominent use case is\u00a0Cloud Bursting<\/em>:<\/p>\n

Cloud bursting is an application deployment model in which an application runs in a\u00a0private cloud<\/a>\u00a0or data center and\u00a0bursts<\/a>\u00a0into a\u00a0public cloud<\/a>\u00a0when the demand for computing capacity spikes. The advantage of such a\u00a0hybrid cloud<\/a>\u00a0deployment is that an organization only pays for extra compute resources when they are needed.
\n[Definition from\u00a0SearchCloudComputing<\/a>]<\/em><\/p>\n

Cloud Bursting\u00a0appears to be a prominent use case in cloud on-boarding projects.\u00a0In a recent\u00a0post<\/a>, Nati Shalom summarizes nicely the economical rationale for cloud bursting and discusses theoretical approaches for architecture. In this post I\u2019d like to examine the architectural challenges more closely and explore possible designs for Cloud Bursting.<\/p>\n

Examining Cloud Bursting Architecture<\/strong><\/p>\n

Overflowing\u00a0compute<\/em>\u00a0to the cloud is addressed by\u00a0workload migration<\/em>: when we need more compute power we just spin up more VMs in the cloud (the secondary site) and install instances of the application. The challenge in workload migration is around how to build a consistent environment in the secondary site as in the primary site, so the system can overflow transparently. This is usually addressed by\u00a0DevOps<\/a>\u00a0tools such as\u00a0Chef<\/a>,\u00a0Puppet<\/a>,\u00a0CFEngine<\/a>\u00a0and\u00a0Cloudify<\/wbr><\/a>,\u00a0which capture the setup and are able to bootstrap the application stack on different environments. In my example I used Cloudify to provide consistent installation between EC2 and RackSpace clouds.<\/p>\n

The Cloud Bursting problem becomes more interesting when\u00a0data<\/em>\u00a0is concerned. In his\u00a0post<\/a>\u00a0Nati mentions two approaches for handling data during cloud bursting:<\/p>\n

1. The primary site approach<\/strong>\u00a0– Use the private cloud as the primary data site, and then point all the burst activity to that site.
\n2. Federated site approach<\/strong>\u00a0– This approach is similar to the way Content Distribution Networks (CDN) work today. With this approach we maintain a replica of the data available at each site and keep their replicas in sync.<\/p>\n

The primary site approach incurs heavy penalty in latency, as each computation needs to make the round trip to the primary site to get the data for the computation. Such architecture is not applicable to online flows.<\/p>\n

The federated site approach uses\u00a0data synchronization<\/em>\u00a0to bring the data to the compute, which saves the above latency and enables online flows. But if we want to support \u201chot\u201d bursting to the cloud, we have to replicate the data between the sites in an ongoing streaming fashion, so that the data is available on the cloud as soon as the peak occurs and we can spin up compute instances and immediately start to redirect load. Let\u2019s see how it\u2019s done.<\/p>\n

Cloud Bursting \u2013 Examining the Federated Site Approach<\/strong><\/p>\n

Let\u2019s put up our sleeves and start experimenting hands-on with the federated site approach for Cloud Bursting architecture. As reference application\u00a0let\u2019s take Spring\u2019s\u00a0PetClinic Sample Application<\/a>\u00a0and run it on an\u00a0Apache Tomcat<\/a>\u00a0web container. The application will persist its data locally to a\u00a0MySQL<\/a>\u00a0relational database.<\/p>\n

The primary site, representing our private data center, will run the above stack and serve the PetClinic online service. The secondary site, representing the public cloud, will only have a MySQL database, and we will replicate data between the primary and secondary sites to keep data synchronized. As soon as the load on the primary site increases beyond a certain threshold, we will spin up a machine with an instance of Tomcat and the PetClinic application, and update the load balancer to offload some of the traffic to the secondary site.<\/p>\n

On my experiment I used\u00a0Amazon EC2<\/a>\u00a0and\u00a0RackSpace<\/a>\u00a0IaaS providers to simulate the two distinct environments of the primary and secondary sites, but any on-demand environments will do.<\/p>\n

\n

REPLICATING RDBMS DATA OVER WAN<\/strong><\/p>\n<\/div>\n

How do we replicate data between the MySQL database instances over WAN? On this experiment we\u2019ll use the following pattern:<\/p>\n

1.\u00a0\u00a0\u00a0\u00a0\u00a0Monitor data mutating SQL statements on source site<\/strong>. Turn on the\u00a0MySQL query log<\/a>, and write a listener (\u201cFeeder\u201d) to intercept data mutating SQL statements, then write them to GigaSpaces\u00a0In-Memory Data Grid<\/a>.<\/p>\n

2.\u00a0\u00a0\u00a0\u00a0\u00a0Replicate data mutating SQL statements\u00a0over WAN<\/strong>. I used GigaSpaces\u00a0WAN Replication<\/a>\u00a0to replicate the SQL statements \u00a0between the data grids of the primary and secondary sites in a real-time and transactional manner.<\/p>\n

3.\u00a0\u00a0\u00a0\u00a0\u00a0Execute data mutating SQL statements on target site<\/strong>. Write a listener (\u201cProcessor\u201d) to intercept incoming SQL statements on the data grid and execute them on the local MySQL DB.<\/p>\n

\"\"<\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

To support bi-directional data replication we simply deploy both the Feeder and the Processor on each site.<\/p>\n

\n

AUTO-BOOTSTRAP SECONDARY SITE<\/strong><\/p>\n<\/div>\n

When peak load occurs, we need to react immediately, and perform a series of operations to activate the secondary site:<\/p>\n

1.\u00a0\u00a0\u00a0\u00a0\u00a0spin up compute nodes (VMs)<\/p>\n

2.\u00a0\u00a0\u00a0\u00a0\u00a0download and install Tomcat web server<\/p>\n

3.\u00a0\u00a0\u00a0\u00a0\u00a0download and install the PetClinic application<\/p>\n

4.\u00a0\u00a0\u00a0\u00a0\u00a0configure the load balancer with the new node<\/p>\n

5.\u00a0\u00a0\u00a0\u00a0\u00a0when peak load is over \u2013 perform the reverse flow to tear down the secondary site<\/p>\n

We need to automate this bootstrap process to support real-time response to peak-load events. How do we do this automation? I used GigaSpacesCloudify<\/a>\u00a0open-source product\u00a0as the automation tool for setting up and for taking down the secondary site, utilizing the out-of-the-box connectors for EC2 and RackSpace. Cloudify also provides self-healing \u00a0in case of VM or process failure, and can later help in scaling the application (in case of clustered applications).<\/p>\n

Implementation Details<\/strong><\/p>\n

The result of the above experimentation is available on\u00a0GitHub<\/a>. It contains:<\/p>\n

\u00a7\u00a0\u00a0DB scripts for setting up the logging, schema and demo data for the PetClinic application<\/p>\n

\u00a7\u00a0\u00a0PetClinic application (.war) file<\/p>\n

\u00a7\u00a0\u00a0WAN replication gateway module<\/p>\n

\u00a7\u00a0\u00a0Cloudify recipe for automating the PetClinic deployment<\/p>\n

See the documentation on\u00a0GitHub<\/a>\u00a0for detailed instructions on how to configure the above with your specific deployment details.<\/p>\n

Conclusion<\/strong><\/p>\n

Cloud Bursting is a common use case for cloud on-boarding, which requires good architecture patterns. In this post I tried to suggest some patterns and experiment with a simple demo, sharing it with the community to get feedback and raise discussion on these cloud architectures.<\/p>\n

More information can be seen at an upcoming GigaSpaces webinar on Transactional Cross-Site Data Replication on June 20th (register at:\u00a0http:\/\/bit.ly\/IM0w9F<\/a>)<\/p>\n

\"\"<\/div>\n

<\/img><\/a>
\n<\/img><\/a><\/p>\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

Guest Post by Dotan Horovits, Senior Solutions Architect at GigaSpaces Who needs Cloud Bursting? We see many organizations examining Cloud as replacement for their existing in-house IT. But we see interest in cloud even among organizations that have no plan of replacing their traditional data center. One prominent use case is\u00a0Cloud Bursting: Cloud bursting is […]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[521,522,78,523,443],"tags":[112],"class_list":["post-2157","post","type-post","status-publish","format-standard","hentry","category-apache-tomcat","category-cloudify","category-guest-post","category-mysql","category-rackspace","tag-devops"],"_links":{"self":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts\/2157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/comments?post=2157"}],"version-history":[{"count":0,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts\/2157\/revisions"}],"wp:attachment":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/media?parent=2157"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/categories?post=2157"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/tags?post=2157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}