{"id":2223,"date":"2012-06-20T12:00:15","date_gmt":"2012-06-20T12:00:15","guid":{"rendered":"http:\/\/cloudnewsdaily.com\/?p=10689"},"modified":"2012-06-20T12:00:15","modified_gmt":"2012-06-20T12:00:15","slug":"taking-in-memory-nosql-to-the-next-level","status":"publish","type":"post","link":"https:\/\/icloud.pe\/blog\/taking-in-memory-nosql-to-the-next-level\/","title":{"rendered":"Taking In-Memory NoSQL to the Next Level"},"content":{"rendered":"<\/p>\n<p><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Guest Post by Ofer Bengal, Co-Founder &amp; CEO, <a href=\"http:\/\/www.garantiadata.com\/\">Garantia Data<\/a> <\/strong><\/span><\/span><\/p>\n<p>&nbsp;<\/p>\n<div id=\"attachment_10690\" class=\"wp-caption alignleft\" style=\"width: 150px\">\n\t<img loading=\"lazy\" decoding=\"async\" class=\"size-thumbnail wp-image-10690\" title=\"Ofer Bengal Photo\" src=\"http:\/\/d11be2feca2wef.cloudfront.net\/wp-content\/uploads\/2012\/06\/Ofer-Bengal-Photo-150x150.jpg\" alt=\"Ofer Bengal\" width=\"150\" height=\"150\" \/><\/p>\n<p class=\"wp-caption-text\">Ofer Bengal has founded and led companies in data communications, telecommunications, Internet, homeland security and medical devices.<\/p>\n<\/div>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Today Garantia Data is launching the first in-memory NoSQL cloud that promises to change the way people use <\/span><\/span><\/span><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Memcached<\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> and Redis. I think this is a great opportunity to examine the state of these RAM-based data stores and to suggest a new, highly-efficient way of operating them in the cloud.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Challenges of Cloud Computing<\/strong><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Memcached and Redis are being increasingly adopted by today\u2019s web-applications, and are being used to scale-out their data-tier and significantly improve application performance (in many cases improvement is x10 over standard RDBMS implementation).\u00a0However, cloud computing has created new challenges in the way scaling and application availability should be handled and using Memcached and Redis in their simple form may not be enough to cope with these challenges. <strong><\/strong><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Memcached<\/strong><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">It\u2019s no secret Memcached does wonders for websites that need to quickly serve up dynamic content to a rapidly growing number of users. Facebook, Twitter, Amazon and YouTube, are heavily relying on Memcached to help them scale out; Facebook handles millions of queries per second with Memcached.<br \/>\nBut Memcached is not just for giants. Any website concerned with response time and user based growth should consider Memcached for boosting its database performance. That\u2019s why over 70% of all web companies, the majority of which are hosted on public and private clouds, currently use Memcached.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Local Memcached<\/strong><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">\u00a0is the simplest and fastest caching method because you cache the data in the same memory as the application code. Need to render a drop-down list faster? Read the list from the database once, and cache it in a Memcached HashMap. Need to avoid the performance-sapping disk trashing of an SQL call to repeatedly render a user\u2019s personalized Web page? Cache the user profile and the rendered page fragments in the user session.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Although local caching is fine for web applications that run on one or two application servers, it simply isn\u2019t good enough when the data is too big to fit in the application server memory space, or when the cached data is updated and shared by users across multiple application servers and user requests. In such cases user sessions, are not bound to a particular application server. Using local caching under these conditions may end up providing a low hit-ratio and poor application performance.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Distributed Memcached<\/strong><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">\u00a0tends to improve local caching by enabling multiple application servers to share the same cache cluster. Although the Memcached client and server codes are rather simple to deploy and use, Distributed Memcached suffers from several inherent deficiencies:<\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Lack of high-availability<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 When a Memcached server goes down the application\u2019s performance suffers as all data queries are now addressed to the RDBMS, which is providing a much slower response time. When the problem is fixed, it could take between a few hours to several days until the recovered server becomes \u201chot\u201d with updated objects and fully effective again. In more severe case, where session data is stored in Memcached without persistent storage, losing a Memcached server may cause forced logout of users or flush of their shopping carts (in ecommerce sites). <\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Failure hassle<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 The operator needs to set all clients for the replacement server and wait for it to \u201cwarm-up\u201d. Operators sometimes add temporary slave servers to their RDBMS, for offloading their Master server, until their Memcached recovers.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Scaling hassle<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 When the application dataset grows beyond the current Memcached resource capacity, the operator needs to scale out by adding more servers to the Memcached tier. However, it is not always clear when exactly this point has been reached and many operators scale out in a rush only after noticing degradation in their application\u2019s performance.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Scaling impact on performance<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> &#8211; Scaling out (or in) Memcached typically causes partial or entire loss of the cached dataset, resulting, again, in degradation of the application\u2019s performance.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Manpower<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> &#8211; Operating Memcached efficiently requires manpower to monitor, optimize and scale when required. In many web companies these tasks are carried out by expensive developers or devops.<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Amazon has tried to simplify the use of Memcached by offering ElastiCache, a cloud-based value-added service, where the user does not have to install Memcached servers but rather rent VMs (instances) pre-loaded with Memcached (at a cost higher than plain instances). However, ElastiCache has not offered a solution for any of the Memcached deficiencies mentioned above. Furthermore, ElastiCache scales-out by adding a complete EC2 instance to the user\u2019s cluster, which is a waste of $$ for users who only require one or two more GBs of Memcached. With this model ElastiCache misses on delivery of the true promise of cloud computing &#8211; \u201cconsume and pay only for what you really need\u201d (same as for electricity, water and gas).<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>Redis<\/strong><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Redis an open source, key-value, in-memory, NoSQL database began ramping-up in 2009 and is now used by Instagram, Pinterest, Digg, Github, flickr, Craigslist and many others and has an active open source community, sponsored by VMware.<\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Redis can be used as an enhanced caching system alongside RDBMS, or as a standalone database.<br \/>\nRedis provides a complete new set of data-types built specifically for serving modern web applications in an ultra-fast and more efficient way. It solves some of the Memcached deficiencies, especially when it comes to high availability, by providing replication capabilities and persistent storage. However, it still suffers from the following drawbacks:<\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Failure hassle<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 There is no auto-fail-over mechanism; when a server goes down, the operator still needs to activate a replica or build a replica from persistent storage.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Scalability<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Redis is still limited to a single master server and although cluster management capability is being developed, it probably won\u2019t be simple to implement and manage and will not support all Redis commands, making it incompatible with existing deployments.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Operations<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Building a robust Redis system requires strong domain expertise in Redis replication and data persistence nuances and building a Redis cluster will be rather complex.<\/span><\/span><\/span><\/li>\n<\/ul>\n<div id=\"attachment_10692\" class=\"wp-caption alignleft\" style=\"width: 582px\">\n\t<img loading=\"lazy\" decoding=\"async\" class=\" wp-image-10692 \" title=\"Fig 1 - DB Caching Evolution\" src=\"http:\/\/d11be2feca2wef.cloudfront.net\/wp-content\/uploads\/2012\/06\/Fig-1-DB-Caching-Evolution.jpg\" alt=\"DB Caching Evolution\" width=\"582\" height=\"245\" \/><\/p>\n<p class=\"wp-caption-text\">The Evolution of Caching<\/p>\n<\/div>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><strong>A new cloud service that will change the way people use Memcached and Redis<\/strong><\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">Imagine connecting to an infinite pool of RAM memory and drawing as much Memcached or Redis memory you need at any given time, without ever worrying about scalability, high-availability, performance, data security and operational issues; and all this, with the click of a button (ok, a few buttons).\u00a0Now imagine paying only for GBs used rather than for full VMs and at a rate similar to what you pay your cloud vendor for plain instances. <\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><em><strong>Welcome to the Garantia Data In-Memory NoSQL Cloud!<\/strong><\/em><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">\u00a0\u00a0 <\/span><\/span><\/span><\/p>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">By In-Memory NoSQL Cloud I refer to an online, cloud-based, in-memory NoSQL data-store service that offloads the burden of operating, monitoring, handling failures and scaling Memcached or Redis from the application operator\u2019s shoulders.\u00a0Here are my top 6 favorite features of such service, now offered by Garantia Data:<\/span><\/span><\/span><\/p>\n<ul>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Simplicity<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Operators will no longer need to configure and maintain nodes and clusters. The standard Memcached\/Redis clients are set for the service DNS and from this moment on, all operational issues are automatically taken care of by the service.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Infinite scalability<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 The service provides an infinite pool of memory with true auto-scaling (out or in) to the precise size of the user\u2019s dataset. Operators don\u2019t need to monitor eviction rates or performance degradation in order to trigger scale-out; the system constantly monitors those and adjusts the user\u2019s memory size to meet performance thresholds.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">High availability<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Built-in automatic failover makes sure data is guaranteed under all circumstances. Local persistence storage of the user\u2019s entire dataset is provided by default, whereas in-memory replication can be configured at a mouse click. In addition, there is no data loss whatsoever when scaling out or in.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Improved application performance<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Response time is optimized through consistent monitoring and scaling of the user\u2019s memory. Several techniques that efficiently evict unused and expired objects are employed to significantly improve the hit-ratio.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Data security<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 For those operators who are concerned with hosting their dataset in a shared service environment, Garantia Data has full encryption of the entire dataset as a key element of its service.<\/span><\/span><\/span><\/li>\n<li><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"><span style=\"text-decoration: underline;\">Cost savings<\/span><\/span><\/span><\/span><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\"> \u2013 Garantia Data frees developers from handling data integrity, scaling, high availability and Memcached\/Redis version compliance issues. Additional savings are achieved by paying only for GBs consumed rather than for complete VMs (instances). The service follows the true spirit of cloud computing enabling memory consumption to be paid for much like electricity, water or gas, so you \u201conly pay for what you really consume\u201d.<\/span><\/span><\/span><\/li>\n<\/ul>\n<p><span style=\"color: #393939;\"><span style=\"font-family: Times New Roman,serif;\"><span style=\"font-size: small;\">We have recently concluded a closed beta trial with 20 participating companies where all these features were extensively tested and verified &#8211; and it worked fine! So this is not a concept anymore, it\u2019s real and it\u2019s going to change the way people use Memcached and Redis! Am I excited today? Absolutely!<\/span><\/span><\/span><\/p>\n<div class=\"zemanta-pixie\" style=\"margin-top: 10px; height: 15px;\"><img decoding=\"async\" class=\"zemanta-pixie-img\" style=\"float: right;\" src=\"http:\/\/img.zemanta.com\/pixy.gif?x-id=e1ecbbfd-09c5-4e43-9dcb-e0acc253fd0f\" alt=\"\" \/><\/div>\n<p><a href=\"http:\/\/feedads.g.doubleclick.net\/~a\/VgCfEP7o1lsB_qUBjcI5uX4F3ug\/0\/da\"><img decoding=\"async\" src=\"http:\/\/feedads.g.doubleclick.net\/~a\/VgCfEP7o1lsB_qUBjcI5uX4F3ug\/0\/di\" border=\"0\" ismap=\"true\"><\/img><\/a><br \/>\n<a href=\"http:\/\/feedads.g.doubleclick.net\/~a\/VgCfEP7o1lsB_qUBjcI5uX4F3ug\/1\/da\"><img decoding=\"async\" src=\"http:\/\/feedads.g.doubleclick.net\/~a\/VgCfEP7o1lsB_qUBjcI5uX4F3ug\/1\/di\" border=\"0\" ismap=\"true\"><\/img><\/a><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/feeds.feedburner.com\/~r\/CloudNewsDaily\/~4\/etZwEnujm0w\" height=\"1\" width=\"1\"\/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Guest Post by Ofer Bengal, Co-Founder &#038; CEO, Garantia Data &#160; Today Garantia Data is launching the first in-memory NoSQL cloud that promises to change the way people use Memcached and Redis. I think this is a great opportunity to examine the state of these RAM-based data stores and to suggest a new, highly-efficient way [&#8230;]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[78,536,168,537,538],"tags":[],"class_list":["post-2223","post","type-post","status-publish","format-standard","hentry","category-guest-post","category-memcached","category-nosql","category-redis","category-relational-database-management-system"],"_links":{"self":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts\/2223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/comments?post=2223"}],"version-history":[{"count":0,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/posts\/2223\/revisions"}],"wp:attachment":[{"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/media?parent=2223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/categories?post=2223"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/icloud.pe\/blog\/wp-json\/wp\/v2\/tags?post=2223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}