There are 182 billion emails sent every day, generating a lot of data about how recipients and ISPs respond. Many marketers take a more-is-better approach to stats, preferring to have the ability to slice and dice their email lists based numerous arbitrary stats. However, fundamentally what really matters is whether or not sending an email to a particular recipient will generate value. Data Scientists can design high-level insights such as engagement prediction models and content clusters that allow marketers to cut through the noise and design their campaigns around strong, predictive signals, rather than arbitrary statistics.
SendGrid sends up to half a billion emails a day for customers such as Pinterest and GitHub. All this email adds up to more text than produced in the entire twitterverse. We track events like clicks, opens and deliveries to help improve deliverability for our customers – adding up to over 50 billion useful events every month. While SendGrid data covers only about 2% of all non-spam email activity in the world, it gives SendGrid a unique snapshot of email activity spanning from the senders to the recipients and inbox providers like gmail and yahoo. To cope with data of this scale SendGrid has designed and implemented custom data structures to store 10s of billions of items in memory on a single commodity machine.
Todas las entradas hechas por Latest News from Cloud Computing Journal
HyperStore: @CloudianStorage Announces ‘#Cloud Storage for Everyone’ | @CloudExpo
Cloudian announced immediate availability of Cloudian HyperStore appliances and Cloudian HyperStore 5.0 software. Flash-optimized, rack-ready HyperStore appliances make it easy to economically deploy full-featured, highly scalable S3-compliant storage with three enterprise-focused configurations. HyperStore appliances come fully integrated with Cloudian HyperStore software to assure unlimited scale, multi-data center storage, fully automated data tiering, and support for all S3 applications.
Safeguarding Data in the Cloud By @Vormetric | @CloudExpo
As enterprises look to take advantage of the cloud, they need to understand the importance of safeguarding their confidential and sensitive data in cloud environments. Enterprises must protect their data from (i) system administrators who don’t need to see the data in the clear and (ii) adversaries who become system administrators from stolen credentials. In short, enterprises must take control of their data: The best way to do this is by using advanced encryption, centralized key management and cutting edge access controls and policies.
Seagate Has Shipped Over 10 Million Storage HHDD’s
Seagate has shipped over 10 Million storage HHDD’s, is that a lot?
By Greg Schulz
Seagate has shipped over 10 Million storage HHDD’s, is that a lot?
Recently Seagate made an announcement that they have shipped over 10 million Hybrid Hard Disk Drives (HHDD) also known as Solid State Hybrid Drives (SSHD) over that past few years. Disclosure Seagate has been a StorageIO client.
I know where some of those desktop class HHDD’s including Momentus XTs ended up as I bought some of the 500GB and 750GB models via Amazon and have them in various systems. Likewise I have installed in VMware servers the newer generation of enterprise class SSHD’s which Seagate now refers to as Turbo models as companions to my older HHDD’s
What is a HHDD or SSHD?
The HHDD’s continue to evolve from initially accelerating reads to now being capable of speeding up write operations across different families (desktop/mobile, workstation and enterprise). What makes a HHDD or SSHD is that as their name implies, they are a hybrid combing a traditional spinning magnetic Hard Disk Drive (HDD) along with flash SSD storage. The flash persistent memory is in addition to the DRAM or non-persistent memory typically found on HDDs used as a cache buffer. These HHDDs or SSHDs are self-contained in that the flash are built-in to the actual drive as part of its internal electronics circuit board (controller). This means that the drives should be transparent to the operating systems or hypervisors on servers or storage controllers without need for special adapters, controller cards or drivers. In addition, there is no extra software needed to automated tiering or movement between the flash on the HHDD or SSHD and its internal HDD, its all self-contained managed by the drives firmware (e.g. software).
Some SSHD and HHDD industry perspectives
Jim Handy over at Objective Analysis has this interesting post discussing Hybrid Drives Not Catching On. The following is an excerpt from Jim’s post.
|
Why were our expectations higher? There were a few reasons: The hybrid drive can be viewed as an evolution of the DRAM cache already incorporated into nearly all HDDs today.
The net result is shipments of 10 million units since its 2010 introduction, for an average of 2.5 million per year, out of a total annual HDD shipments of around 500 million units, or one half of one percent. Continue reading more of Jim’s post here. |
In his post, Jim raises some good points including that HHDD’s and SSHD’s are still a fraction of the overall HDD’s shipped on an annual basis. However IMHO the annual growth rate has not been a flat average of 2.5 million, rather starting at a lower rate and then increasing year over year. For example Seagate issued a press release back in summer 2011 that they had shipped a million HHDD’s a year after their release. Also keep in mind that those HHDD’s were focused on desktop workstations and in particular, at Gamers among others.
The early HHDD’s such as the Momentus XTs that I was using starting in June 2010 only had read acceleration which was better than HDD’s, however did not help out on writes. Over the past couple of years there have been enhancements to the HHDD’s including the newer generation also known as SSHD’s or Turbo drives as Seagate now calls them. These newer drives include write acceleration as well as with models for mobile/laptop, workstation and enterprise class including higher-performance and high-capacity versions. Thus my estimates or analysis has the growth on an accelerating curve vs. linear growth rate (e.g. average of 2.5 million units per year).
| Units shipped per year | Running total units shipped | |
| 2010-2011 | 1.0 Million | 1.0 Million |
| 2011-2012 | 1.25 Million (est.) | 2.25 Million (est.) |
| 2012-2013 | 2.75 Million (est.) | 5.0 Million (est.) |
| 2013-2014 | 5.0 Million (est) | 10.0 Million |
StorageIO estimates on HHDD/SSHD units shipped based on Seagate announcements
However IMHO there is more to the story beyond numbers of HHDD/SSHD shipped or if they are accelerating in deployment or growing at an average rate. Some of those perspectives are in my comments over on Jim Handy’s site with an excerpt below.
|
In talking with IT professionals (e.g. what the vendors/industry calls users/customers) they are generally not aware that these devices exist, or if they are aware of them, they are only aware of what was available in the past (e.g. the consumer class read optimized versions). I do talk with some who are aware of the newer generation devices however their comments are usually tied to lack of system integrator (SI) or vendor/OEM support, or sole source. Also there was a focus on promoting the HHDD’s to âgamersâ or other power users as opposed to broader marketing efforts. Also most of these IT people are not aware of the newer generation of SSHD or what Seagate is now calling âTurboâ drives. When talking with VAR’s, there is a similar reaction which is discussion about lack of support for HHDD’s or SSHD’s from the SI/vendor OEMs, or single source supply concerns. Also a common reaction is lack of awareness around current generation of SSHD’s (e.g. those that do write optimization, as well as enterprise class versions). When talking with vendors/OEMs, there is a general lack of awareness of the newer enterprise class SSHD’s/HHDD’s that do write acceleration, sometimes there is concern of how this would disrupt their âhybridâ SSD + HDD or tiering marketing stories/strategies, as well as comments about single source suppliers. Have also heard comments to the effect of concerns about how long or committed are the drive manufactures going to be focused on SSHD/HHDD, or is this just a gap filler for now. Not surprisingly when I talk with industry pundits, influencers, amplifiers (e.g. analyst, media, consultants, blogalysts) there is a reflection of all the above which is lack of awareness of what is available (not to mention lack of experience) vs. repeating what has been heard or read about in the past. IMHO while there are some technology hurdles, the biggest issue and challenge is that of some basic marketing and business development to generate awareness with the industry (e.g. pundits), vendors/OEMs, VAR’s, and IT customers, that is of course assuming SSHD/HHDD are here to stay and not just a passing fad⦠|
What about SSHD and HHDD performance on reads and writes?
What about the performance of today’s HHDD’s and SSHD’s, particular those that can accelerate writes as well as reads?
Enterprise Turbo SSHD read and write performance (Exchange Email)
What about the performance of today’s HHDD’s and SSHD’s, particular those that can accelerate writes as well as reads?
Enterprise Turbo SSHD read and write performance (TPC-B database)
Enterprise Turbo SSHD read and write performance (TPC-E database)
Additional details and information about HHDD/SSHD or as Seagate now refers to them Turbo drives can be found in two StorageIO Industry Trends Perspective White Papers (located here and another here).
Where to learn more
Refer to the following links to learn more about HHDD and SSHD devices.
StorageIO Momentus Hybrid Hard Disk Drive (HHDD) Moments
Enterprise SSHD and Flash SSD
Part of an Enterprise Tiered Storage Strategy
Part II: How many IOPS can a HDD, HHDD or SSD do with VMware?
2011 Summer momentus hybrid hard disk drive (HHDD) moment
More Storage IO momentus HHDD and SSD moments part I
More Storage IO momentus HHDD and SSD moments part II
New Seagate Momentus XT Hybrid drive (SSD and HDD)
Another StorageIO Hybrid Momentus Moment
SSD past, present and future with Jim Handy
Part II: How many IOPS can a HDD, HHDD or SSD do with VMware?
Closing comments and perspectives
I continue to be bullish on hybrid storage solutions from cloud, to storage systems as well as hybrid-storage devices. However like many technology just because something makes sense or is interesting does not mean its a near-term or long-term winner. My main concern with SSHD and HHDD is if the manufactures such as Seagate and WD are serious about making them a standard feature in all drives, or simply as a near-term stop-gap solution.
What’s your take or experience with using HHDD and/or SSHDs?
Ok, nuff said (for now)
Cheers
Gs
Greg Schulz – Author Cloud and Virtual Data Storage Networking (CRC Press), The Green and Virtual Data Center (CRC Press) and Resilient Storage Networks (Elsevier)
twitter @storageio
All Comments, (C) and (TM) belong to their owners/posters, Other content (C) Copyright 2006-2014 StorageIO All Rights Reserved
BlueBox Named «Silver Sponsor» of @DevOpsSummit | @BlouBox [#DevOps]
BlueBox bridge the chasm between development and infrastructure. Hosting providers are taking standardization and automation too far. For many app developers it does nothing but spawn mayhem and more work. They have to figure out how their creations live on a pre-fab infrastructure solution full of constraints. Operations-as-a-Service is what BlueBox does. BlueBox utilizes development tools such as OpenStack, EMC Razor, Opscode’s Chef and BlueBox’s proprietary tools give the power to do the unorthodox things which most hosting providers shun.
Larry Ellison Rocks the Oracle Boat
Larry Ellison turned 70 and has decided to turn over the CEO reins at Oracle. Safra Catz and Mark Hurd, both in their 50s, will function as a “Ms. Inside and Mr. Outside” as co-CEOs, at least for awhile.
Serious reverberations will be felt within this highly competitive company and the highly competitive industry in which it makes its money.
Even while guiding his yacht to an America’s Cup title, Larry Ellison remained in firm control of the company he founded in 1977. He still has an ownership stake of about 20% of the company–1 billion or so shares of Oracle stock worth about $40 billion. Who can imagine that he’ll be a docile, passive Chairman?
Yes, he is returning as Chairman, with Jeff Henley, currently in that role, moving aside to be Vice-Chairman. Ellison reports he will also serve as Chief Technology Officer. So it’s clear he’s not fading from the scene. But he will not be able to micromanage the company by any measure.
What Does It Mean?
Think of all of the very strong executives over the years who rose quickly and highly in Oracle, only to be banished from the kingdom and/or to start their own big companies. Ray Lane, Marc Benioff, and Tom Siebel spring immediately to mind (with Siebel Systems eventually coming back to the mother ship). There are dozens, if not hundreds, more.
How will today’s generation of willful executives develop and fare now that the lion has retreated?
I’ve already read reports about how Ellison’s decision—which was announced after the markets closed, of course—comes at a “difficult” or “momentous” time for the company. This is the era of cloud computing, Oracle was slow to the dance (with Ellison infamously mocking it a few years ago), and is now furiously developing strategies to maintain its grip as one of the key companies in enterprise IT.
The reality is that things have always been difficult for Oracle. In its early days, it had to fight IBM and a group of seeming equals—Informix, Sybase, and Ingres—for early dominance in the then-new era of relational databases. By cleverly marketing itself as the one company that worked on all platforms and by dint of a hyper-aggressive sales effort, Oracle emerged as the clear leader.
Then its stock got plastered in the early 90s when the company showed its first loss. The stock fell to $5 and Ellison himself lost billions and billions of paper wealth.
I remember him saying a few years ago that he would encourage today’s generation of young and newly rich entrepreneurs to buy tangible things like land and buildings and traditional businesses. Paper wealth comes and goes, but tangible things tend to stay that way. He’s followed his own advice with several eye-popping real-estate purchases in recent years.
We all remember him saying a few years ago that he saw the future of Silicon Valley perhaps looking like Detroit, given that consolidation was going to extinguish most companies and the innovative spirit that drives them. Certainly Oracle has acquired numerous big companies over the past decade, PeopleSoft and Sun Microsystems among this numerous group.
Then there’s the current era of cloud. Oracle’s always been in a struggle, and today’s era is no different.
I’ve pinged a couple of Oracle execs for comment, but nothing has returned my way.
I have no idea what this move means for the future of Oracle, cloud, Silicon Valley, or the world, but I can imagine fistfights in the cafeteria as execs and divisions jockey for position.
Who knows, it might drive an era of incredible innovation at the company now that Larry’s not around to approve every comma in every speech and every line of code in every program.
Safra Catz is known as a hyper-competent operations person and Mark Hurd could sell sand in the Mojave. If they can continue to work splendidly together, as Ellison alleged today they have been doing, Oracle could not only be an enteprise gorilla but dare we say a cool one at that.
My question to Safra and Mark now that they’ve had a few minutes to adjust to the new job: what are you doing in the IoT? That’s your next struggle, and all the other big players area already staking some big claims in the biggest technology market ever.
Why Everyone Is Talking About the ‘Internet of Things’ (#IoT) | @ThingsExpo
Over the summer Gartner released its much anticipated annual Hype Cycle report and the big news is that Internet of Things has now replaced Big Data as the most hyped technology. Indeed, we’re hearing more and more about this fascinating new technological paradigm. Every other IT news items seems to be about IoT and its implications on the future of… Read the full post
CEO of @PubNub To Present at @ThingsExpo [#IoT]
As the Internet of Things gains momentum, the focus has been on securing billions of IoT devices and the servers that orchestrate their connectivity. However, the greatest security and authentication risks reside within the communications among devices and servers.
In his session at Internet of @ThingsExpo, Todd Greene, Founder & CEO of PubNub, will discuss the top 10 challenges in securing IoT communications that, unsolved, render it impossible to deliver a secure IoT rollout. Learn the requirements for a ubiquitous, secure, bi-directional communication protocol for IoT. Specific design patterns to deliver secure device updates, as well as comprehensive solutions for malware defense and security credential management will also be shared.
Devil’s in the Details for NAS Storage Says @iXsystems | @CloudExpo
The devil made him do it. Or maybe it’s that the devil is in the details for Matt Olander, Chief Science Officer of iXsystems and a long-time contributor to FreeBSD.
A red devil icon plays a prominent role in this NAS storage company’s branding and image, which is open souce “in everything we do,” according to the company.
The company now has decades of experience “producing rock-solid solutions on an open source foundation. With more than1,000 clients, we have a proven history of building and delivering the right solution every time – for open source and beyond,” the company says.
We thought we’d find out what Matt has to say about open source and the state of NAS storage today:
Cloud Computing Journal: What advantages does NAS storage bring as cloud computing continues to grow?
Matt Olander: Network Attached Storage (NAS) allows new or existing cloud-computing platforms to easily extend storage capacity as service offerings grow. Since FreeNAS & TrueNAS offer both block & file access, no matter what architecture a deployment is using, our NAS solution can easily integrate into the chosen environment.
CCJ: What advantages have you seen Open Source provide for your customers?
Matt: There are many advantages offered by leveraging Open Source. Using Open Source alleviates one of the main problems with current storage solutions, which is vendor lock-in. By deploying something that is based on an Open Source solution, a client is free to explore other options and can rest assured knowing that they have other options at all times during the lifecycle of their investment.
Using Open Source can reduce costs by eliminating or reducing the proprietary licensing fees of a commercial, closed-source solution.
An Open Source solution also typically means that there are more people looking at the source-code. This means that the Open Source alternative can be more secure as well as higher quality, since more people are using, trying, and reviewing the software.
CCJ: What sort of trends do you see as cloud, Big Data, and the IoT grow? For example, the growth of solid-state storage.
Matt: According to Science Daily last year, 90% of the world’s data has been generated over the last 2 years. This is not slowing down and in fact, as more people and devices access the Internet, the need for storing and analyzing that data is growing.
We are moving into a time where devices at home will communicate with devices at work and even devices on the consumers body, in the form of wearable technology. All of these devices will benefit from a centralized location to share and store data. Storage will get faster and cheaper while the need for that storage will continue to grow.
How to Run @AppDynamics in @Azure | @CloudExpo
I hear this question fairly regularly and would like to walk you through how to host the controller in your own Azure cloud. First off, the pros of having AppDynamics with Azure:
Have full control and ownership of the data collected by AppDynamics
Provide additional security to access the data (for example, lock it down to a corporate VPN only).