Category Archives: Amazon

Amazon, Amazon AWS, Amazon Elastic MapReduce, Amazon Web Services, AWS Data Pipeline, Big Data, Business Intelligence, Business Processes, Workflow

New AWS Pipeline Tool Aims to Make Effective Use of Your Business Data

AWS Pipeline Diagram

Amazon’s new AWS Data Pipeline product “will help you move, sort, filter, reformat, analyze, and report on  data in order to make use of it in a scalable fashion. ” You can now automate the movement and processing of any amount of data using data-driven workflows and built-in dependency checking.

A Pipeline is composed of a set of data sources, preconditions, destinations, processing steps, and an operational schedule, all definied in a Pipeline Definition.

The definition specifies where the data comes from, what to do with it, and where to store it. You can create a Pipeline Definition in the AWS Management Console or externally, in text form.

Read more.


Amazon, Amazon AWS, Amazon S3, Amazon Web Services, Reduced Redundancy Storage, Storage

Yet Another Round of AWS Storage Price Cuts

Timed for announcement at AWS re:Invent, and nicely juxtaposed against this week’s similar storage price cuts by Google, Amazon has trimmed S3 and EBS prices.

They’ve reduced the price of Amazon S3 storage by 24-28% in the US Standard Region, and made similar price reductions in all nine regions worldwide  as well as reducing the price of Reduced Redundancy Storage (RRS). Here are the new prices for Standard Storage in the US Standard Region:

Tier Old Price
(GB / month)		 New Price Change
First 1 TB / month $0.125 $0.095 24%
Next 49 TB $0.110 $0.080 27%
Next 450 TB $0.095 $0.070 26%
Next 500 TB $0.090 $0.065 28%
Next 4000 TB $0.080 $0.060 25%
Over 5000 TB $0.055 $0.055 No change

The new prices are listed on the Amazon S3 pricing announcement page. The new prices take effect on December 1, 2012 and will be applied automatically.

Amazon also reduced the per-gigabyte storage cost for EBS snapshots, again world-wide. Here are the new prices:

Region Old Price
(GB / month)		 New Price Change
US East (N. Virginia) $0.125 $0.095 24%
US West (Oregon) $0.125 $0.095 24%
US West (Northern California) $0.140 $0.105 25%
EU (Ireland) $0.125 $0.095 24%
Asia Pacific (Singapore) $0.125 $0.095 24%
Asia Pacific (Tokyo) $0.130 $0.100 23%
Asia Pacific (Sydney) $0.140 $0.105 25%
South America (Sao Paulo) $0.170 $0.130 24%


Amazon, Candia, Midokura, Networking, Nicira, Start Up, Virtualization, VMware, Wired

NoSQL Pioneer Pino de Candia Taking Aim at Virtual Networking

Wired has a good article today on some of the geniuses and startups in the Virtual Networking space, and how they might “remake the internet.”

Together with Dan Dumitriu — another Amazon vet steeped in the science of massive computing systems — de Candia is one of the key engineers behind a company called Midokura. Much like the oft-discussed Silicon Valley startup Nicira, Midokura deals in virtual networks — computer networks that exist only as software.

Over the past decade, VMware, Microsoft, and others have helped move the world’s computing applications onto virtual servers — machines that exist only as software — and now, a new of wave of companies is fashioning software for building complex virtual networks that tie all those virtual servers together. That’s a hard concept to grasp, but basically, these companies are moving the brains of the network out of hardware and into software.

Read the article.


Amazon, Amazon AWS, Amazon.com, New York Times

Amazon “just gets cheaper, and cheaper, and cheaper”

Good article in the New York Times on the effect of AWS on startup access to major computing resources. Great takeaway quote:

“I have 10 engineers, but without A.W.S. I guarantee I’d need 60,” said Daniel Gross, Cue’s 20-year-old co-founder. “It just gets cheaper, and cheaper, and cheaper.” He figures Cue spends something under $100,000 a month with Amazon but would spend “probably $2 million to do it ourselves, without the speed and flexibility.”

He conceded that “I don’t even know what the ballpark number for a server is — for me, it would be like knowing what the price of a sword is.”

Read the full article.


Amazon, Amazon AWS, Amazon EBS, Amazon Elastic Block Store, Amazon Elastic Compute Cloud, Amazon Web Services, AWS, High Performance Computing, IOPS, Storage

Amazon Web Services Launches High Performance Storage Option for Amazon Elastic Block Store

Image representing Amazon as depicted in Crunc...

Amazon Web Services today announced new features for customers looking to run high performance databases in the cloud with the launch of Amazon Elastic Block Store (Amazon EBS) Provisioned IOPS. Provisioned IOPS (input/output operations per second) are a new EBS volume type designed to deliver predictable, high performance for I/O intensive workloads, such as database applications, that rely on consistent and fast response times. With Provisioned IOPS, customers can flexibly specify both volume size and volume performance, and Amazon EBS will consistently deliver the desired performance over the lifetime of the volume. To get started with Amazon EBS, visit http://aws.amazon.com/ebs.

Provisioned IOPS volumes are engineered to allow customers to develop, test, and deploy production applications and be confident that they will receive their desired performance. With a few clicks in the AWS Management Console, customers can create an EBS volume provisioned with the storage and IOPS they need and attach it to their Amazon EC2 instance. Amazon EBS currently supports up to 1,000 IOPS per Provisioned IOPS volume, with plans to deliver higher limits soon. Customers can attach multiple Amazon EBS volumes to an Amazon EC2 instance and stripe across them to deliver thousands of IOPS to their application.

To enable Amazon EC2 instances to fully utilize the IOPS provisioned on an EBS volume, Amazon EC2 is introducing the ability to launch selected Amazon EC2 instance types as EBS-Optimized instances. EBS-Optimized instances deliver dedicated throughput between Amazon EC2 and Amazon EBS, with options between 500 Megabits per second and 1,000 Megabits per second depending on the instance type used. The combination of EBS Provisioned IOPS and EBS-Optimized instances allows customers to run their most performance-sensitive applications on Amazon EC2, giving them predictable scaling with the same ease of use, durability, and flexibility of provisioning benefits they expect from Amazon EC2 and Amazon EBS.

“AWS introduced Amazon EBS in 2008 to provide a highly scalable virtual storage service and now, four years later, our customers are running applications on Amazon EC2 using EBS volumes at tremendous scale,” said Peter De Santis, Vice President of Amazon EC2. “Customers have been asking for the ability to set their performance rate to achieve consistently high performance. With EBS Provisioned IOPS volumes, EBS-Optimized instances and the recently launched High I/O SSD-based EC2 instances, customers have a range of choices for running their most demanding applications and databases on AWS while achieving peak performance in a predictable manner.”

At NASA’s Jet Propulsion Laboratory, Amazon EBS is used to support various missions and research programs. Consistent performance of I/O is a major requirement for numerous use cases across NASA ranging from scientific computing to large scale database deployments. JPL now routinely provisions cloud compute capacity in an elastic manner but database latencies have proven difficult. To help meet this challenge, JPL’s missions and its Office of the CIO prototyped the new EBS Provisioned IOPS capability to provision flexible compute capacity and overcome database latency restrictions. The results were highly successful and the release of EBS Provisioned IOPS, coupled with Amazon EC2 High I/O SSD-based instances, will introduce a whole new realm of I/O intensive scientific applications for JPL from radar data processing to the quest of black holes.

Stratalux is a leader in building and managing tailored cloud solutions for customers of all sizes. “A common request we see from both our large and small customers is the need to support high performance database applications. Throughput consistency is critical for these workloads,” said Jeremy Przygode, CEO at Stratalux. “Based on positive results in our early testing, the combination of EBS Provisioned IOPS and EBS-Optimized instances will enable our customers to consistently scale their database applications to thousands of IOPS, enabling us to increase the number of I/O intensive workloads we support.”

Amazon EBS Provisioned IOPS volumes are currently available in the US-East (N. Virginia), US-West (N. California), US-West (Oregon), EU-West (Ireland), Asia Pacific (Singapore), and Asia Pacific (Japan) regions with additional Region launches coming soon.


Amazon, Amazon AWS, Amazon Web Service, Amazon.com, AWS, Data center, Emergency power system, Outage Alert

AWS Outage Postmortum: “the generators did not pick up the load”

Amazon has provided their take on how the big derecho storm that hit the Eastern US (and still leaves millions without power during a heat wave) brought down one of their data centers. Basically it was “hardware failure” — in this case a couple of emergency generators.

In the single datacenter that did not successfully transfer to the generator backup, all servers continued to operate normally on Uninterruptable Power Supply (“UPS”) power. As onsite personnel worked to stabilize the primary and backup power generators, the UPS systems were depleting and servers began losing power at 8:04pm PDT.

Read the AWS statement for more detail.