Category Archives: Amazon Web Services

AWS re:Invent 2017 – re:Cap from Chris

AWS re:Invent 2017By Chris Williams

This was my very first AWS re:Invent conference and there are so many things to talk about!

I’m going to briefly talk about the new services that have were announced, but I’m not going to do a deep dive on any one of them… the reason being that (a) there is going to be MUCH more coming out on the services in the future & anything I say here will probably be incorrect and (b) you can go straight to the source... so I’ll link where appropriate, and (c) there were so many new services that I’m only going to hit on the big ones that I have better than passing knowledge of anyway 🙂

  • Amazon Sumerian 
    • Create and run VR, AR, and 3D applications without requiring specialized programming or 3D expertise.
  • Alexa for Business
    • Start meetings, shared devices, use for conference calls, create private skills for your business… the sky’s the limit with this & I’m excited to see where it goes.
  • Amazon ECS for Kubernetes (EKS)
    • This has been a long time coming.  Most of the Docker instances on EC2 were managed by K8s anyway.
  • New EC2 instances:
    • M5
    • H1
    • T2 Unlimited
  • Lambda
  • Amazon Time Sync Service (NTP)
  • AWS Serverless Application Repository
    • … Just signed up for the preview – can someone do WordPress please!?  😉
  • AWS Server Migration Service for Hyper-V
  • Aurora
    • Multi-Master – scale write nodes across multiple AZs increasing scalability and availability
    • Serverless – A new serverless mode that will save time and money by automatically adjusting database capacity to match application needs
  • DynamoDB
    • On-Demand Backup
    • Global Tables – Fully managed, multi-region, multi-master.
  • AWS Cloud9
    • It is a browser-based IDE where you can write/run/debug.  As long as you have a web browser… you have access to your code to work and share.  I haven’t tried it yet but it sounds very cool.
  • IoT
  • Machine Learning (or as I call it the Skynet initiative…)
    • DeepLens – Skynet eyeball, but I still WANT ONE
    • Comprehend – “analyzes text and tells you what it finds, starting with the language, from Afrikans to Yoruba, with 98 more in between. It can identify different types of entities (people, places, brands, products, and so forth), key phrases, sentiment (positive, negative, mixed, or neutral), and extract key phrases, all from text in English or Spanish. Finally, Comprehend‘s topic modeling service extracts topics from large sets of documents for analysis or topic-based grouping.” … not super creepy at all…
    • Rekognition Video – Rekognition, but for video now
    • SageMaker
    • Translate – a neural machine translation service that uses advanced machine learning technologies to provide fast language translation of text-based content
    • Transcribe – add speech to text capability to your apps!

There were many more announcements.  If you want the whole comprehensive list please go here: https://aws.amazon.com/new/reinvent/

Alternatively, if you are on the road I recommend listening to the AWS Podcast & the AWS Tech Chat, both are excellent.

Something else to keep in mind:

I suggest you go to as many hackathons, chalk talks, hands-on labs, and workshops that you can. Basically, anything that isn’t being recorded that you can watch later that you want to do, go do that. I met some really excellent people at the 1 hackathon that I attended. Next year I will try to hit at least 2. In terms of getting the most value out of my time, Hackathons, the Expo hall (here’s my interview with Linux Academy!), and simply walking up and talking to people were my top 3.

My hackathon team thanks to Calvin’s 360 camera… which I am now ordering 🙂 

 

Team Nakatomi in the house #reinvent #awsjamsessions – Spherical Image – RICOH THETA

All in all, I had a great time. Sure the conference had some issues that needed tweaking, but I learned more, came away with more energy and excitement than I have from a conference in a long time. Now I need to rest up on the couch with #ITKitty, Kim, and a giant iced coffee 🙂

Thanks for checking out my AWS re:Invent re:Cap 2017!

Preparing for the AWS-SA Exam with GreenPages’ Chris Williams

GreenPages Enterprise Consultant and Cloud Architect, Chris Williams, is currently preparing for his AWS-SA professional certification. Check out his 9 article series on the AWS Solutions Architect exam. Start with the intro here: http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes/

Remaining eight articles here:

http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes/

http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes-domain-1/

http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes-domain-2/

http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes-domain-3/

http://mistwire.com/2016/10/aws-certified-solutions-architect-pro-study-notes-domain-4/

http://mistwire.com/2016/11/aws-certified-solutions-architect-pro-study-notes-domain-5/

http://mistwire.com/2016/11/aws-certified-solutions-architect-pro-study-notes-domain-6/

http://mistwire.com/2016/11/aws-certified-solutions-architect-pro-study-notes-domain-7/

http://mistwire.com/2016/11/aws-certified-solutions-architect-pro-study-notes-domain-8/

To learn more, email us here.

AWS launches new wind farm in green drive

Wind farmAmazon Web Services has contracted green energy specialist EDP Renewables to build and run the 100 megawatt (MW) Amazon Wind Farm US Central in Ohio.

The project is due to complete by May 2017 when it will begin producing enough to power run, 29,000 average US homes for a year, it claims. AWS says the latest addition to its green energy stable would generate 320,000 MWh of electricity a year from a new wind farm in the US.

Amazon also claims the energy generated will feed the electrical grid supplying both current and future AWS cloud data centres.

In November 2014 AWS committed to running its infrastructure entirely on renewable energy – in the long term – and claimed that 25% of the electricity running its cloud services was green. By the end of 2016 it aims to have pushed the proportion to 40%.

Earlier this year it announced a renewable project with the Amazon Wind Farm (Fowler Ridge) in Indiana could generate 500,000 MWh of wind power annually. In April it then began a pilot project using Tesla’s energy storage batteries to power data centres at times when wind power and solar are not available. In the same month AWS joined the American Council on Renewable Energy and the US Partnership for Renewable Energy Finance to work with government policy makers on developing more renewable energy options.

In June 2015 it said its new AWS Solar Farm in Virginia could generate 170,000 MWh of solar power annually and a month later it added another wind farm in North Carolina that could generate 670,000 MWh a year. In total, AWS claims to have the potential to create 1.6 million MWh of power.

“We continue to pursue projects that help to power AWS data centres and bring us closer to achieving total renewable energy,” said Jerry Hunter, VP of Infrastructure at AWS.

Rackspace ups AWS functionality and support, becomes authorised reseller

AWSManaged hosting provider Rackspace has announced a ramped up suite of products featuring enhanced support and functionality with Amazon Web Services.

The agreement with AWS, announced at re:Invent in Las Vegas this week, will see Rackspace launch managed service offerings including tools, expertise, application management and operational support for AWS Cloud. “Fanatical Support for AWS” is the core service offering featured as part of the agreement, with three beta offerings supplementing the managed service – Managed Security for AWS, Compliance Assistance for AWS and Managed Cloud for Adobe Experience Manager.

Through Fanatical Support, Rackspace tells its customers to “leave the heavy lifting to us” as it claims to take care of migration, architecture, security and operations for companies looking to adopt AWS for application hosting.

Rackspace has also become an authorised reseller at AWS, as well as a managed services partner, and has joined the AWS Partner Network. CEO Taylor Rhodes spoke about the announcement on the company’s blog page.

“Over the past year, more and more of them [customers] have told us that they love Rackspace expertise and Fanatical Support, and would like to get it for the workloads that they prefer to run on AWS,” he said. “We have spent the past several months working with those customers and with AWS, to create the best managed-service offering on that platform.”

Rhodes went on to say that AWS adds to Rackspace’s existing commitment to support a number of other platforms.

“We help businesses tap the power of the cloud without the pain and expense of managing it all themselves,” he said. “We have gone deep on support for platforms such as OpenStack, Microsoft and VMware. Our success in leading the managed cloud market for those technologies has been validated by industry experts such as Gartner, and most importantly, by our 300,000-plus business customers.”

Finally, Rhodes then highlighted how Fanatical Support has evolved with today’s announcements, and how it will benefit various customer segments.  He claims it will appeal to businesses that have desired AWS integration with existing Fanatical Support functionality; for rapidly growing businesses needing to outsource some IT functionality in order to reallocate technical resource to other areas; and for customers new to both AWS and Rackspace.

Meanwhile, AWS’s VP of worldwide partner ecosystem Terry Wise, welcomed Rackspace’s increased integration and functionality of AWS.

“We’re pleased to see Rackspace support AWS customers and achieve membership in the AWS Managed Service Program,” he said. “A growing number of businesses who want the benefit of the AWS Cloud will find value in working with AWS Managed Service Partners like Rackspace. We have been impressed with Rackspace’s commitment to include beta customers in their AWS managed services offerings, along with certifying a large number of their technical staff.”

Hortonworks buys SequenceIQ to speed up cloud deployment of Hadoop

CloudBreak

SequenceIQ will help boost Hortonworks’ position in the Hadoop ecosystem

Hortonworks has acquired SequenceIQ, a Hungary-based startup delivering infrastructure agnostic tools to improve Hadoop deployments. The company said the move will bolster its ability to offer speedy cloud deployments of Hadoop.

SequenceIQ’s flagship offering, Cloudbreak, is a Hadoop as a Service API for multi-tenant clusters that applies some of the capabilities of Blueprint (which lets you create a Hadoop cluster without having to use the Ambari Cluster Install Wizard) and Periscope (autoscaling for Hadoop YARN) to help speed up deployment of Hadoop on different cloud infrastructures.

The two companies have partnered extensively in the Hadoop community, and Hortonworks said the move will enhance its position among a growing number of Hadoop incumbents.

“This acquisition enriches our leadership position by providing technology that automates the launching of elastic Hadoop clusters with policy-based auto-scaling on the major cloud infrastructure platforms including Microsoft Azure, Amazon Web Services, Google Cloud Platform, and OpenStack, as well as platforms that support Docker containers. Put simply, we now provide our customers and partners with both the broadest set of deployment choices for Hadoop and quickest and easiest automation steps,” Tim Hall, vice president of product management at Hortonworks, explained.

“As Hortonworks continues to expand globally, the SequenceIQ team further expands our European presence and firmly establishes an engineering beachhead in Budapest. We are thrilled to have them join the Hortonworks team.”

Hall said the company also plans to contribute the Cloudbreak code back into the Apache Foundation sometime this year, though whether it will do so as part of an existing project or standalone one seems yet to be decided.

Hortonworks’ bread and butter is in supporting enterprise adoption of Hadoop and bringing the services component to the table, but it’s interesting to see the company commit to feeding the Cloudbreak code – which could, at least temporarily, give it a competitive edge – back into the ecosystem.

“This move is in line with our belief that the fastest path to innovation is through open source developed within an open community,” Hall explained.

The big data M&A space has seen more consolidation over the past few months, with Hitachi Data Systems acquiring big data and analytics specialist Pentaho and Infosys’ $200m acquisition of Panaya.

Cloud Computing Entering Hypergrowth Phase

Cloud services and cloud platforms are now an undeniable part of the IT landscape. Forrester research indicates the shift has begun from exploration of cloud as a potential option, to rationalization of cloud services within the overall IT portfolio.

Cloud platforms, most notably Amazon Web Services, were only collectively $4.7 billion last year but are maturing quickly thanks to stronger recent solutions from traditional IT partners IBM, HP and Microsoft. The growth in use, maturity, and financial viability of public cloud platforms are proving their longstanding value as legitimate deployment options for enterprise applications. While not a one-for-one replacement for on-premise, hosting, or colocation, cloud platforms fit well as ideal deployment options for elastic and transient workloads built in modern application architectures.

For applications and services built in an agile mode with modern architectures, discrete cloud services, such as database, storage, integration and other standalone cloud middleware components, will empower developers by freeing them from the management and maintenance of these components and reduce overall deployment footprint and cost. They are also managed and enhanced by vendors as often as daily delivering new capabilities that can help a company maintain pace with the changing desires of an empowered customer base

As the largest clouds continue to invest in efficiencies that can only be achieved at their massive scales, the gulf between the cost efficiencies that can be had from the cloud and what is possible on-premise or through other outsourcing and hosting options will widen dramatically.

How Forrester came to these conclusions.

A Big, Perhaps Watershed Week of Cloud Annoucements

  • Google harmonized its cloud computing business to a single entity, with a pricing model intended to hold customers by enticing them to build ever cheaper and more complex software. 
  • Cisco announced it would spend $1 billion on a “cloud of clouds” project. 
  • Microsoft’s new CEO made his first big public appearance, offering Office for the Apple iPad, partly as a way to sell more of its cloud-based Office 365 product.
  • Amazon Web Services announced the general release of its cloud-based desktop computing business, as well as a deal with to offer cloud-based enterprise software tools to industries like healthcare and manufacturing.

For more detail and opinions read this, and listen to this.

Developers Hit With Big, Unexpected AWS Bills, Thousands on GitHub Exposed

Amazon Web Services (AWS) is urging developers using the code sharing site GitHub to check their posts to ensure they haven’t inadvertently exposed their log-in credentials.

When opening an account, users are told to “store the keys in a secure location” and are warned that the key needs to remain “confidential in order to protect your account”. However, a search on GitHub reveals thousands of results where code containing AWS secret keys can be found in plain text, which means anyone can access those accounts.

From a security perspective it means they can basically go in and gain access to any of the files that are stored in the AWS account.

According to an AWS statement,  ”When we become aware of potentially exposed credentials, we proactively notify the affected customers and provide guidance on how to secure their access keys,”

There is more detail (and some cautionary tales involving big, and unexpected, AWS bills) here.

AWS Files Complaint Over CIA $600 Million Procurement Bid Complaint by IBM

Amazon Web Services has filed a complaint in the U.S. Court of Federal Claims related to Central Intelligence Agency action in following recommendations made by the Government Accountability Office following an action filed in February by IBM after the CIA awarded AWS a contract worth up to $600 million over four years to build a private cloud for the entire intelligence community.

FCW has the details.

OpenStack’s Third Birthday – a Recap with a Look into the Future

Guest Post By Nati Shalom, CTO and Founder of GigaSpaces

OpenStack was first announced three years ago at the OSCON conference in Portland. I remember the first time I heard about the announcement and how it immediately caught my attention. Ever since that day, I have become a strong advocate of the technology. Looking back, I thought that it would be interesting to analyze why.

Is it the fact that it’s an open source cloud? Well partially, but that couldn’t be the main reason. OpenStack was not the first open source cloud initiative; we had Eucalyptus, then later Cloud.com and other open source cloud initiatives before OpenStack emerged.

There were two main elements missing from these previous open source cloud initiatives: the companies behind the initiatives and the commitment to a true open movement. It was clear to me that a true open source cloud movement could not turn into an industry movement, and thus meet its true potential if it was led by startups. In addition, the fact that companies whose businesses run cloud services, such as Rackspace, brought its own experience in the field and a large scale consumer of such infrastructure such as NASA, gave OpenStack a much better starting point. Also, knowing some of the main individuals behind the initiatives and their commitment to the Open Cloud made me feel much more confident that the OpenStack project would have a much higher chance for success than its predecessors. Indeed, after three years, it is now clear that the game is essentially over and it is apparent who is going to win the open source cloud war. I’m happy to say that I also had my own little share in spreading the word by advocating the OpenStack movement in our own local community which also grew extremely quickly over the past two years.

OpenStack as an Open Movement

Paul Holland, an Executive Program Manager for Cloud at HP, gave an excellent talk during the last OpenStack Summit, comparing the founding of the OpenStack Foundation to the establishment of the United States. Paul drew interesting parallelization between the factors that brought a group of thirteen individual states to unite and become the empire of today, with that of OpenStack.

OpenStack1

Paul also drew an interesting comparison between the role of the common currency that fostered the open market and trade between the different states with its OpenStack equivalent: APIs, common language, processes, etc. Today, we take those things for granted, but the reality is that common currency isn’t yet trivial in many countries even today, yet we cannot imagine what our global economy would look like without the Dollar as a common currency or English as a common language, even if they have not been explicitly chosen as such by all countries.

OpenStack2

As individuals, we often tend to gloss over the details of the Foundation and its governing body, but it is those details that make OpenStack an industry movement that has brought many large companies, such as Red Hat, HP, IBM, Rackspace and many others (57 in total as of today), to collaborate and contribute to a common project as noted in this report. Also, the fact that the number of individual developers has been growing steadily year after year is another strong indication of the real movement that this project has created.

OpenStack3

Thinking Beyond Amazon AWS

OpenStack essentially started as the open source alternative to Amazon AWS. Many of the sub-projects often began as Amazon equivalents. Today, we are starting to see projects with a new level of innovation that do not have any AWS equivalent. The most notable one IMHO is the Neutron (network) and BareMetal projects. Both have huge potential to disrupt how we think about cloud infrastructure.

Only on OpenStack

We often tend to compare OpenStack with other clouds on a feature-to-feature basis.

The open source and community adoption nature of OpenStack enables us to do things that are unique to OpenStack and cannot be matched by other clouds. Here are a few examples:

  • Run the same infrastructure on private and public clouds.
  • Work with multiple cloud providers; have more than one OpenStack-compatible cloud provider with which to work.
  • Plug in different HW as cloud platforms for private clouds from different vendors, such as HP, IBM, Dell, Cisco, or use pre-packaged OpenStack distributions, such as the one from Ubuntu, Red Hat, Piston etc.
  • Choose your infrastructure of choice for storage, network etc, assuming that many of the devices come with OpenStack-supported plug-ins.

All this can be done only on OpenStack; not just because it is open source, but primarily because of the level of adoption of OpenStack that has made it the de-facto industry standard.

Re-think the Cloud Layers

When cloud first came into the world, it was common to look at the stack from a three-layer approach: IaaS, PaaS and SaaS.

Typically, when we designed each of the layers, we looked at the other layers as *black-boxes* and often had to create parallel stacks within each layer to manage security, metering, high availability etc.

The fact that OpenStack is an open source infrastructure allows us to break the wall between those layers and re-think where we draw the line. For example, when we design our PaaS on OpenStack, there is no reason why we wouldn’t reuse the same security, metering, messaging and provisioning that is used to manage our infrastructure. The result is a much thinner and potentially more efficient foundation across all the layers that is easier to maintain. The new Heat project and Ceilometer in OpenStack are already starting to take steps in this direction and are, therefore, becoming some of the most active projects in the upcoming Havana release of OpenStack.

Looking Into the Future

Personally, I think that the world with OpenStack is by far healthier and brighter for the entire industry, as opposed to a world in which we are dependent on one or two major cloud providers, regardless of how good of a job they may or may not do. There are still many challenges ahead in turning all this into a reality and we are still at the beginning. The good news, though, is that there is a lot of room for contribution and, as I’ve witnessed myself, everyone can help shape this new world that we are creating.

OpenStack Birthday Events

To mark OpenStack’s 3rd Birthday, there will be a variety of birthday celebrations taking place around the world. At the upcoming OSCON event in Portland from July 22-26, OpenStack will host their official birthday party on July 24th. There will also be a celebration in Israel on the 21st, marking the occasion in Tel Aviv.

For more information about the Foundation’s birthday celebrations, visit their website at www.openstack.org.

Nati-GigaSpaces

Nati Shalom is the CTO and founder of GigaSpaces and founder of the Israeli cloud.org consortium.