According to Netcraft, who tracks these kinds of things, SSL usage has doubled from 2008 and 2011. That’s a good thing, as it indicates an upswing in adherence to security best practices that say «SSL Everywhere» just makes good sense.
The downside is overhead, which despite improvements in processing power and support for specific cryptographic processing in hardware still exists. How much overhead is more dependent on the size of data and the specific cryptographic algorithms chosen. SSL is one of those protocols that has different overhead and impacts on performance based on the size of the data. With data less than 32kb, overhead is primarily incurred during session negotiation. After 32kb, bulk encryption becomes the issue.
The problem is that a server is likely going to feel both, because it has to negotiate the session and the average response size for web applications today is well above the 32kb threshold, with most pages serving up 41kb in HTML alone – that’s not counting scripts, images, and other objects.