The Ambiguously Vague Duo: Scale-Out and Scale-Up

So, a number of folks wanted more meaty content than was being offered in the CIO-targeted MySQL marketing campaign recently, "The Twelve Days of MySQL Scale-Out". I wanted to write a blog entry which addressed this carnivorous appetite of the MySQL community by going into a discussion on what precisely this term "scale-out" means.

Comparisons of Scaling Out versus Scaling Up

What is scaling anyway? Simply put, it's the ability of an application to address growth in throughput, usage, and capacity. Both scale out and scale up strategies address the ability of a system to address this growth. I think there is a tendency, as Jeremy recently pointed out, to simplify the relationship between each strategy and to put the scale out architecture on some sort of pedestal without really understanding the challenges involved in its implementation.

Sometimes, I think folks hear "scale out" and equate the strategy with a clustered computing approach where hundreds of low-end processors and large banks of memory and disks essentially act as a single computer. Clustered computing is not what scaling is about — neither scaling out nor scaling up.

In general, the term "scaling up" refers to the strategy of adding capacity by increasing the capacity of the underlying hardware — buying a larger box with more processors or more memory to run your application. "Scaling out" approaches, on the other hand, can generally be thought of as adding capacity by adding additional servers into the application architecture. In simple terms, these are the main three things that I feel differentiate the scale up vs scale out approaches:

1. Reliance on hardware versus reliance on software
2. Equation with "first-class" hardware versus "commodity" hardware
3. Massive versus incremental capacity increases
4. Centralized versus partitioned application architectures

Reliance on hardware versus reliance on software

For shops employing a scale-up approach, when throughput for database-centric applications gets maxed out on the existing hardware, the solution is to increase the capacity of the database server so that it may handle more query and transaction requests with no change to the application code. The bolded point is important: by increasing the capacity of the hardware running the database server, the application code does not need to be changed at all; this is a benefit, of course, to the application development team, as it means less work for them!

However, there are a number of issues to this approach to be aware of, which I highlight in the sections on adding incremental capacity and increased application complexity.

Equation with "first-class" hardware versus "commodity" hardware

As I described in the section above, scaling up typically means adding capacity by increasing the capacity of the underlying hardware. There is an additional point to make here; that the hardware itself is typically quite different between scale up and scale out models. Scale up models tend to rely on brand name, "enterprise class" hardware, where scale out approaches tend to rely on "commodity hardware". I'd like to bring up two quotes regarding this from Jeremy Cole and from Raj Thukral at Pythian which I believe shed some light on this difference between scale up and scale out.

First, Jeremy wanted to dispel the myth that commodity hardware meant "super-cheap" (highlight added by me):

You often hear the term “commodity hardware” in reference to scale out. While crappy hardware is also commodity, what this means is that instead of getting stuck on the low-end $40k machine, with thoughts of upgrading to the $250k machine, and maybe later the $1M machine, you use data partitioning and any number of let’s say $5k machines. That doesn’t mean a $1k single-disk crappy machine as said above. What does it mean for the machine to be “commodity”? It means that the components are standardized, common, and the price is set by the market, not by a single corporation. Use commodity machines configured with a good balance of price vs. performance.

I could not agree more with Jeremy's assessment here. Components that are standardized, common, and not controlled by a single corporation have become inextricably linked to the scale out model, in a similar way to how "enterprise-class" hardware has been linked to the scale up model. Raj offered an explanation to this in a recent email to me:

...most people running Oracle tend to do so on brand name high horsepower boxes. I guess if you're paying in the 6 figures for a license, you can afford a good box. With MySQL, it generally tends to be lower end white-box class hardware.

Perhaps Raj is spot on. Maybe the reason scale up models are tied to the higher end machines has simply to do with matching the cost of the software to the cost of the hardware? After all, it is human nature to think that the more expensive something is, the more it should be surrounded by other expensive things...

I think there is another reason for this; that Oracle simply uses better hardware more efficiently than MySQL. More on that later...

Adding incremental capacity?

In a scale up model, it is highly unlikely that capacity is added to the application in an incremental manner. For example, let's assume I have an application which runs on Oracle 10g Enterprise Edition on a decent size Sun box with 16GB of RAM and, say, 4 processors. I have hit a point where application performance is suffering because Oracle is using the hardware as much as it can, but is out of memory. I decide that I need to increase performance, and instead of making any changes in application code, I decide to increase the capacity of the underlying hardware by requisitioning a new Sun server with 8 processors and 32GB of RAM.

OK, so I have solved my performance problem, as now Oracle is able to address more memory and use more processors for resolving query requests. However, it is unlikely that my new server is being maxed out and that I am getting good value for the amount of money I spent for new hardware capacity. Let's assume it took me two years to max out the capacity of my original 4 processor/16 GB RAM Sun server. It will likely take a year or more to max out the capacity of my new hardware. While this is all fine and dandy that I don't need to worry about performance of the database server for a while, it is essentially a waste of hardware power while my throughput needs build up over the next year. The figure below illustrates the point here: that hardware capacity and processing power is wasted during the time period while we "wait" for the new hardware to max out — if it ever does... The purple area represents the total wasted processing power of the hardware over time.

In a scale out model, hardware capacity is not added in such a dramatic fashion. Servers which individually provide less capacity than the scale-up servers described above, are added to the application incrementally over time, adding capacity in a staggered, more consistent fashion. If I were deploying a scale out strategy to handle the same growth in application load, I would be adding additional servers incrementally to provide the capacity needed. In the graphic below, I'm adding a server each quarter which increases application capacity by 13K (requests, transaction, whatever). As you can see, the purple area which represents the wasted processing power of the hardware is dramatically reduced.

The benefit of this reduced waste is fairly obvious. Instead of a one-time large expenditure for the beefier Sun box, I spread that cost out over time. Investors and executives are happy when costs are controlled and incremental, as the reduced immediate costs offset risk of a slowdown in application growth, which may eliminate the need for more processing power.

Increased complexity of the application?

But, this incremental cost and capacity is not without other costs — namely an increased complexity in the application architecture to handle the segmentation of application requests to the various servers in our architecture topology. This scale-out architecture is more difficult to handle, both conceptually and in implementation. Knowledge of scale-out application architectures is necessary, and the professionals which have this knowledge aren't cheap.

MySQL's ability to scale up does not compare to Oracle's

Here is a chicken and egg question for you. What came first: the scale out architecture which MySQL promotes or the design of MySQL in scaling out? Funny question? Not really. As much as I love MySQL, I don't think that the ability of MySQL to fit well with a scale out architecture preceded the concept of MySQL's preferred scale out architecture.

I actually think that what we now consider the "scale out model" — the model which the Twelve Days of Scale Out campaign has been promoting — arose due to the inability of MySQL to scale up in a way that Oracle can.

Shocked that I would say such blasphemy? Don't be. It's just an observation that I think is accurate: the scale out model — a model which I truly believe is the scaling model of the future — is the only model which MySQL succeeds at. MySQL's architecture and internals are lacking certain features compared to Oracle's in a number of key areas that make a scale up model for MySQL a pointless exercise:

• Inefficient use of multiple processors
• Less able to prevent a bad query from ruining everyone else's day

Because of the above deficiencies, the benefit of throwing better hardware at MySQL is reduced compared to Oracle. However, in a scale out architecture, the above deficiencies are mitigated. In the case of inefficient use of multiple processors, that problem essentially goes away in a scale out model, as the servers being added into the "mix" typically have between one and four processors, and MySQL running locally on the server. From one to four processors, the inefficiency of MySQL in handling additional processors isn't as obvious. For the bad query ruining people's day, that problem is mitigated by the multiple database servers in the scale out architecture which serve only a subset of the application's requests. Essentially, the architecture of scale out isolates bad queries to a small portion of the application's users; something which would not happen in a scale-up, single MySQL database server model.

Centralized versus partitioned application architectures

The fourth major difference between scale up and scale out architectures has to do with the overall topology of the applications following each strategy. In scale up models, the applications tend to be more centralized than in a scale out model. By "centralized", I do not mean that the application code itself lives on a single server. I mean that the data generally resides in a single database server, and that one or more application servers connect to this single database server for the application data needs.

By "partitioned" I mean that the data itself tends to be partitioned into several "shards", with one or more applications directing requests to one or more of these database partitions. The partitioning of data across database servers can be done either with a homegrown solution or with the partitioning features in MySQL 5+.

In either case, the partitioning model is chosen so that the distribution of data across the application is done in a consistent and even way. Sometimes, customer account IDs are used to spread the data amongst various servers. Other times, a hashing function or date ranges are used. Regardless, the point is that the scale out architecture encourages this splitting of the data into a non-centralized, distributed topology.

Much of the additional application complexity I wrote about above stems from this data-level partitioning. Additional code is needed to act as the traffic cop directing a request to its needed data store. Plus, getting aggregate information tends to be harder, as processes must be built to pull data from each of the shards into a centralized data warehouse for analysis. I consider this drawback to be minor, as even in scale up models, data is often pulled off the central database server onto a separate warehousing system for offline analysis.

But, along with this increased complexity comes the benefits of the scale out model: incremental increases in capacity and the ability to distribute load across a farm of database servers. Also, in a scale out model, there tends not to be a single point of failure, since there is not a single, monolithic database server acting as the data store for the entire application.