Some work benefits from adding more workers (i.e., scaling out), and some work benefits from helping one worker work very quickly (i.e., scaling up). Consider the difference between painting a fence and painting a picture. On the fence job, 30 workers will get the job done more quickly than 1 worker. An abundance of painters, however, won't benefit the painting of a picture.
In the world of computers, when you scale up, you add computing capacity by increasing one computer's resources—for example, adding more memory or more processors. When you scale out, you increase computing capacity by adding computers to a load-sharing cluster. Stateful applications are analogous to painting a picture. With stateful applications, you can increase throughput capacity through scale-up. Availability increases through failover to additional nodes that Microsoft Cluster service provides. Stateless applications are analogous to painting a fence. Stateless applications achieve increased capacity and availability by adding nodes through Network Load Balancing (NLB) or Component Load Balancing (CLB).
A good example of a stateless application is a Web server that delivers static HTML pages. Each page request is relatively independent of other requests. Stateful applications benefit from more resources within one system rather than more systems within a cluster. An example of a stateful application is a relational database management system, which performs better if it has additional memory in which to hold tables, views, connections, and transactions.