2011/02/04

Why (on Linux) am I seeing so much RAM usage?

This is a guest post by Dayid http://dayid.org

There are better options to see your memory usage; however it seems `free` is more attune to creating the confusion I’m attempting to quell here. That said, see the Redhat docs about /proc/meminfo

Other commands to use to see memory usage

$ vmstat -aS M #see the "inactive" column for a rough "free" idea.
The real answer
There’s no reason to clear what’s in RAM until you need more space to write to it.


The short answer analogy
Buffers and cache in RAM being cleared is silly. Imagine a professor, who rather than writing all the way across the chalkboard, finishes a sentence and immediately erases and starts writing in the upper left corner AGAIN and AGAIN and AGAIN.

OR imagine you like a song. You record it to the beginning of a cassette tape. When you want a new song, do you re-record over the first song or record after it?

AKA: The horrible House/Barn analogy
Many people new to Linux or computers in general have a poor understanding of how RAM works. On Linux systems, most users will look at `top` or use `free` to see the amount of memory installed and/or free. Below is an example:

dayid@emiline ~ $ free -m
                   total  used  free   shared  buffers   cached
Mem:                2024  1970    53        0       19     1669
-/+ buffers/cache:         281  1742
Swap:               1953     4  1948
At first glance, they may look at their machine with 2GB of RAM and wonder how they only have 53MB free! While this is true, the surprise, fear, or angst about this comes from a misunderstanding.

We could take a trip to a million places for this horrible analogy, but let’s pretend we’re on a country farm.
Rather than working with 2024MB of RAM and 1953MB of SWAP, we’ll say we’ve got 20 beds in the house, and 20 beds in the barn.
Rather than programs we’ll have people occupying the space.
For our purposes, ignore costs of cleaning the bedding, water, etc.
The house can hold active workers or non-active workers.

Due to its distance and the time to get to/from it, the barn can only hold non-active workers. When a worker is called from the barn they will have to pass through the house and stay in the house while they work.

10 laborers show up to a job. Since the house is closer to the food, showers, and work they’ll be doing, we let them stay in the house.
10 of our 20 beds are used by active workers.
Our farm in `free -m`:

                   total   used   free   shared   buffers   cached
House:                20     10     10        0         0        0
-/+ buffers/cache:           10     10
Barn:                 20      0     20
8 more people show up for another job. They also stay in the house since we have the space for them.
18 of our 20 beds are used by active workers.
Our farm in `free -m`:

                   total   used   free   shared   buffers   cached
House:                20     18      2        0         0        0
-/+ buffers/cache:           18      2
Barn:                 20      0     20
The first job is over, we no longer need to keep around the first 10 laborers; however, letting them stay doesn’t cost us anything, as if they weren’t there the beds would just be empty (i.e., go to waste).
18 of 20 beds are used. 8 by active workers, 10 by non-active workers.
Our farm in `free -m`:

                   total   used   free   shared   buffers   cached
House:                20     18      2        0         0       10
-/+ buffers/cache:            8     12
Barn:                 20      0     20
Let’s take a timeout and review the above output. Right now we have 20 rooms. 18 are being used so only 2 are free. However, since 10 workers aren’t being used, they are in cache – kept around because we have no reason to kick them out – so, we actually have an operating space of 12 workers we could hire. 2 to stay in the unused rooms, and 10 to replace those that are already here.
We have a new job on the farm, so we have 4 new people show up. We do not have enough beds for them. 2 of the 10 who are not active leave. We move in those 4 new people.
20 of 20 beds are used. 12 by active workers, 8 by non-active workers.
Our farm:

                   total   used   free   shared   buffers   cached
House:                20     20      0        0         0        8
-/+ buffers/cache:           12      8
Barn:                 20      0     20
Right now we have 20 rooms filled. 8 are filled by people who aren’t working though, so technically we have 8 beds we can use if we need to. Now let’s get crazy.
It’s production season and we have a lot to do around the farm. We setup another program and need to hire 14 new workers for it. We’ll have to kick out the 8 non-active workers and move in 8 of the new workers. However, because we run out of rooms in the house, our least important workers will have to stay in the barn. The barn is still good storing area, but it will take them longer to get to and from the job each time they are required to.
20 of 20 beds are used by active workers. 6 rooms in the barn are used.
Our farm:

                   total   used   free   shared   buffers   cached
House:                20     20      0        0         0        0
-/+ buffers/cache:           20      0
Barn:                 20      6     14
Now, things calm down again and only 4 workers are going to remain active. We’re not going to toss out the rest though as they’re not harming anything just taking up space (at least not until we need the space again)
Our farm:

                   total   used   free   shared   buffers   cached
House:                20     20      0        0         0       16
-/+ buffers/cache:            4     16
Barn:                 20      6     14
That’s right, our “free” stays 0, as we still have no space available. The important thing to look at here is how much do we have available if we clean out the buffers and cache – which are not necessary to keep, but we generally keep until it needs to be discard.