IBM REDP-4285-00 User Manual

Chapter 3. Analyzing performance bottlenecks 

Draft Document for Review May 4, 2007 11:35 am

For servers whose primary role is that of an application or database server, the CPU is a 
critical resource and can often be a source of performance bottlenecks. It is important to note 
that high CPU utilization does not always mean that a CPU is busy doing work; it may, in fact, 
be waiting on another subsystem. When performing proper analysis, it is very important that 
you look at the system as a whole and at all subsystems because there may be a cascade 
effect within the subsystems.

Determining bottlenecks with the CPU can be accomplished in several ways. As discussed in 
Chapter 2, “Monitoring and benchmark tools” on page 39, Linux has a variety of tools to help 
determine this; the question is: which tools to use? 

One such tool is uptime. By analyzing the output from uptime, we can get a rough idea of 
what has been happening in the system for the past 15 minutes. For a more detailed 
explanation of this tool, see 2.3.3, “uptime” on page 43.

18:03:16  up 1 day,  2:46,  6 users,  load average: 182.53, 92.02, 37.95

Using KDE System Guard and the CPU sensors lets you view the current CPU workload.

Using top, you can see both CPU utilization and what processes are the biggest contributors 
to the problem (Example 2-1 on page 41). If you have set up sar, you are collecting a lot of 
information, some of which is CPU utilization, over a period of time. Analyzing this information 
can be difficult, so use isag, which can use sar output to plot a graph. Otherwise, you may 
wish to parse the information through a script and use a spreadsheet to plot it to see any 
trends in CPU utilization. You can also use sar from the command line by issuing sar -u or 
sar -U processornumber

. To gain a broader perspective of the system and current utilization 

of more than just the CPU subsystem, a good tool is vmstat (2.3.2, “vmstat” on page 42).

SMP-based systems can present their own set of interesting problems that can be difficult to 
detect. In an SMP environment, there is the concept of 

, which implies that you 

bind a process to a CPU. 

The main reason this is useful is CPU cache optimization, which is achieved by keeping the 
same process on one CPU rather than moving between processors. When a process moves 
between CPUs, the cache of the new CPU must be flushed. Therefore, a process that moves 
between processors causes many cache flushes to occur, which means that an individual 
process will take longer to finish. This scenario is very hard to detect because, when 

Note: There is a common misconception that the CPU is the most important part of the 
server. This is not always the case, and servers are often overconfigured with CPU and 
underconfigured with disks, memory, and network subsystems. Only specific applications 
that are truly CPU-intensive can take advantage of today’s high-end processors.

Tip: Be careful not to add to CPU problems by running too many tools at one time. You 
may find that using a lot of different monitoring tools at one time may be contributing to the 
high CPU load.