Intel Xeon 5600 (Westmere) family

In the series of posts related to server configuration I reviewed the memory configuration for optimal performance. I figured that a short introduction to the 5600 family would be useful to better understand the considerations that will follow in my next post.   

About the Xeon 5600
Part of Intel Tick-Tock cadence, The Intel Xeon 5500 (Nehalem) was shrink and replaced by Intel Xeon 5600 (Westmere) family, to be followed by Sandy Bride next year which will introduce a new microarchitecture.
The 5600 is based on the new 32nm process used in chip manufacturing and offers better performance at lower power consumption.

The 5500 and 5600 series introduced a new concept in the Xeon family, they integrate a DDR3 memory controller, that allows via 3 channels, a direct connection to a dedicated memory, up to 3 DIMMs per channel. It also includes a link controller that handles the communication with the neighbor processor in what Intel calls QuickPath interconnect (QPI). the QPI allows up to 6.4GT/s in each direction per link.

For virtualization it includes the Extended VT-x, VT-c and the VT-d technologies.
The storage interface include a 6 ports SATA2 with software raid5.

Intel introduced to these processors a new acceleration technology called, Turbo Boost Technology, which automatically allows the processor to run faster then the marked frequency in cretin conditions. The max turbo boost frequency is dependent on the number of active cores within a processor.

 Figure 1: Xeon 5600 DP architecture

The Xeon 5600 platform advantages  

• Higher clock speeds
• More cores
• More cache
• More memory
• New instructions
• Supports Low voltage DDR3 (LDDR3)

The Xeon 5600 is available in the following configurations:

Processor Model	Processor Base Frequency	Cores	L3 Cache	Power	Intel ® QPI Speed	Max Turbo Frequency	Number of Threads
X5680	3.33 GHz	6	12MB	130 W	6.4 GT / s	3.6 GHz	12
X5677	3.46 GHz	4	12MB	130 W	6.4 GT / s	3.733 GHz	8
X5670	2.93 GHz	6	12MB	95 W	6.4 GT / s	3.333 GHz	12
X5667	3.066 GHz	4	12MB	95 W	6.4 GT / s	3.466 GHz	8
X5660	2.8 GHz	6	12MB	95 W	6.4 GT / s	3.2 GHz	12
X5650	2.66 GHz	6	12MB	95 W	6.4 GT / s	3.066 GHz	12
L5640	2.26 GHz	6	12MB	60 W	5.86 GT / s	2.666 GHz	12
L5630	2.13 GHz	4	12MB	40 W	5.86 GT / s	2.4 GHz	8
L5609	1.86 GHz	4	12MB	40 W	4.8 GT / s	1.866 GHz	4
E5640	2.66 GHz	4	12MB	80 W	5.86 GT / s	2.933 GHz	8
E5630	2.53 GHz	4	12MB	80 W	5.86 GT / s	2.8 GHz	8
E5620	2.4 GHz	4	12MB	80 W	5.86 GT / s	2.666 GHz	8

Table 1: Xeon 5600 configurations

Notice the interesting low voltage processors that are available in 40 and 60 watt versions. One possible application is to use them in a passively cooled chassis (i.e. without fans). 

Now that we've got the understanding of these basic elements, it is time to put things in to practice. In my next post we'll start configuring the idle server.
 

DIMM Ranks and configuration

Server performance is highly related to its hardware configuration, In order to understand how best to configure the server a deeper understanding of its component is required. In my recent post, Introduction to DDR3 I wrote about the DDR3 technology, the different memory frequencies, 800, 1333, 1600 MHz that are adopted in recent servers CPU technology.

Another important factor to consider when configuring server memory is memory ranks, as populating a system with the wrong memory modules could result in reduction in the total capacity of the server and could create problems in performance. Some servers limit the number of memory ranks, if the number of memory ranks exceeds the specified maximum ranks set by the motherboard chipset; the server may not boot or may not operate reliably.

Memory DIMMs are configured with DRAM chips, each DRAM chip provides either 4bits (i.e x4) or 8bits ( i.e x8) of a 64bit data word. For Error Correction Code (ECC) you need 72bit, therefore 9 chips of x8 are needed to construct a 72bit data word, or 18 chips of x4.

Single-sided and double-sided DIMMs

An ECC DIMM with all nine DRAM chips on one side is called single-sided, and an ECC DIMM with nine DRAM chips on each side is called double-sided ( see Figure 1). A single-sided x8 ECC DIMM and a double-sided x4 ECC DIMM each create a single block of 72bits (64bits plus 8 ECC bits).

Single-rank, dual-rank, and quad-rank DIMMs 

DIMMs are also classified to ranks. The definition of a rank is an area or block of 64-bits (72bits for ECC memory). A single-rank ECC DIMM uses all of its DRAM chips to create a single block of 72bits, and all the chips are activated by one chip-select (CS) signal from the memory controller. A dual-rank will require two chip-select signals as it produces two blocks from two sets of chips on the DIMM. The same logic applies to Quad-rank, where you need four CS signals.

Figure 1: 72bit SDRAM DIMMs and corresponding DIMM rank

The first advantage for higher rank DIMM is the greater capacity they can offer per the available DIMM slots on the server’s motherboard. The second advantage is related to latency, where there is a higher probability for the memory controller to relocate pending memory pages.
The number of ranks per channel also effects the performance, with an odd number of ranks per channel there is performance disadvantage of realistically 2-3%. Since there 3 channels per CPU, the use dual-rank DIMMS will always result in an even number of ranks, which is another good reason why to use it.

Introduction to DDR3

Recent CPU technology has adopted the use of high speed memory module, which makes server configuration a bit tricky. I often had to figure out what kind of memory modules to use on my servers, be it a Database Server, an High Performance Computing (HPC) node or a Virtualization platform.

I hope the next series of posts will help shed some light on this matter and assist building the optimized platform for your application.

First we must understand the basics of memory technology that is currently used.  

JEDEC - Joint Electronic Device Engineering Council, is the group that set memory industry standards ,aspects such as features, functionality, AC and DC parameters, data interfaces and more.

The DDR3 SDRAM is the latest one used for servers. It is a high-speed dynamic random-access memory internally configured as an eight-bank DRAM. The DDR3 SDRAM uses a 8n prefetch architecture to achieve high-speed operation. The 8n prefetch architecture is combined with an interface designed to transfer two data words per clock cycle at the I/O pins.
Theoretically, these modules could transfer data at the effective clockrate of 800-1600MHz (for a single clock bandwidth of 400-800MHz), producing a throughput of 12800 MB/s.

Chart 1: SDRAM technologies throughput

Compatibility

DDR3 DIMMS have 240 pins, the same number as DDR2; however, since the supply voltage is different DDR3 uses 1.5V where DDR2 uses 1.8V they are incompatible. To identify between them, the DIMMS are also physically incompatible. The key notch location is different. so DO NOT try forcibly to install them. 


 DDR3 Advantages

• Higher Performance – The most important benefit is the ability to transfer 8bits of data in 1 clock cycle, which is twice than DDR2.  Higher throughput equals higher performance.
• Lower Power – DDR3 memory operate at 1.5V supply, which is 17% less than what DDR2 uses. This low power results in improving the thermal design using 30% less power.
• Larger Density – Since the DDR3 allows higher chip capacity (1GB<) it allows a maximum memory modules of 16GB.
• Data Integrity – An integrated thermal sensor refreshes the data in the device and ensures data integrity and data drainage during transfers
• Higher Speed – Twice the speed of DDR2, it has the ability to transfer I/O data eight times the speed of the memory cell it contains.

DDR3 Standard Modules

Module Name	Standard Name	Memory Clock	I/O Bus Clock	Data Rate	Peak Transfer Rate
PC3-6400	DDR3-800	100 MHz	400 MHz	800 MT/s	6400 MB/s
PC3-8500	DDR3-1066	133 MHz	533 MHz	1066 MT/s	8533 MB/s
PC3-10600	DDR3-1333	166 MHz	667 MHz	1333 MT/s	10667 MB/s
PC3-12800	DDR3-1600	200 MHz	800 MHz	1600 MT/s	12800 MB/s

Table 1: DDR3 standard memory modules

Note that Standard’s name correlates to the clockrate, not the Memory Clock or the Bus Clock, i.e DDR3-1600 offers the clockrate of 1600MHz.

Since memory configuration and capacity is effecting server performance, it is very important to understand which modules to use in which configuration. For example, populating all memory banks with the fastest modules (DDR3-1600) will cause the system to work as if DDR3-800 where used. in the next posts I’ll try to explain why.

Google Search made easy

Since it has become the most useful tool when using the internet, and it even got a catchy name for it (i.e. "Google it!"), I thought it might even more useful if I summarize some nice tips to tweak the search for the most accurate results.

Now, you may use the formatted Google Advanced Search  or you can just learn the simple key words that will help you narrow your search.

First for the basics:

•    1. Every word you enter counts
•    2. [ Yin and Yang ] is one query, while [ Yin ] and [ Yang ] are two separate queries.
•    3. Punctuation is ignored, including @#$%^&*()=+[]\ and other special characters.
•    4. Use the words that are most likely to appear on the page.
•    5. The more unique the word is the more likely you are to get relevant results.

The cool stuff:

• Phrase search ("") - Looks for exact words in that exact order without any change.
• Search within a specific website (site:) - Limit your search to a particular domain name.
• Terms you want to exclude (-) - Use the minus sign to omit searches that include words you want to exclude.
• Wild card (*) - The search treats the star as a placeholder for any unknown term(s) and then find the best matches.
• Exactly as is (+) -Since Google employs synonyms automatically, this key tells it to ignore it.
• The OR operator (|) - Since usually every word counts, this key specifically allow either one of several words.
• File extension (filetype:) - Display results that include only specified file types.
• Search title (intitle:) - Display results that include the specified title in a document.

Now let's put this in to action


Example 1 - The search entry:
 "burning down the house" "talking heads"|tom jones" site:youtube.com Try it Now
Will search for "Burning down the house"  by Talking Heads or by Tom Jones only on youtube.com


Example 2 - The search entry:
 intitle:vitae site:uk programmer -submit -apply ext:doc | ext:pdf  Try it Now
Will search for the word "vitae"  on UK domains, include the word programmer exclude the words submit apply and only in files that have the extensions .doc and .pdf


Example 3 - The search entry:
 funny * +screams mirror site:youtube.com  Try it Now 
Will search for the word  funny and then some word, the exact phrase screams and the word mirror on youtube (BTW, that's my girl that is screaming there)

Well, these are the basics, from here on out you can use any combination you choose to narrow your search to get the best results.
To find out more about the advance search options and the exceptions that it offers use Google help pages.