DIMM Ranks and configuration

Server performance is highly related to its hardware configuration, In order to understand how best to configure the server a deeper understanding of its component is required. In my recent post, Introduction to DDR3 I wrote about the DDR3 technology, the different memory frequencies, 800, 1333, 1600 MHz that are adopted in recent servers CPU technology.

Another important factor to consider when configuring server memory is memory ranks, as populating a system with the wrong memory modules could result in reduction in the total capacity of the server and could create problems in performance. Some servers limit the number of memory ranks, if the number of memory ranks exceeds the specified maximum ranks set by the motherboard chipset; the server may not boot or may not operate reliably.

Memory DIMMs are configured with DRAM chips, each DRAM chip provides either 4bits (i.e x4) or 8bits ( i.e x8) of a 64bit data word. For Error Correction Code (ECC) you need 72bit, therefore 9 chips of x8 are needed to construct a 72bit data word, or 18 chips of x4.

Single-sided and double-sided DIMMs

An ECC DIMM with all nine DRAM chips on one side is called single-sided, and an ECC DIMM with nine DRAM chips on each side is called double-sided ( see Figure 1). A single-sided x8 ECC DIMM and a double-sided x4 ECC DIMM each create a single block of 72bits (64bits plus 8 ECC bits).

Single-rank, dual-rank, and quad-rank DIMMs 

DIMMs are also classified to ranks. The definition of a rank is an area or block of 64-bits (72bits for ECC memory). A single-rank ECC DIMM uses all of its DRAM chips to create a single block of 72bits, and all the chips are activated by one chip-select (CS) signal from the memory controller. A dual-rank will require two chip-select signals as it produces two blocks from two sets of chips on the DIMM. The same logic applies to Quad-rank, where you need four CS signals.

Figure 1: 72bit SDRAM DIMMs and corresponding DIMM rank

The first advantage for higher rank DIMM is the greater capacity they can offer per the available DIMM slots on the server’s motherboard. The second advantage is related to latency, where there is a higher probability for the memory controller to relocate pending memory pages.
The number of ranks per channel also effects the performance, with an odd number of ranks per channel there is performance disadvantage of realistically 2-3%. Since there 3 channels per CPU, the use dual-rank DIMMS will always result in an even number of ranks, which is another good reason why to use it.