A note on Server Memory – DDR3 SDRAM

In this article, we will discuss about one of the most important components of a Server, after the Server Processor – Primary Memory (RAM). We will see how server memory is different, what is DDR3 RAM, what are DIMM’s, what are registered and unregistered memory modules with a small reference architecture diagram.

Server memory DIMM architecture diagramArchitecture diagram:

The above architecture diagram is just a reference architecture for server memory, and other configurations are very much possible. In the diagram, there is a server processor, a memory controller (sometimes within the server processor), three individual channels going out of the server processor to connect to the DIMM’s which hold the individual RAM chips – there are three banks (1,2,3), each capable of holding three RAM chips in the DIMM slots.

For example, if 1 GB DDR3 SD RAM chips are used to populate all the DIMM slots, we get a total of 9 GB of memory for the above referenced architecture, with three individual channels (buses) that can access them independently. There is only one socket shown here, in practice multiple sockets (that can hold multiple processors) are very much possible.

Introduction:

The requirements for server memory (RAM) has always been different from the desktop version. Server memory requires to be reliable and self correcting. The capacity of memory that can be accommodated by some servers is huge (for example, a single server can have up to 384 GB of RAM). The additional memory is especially useful in case of virtual servers – as a lot of individual virtual servers reside in a single server and each has its own memory requirements. One more advantage of having a lot of memory is the ability to replace a lot of servers with fewer ones. While looking at improving the performance of servers, it may be prudent to look at increasing the amount of available server memory.

Generally, in a Random Access Memory (RAM), the memory controller or processor can access the memory from any point of the chip (instead of having to read sequentially like the tapes). With RAM, the capacitors hold bit information (0,1) – 0 with no charge and 1 with charged cells (electrically). But fully charged capacitor loses its charge very fast. So, a memory controller is required to recharge capacitors (that contain charge) very frequently (like thousands of times per second).

DIMM/ DDR3 SDRAM:

DIMM stands for Dual In-line Memory Module. Without getting too technical, these are the slots where the DDR3 SDRAM memory chips are inserted. Typically, DDR3 DIMM’s have 240 pins. In the above diagram, the nine small blue boxes represent the empty (assuming) DIMM slots.

DDR3 SDRAM refers to Double Data Rate 3 Synchronous Dynamic Random Access Memory. Its sufficient to understand that this is the latest version of RAM that is popularly used with servers. Earlier, there were DDR and DDR2 type of RAM, which are still in use by some servers. But DDR3 technology is not backwards compatible with DDR2 or DDR and hence the RAM chips manufactured for DDR3 cannot be used with the previous generation.

DDR3 is DRAM interface specification and it does not refer to the arrays that store the data. DDR3 can transfer the data at twice the speed of DDR2. DDR3 works at 1.5V (when compared to 1.8V for DDR2) and hence it utilizes lesser power. DDR3 can transfer power at 800-2133 MT/s. The next generation technology to succeed DDR3 is DDR4, which is in design stage now.

ECC & Registered Memory:

When compared to desktop memory, server memory has more capacity. Other than that, server memory employees two important techniques that increase its reliability – ECC & Registered memory.

ECC – Error Correction Code. Sometimes, when bits are written in to individual cells of a RAM module, there is a possibility that some bit is written with the wrong value (0 instead of 1, for example). So, servers employ a technique called ECC, which uses  mathematical algorithms to both detect and correct single bit errors, often in each byte of RAM. So, ECC DIMM’s have extra data bits (per byte) to detect and correct errors, mostly coordinated by the memory controller. Because of the extra capacity and higher reliability, they are more expensive and slightly slower than normal RAM.

Registered Memory – With a registered memory, there is a hardware register that is placed between the DRAM and the memory controller for a higher degree of reliability. For two or more DIMM’s per channel (which is mostly the case with servers), RDIMM (Registered DIMM) will have a lower latency and better bandwidth than UDIMM (Unregistered DIMM). Also, RDIMM’s can accomodate higher RAM sizes than UDIMM – So, they enable more scalable server memories.

Some salient points about Server DDR3 SDRAM:

  • As you can see in the above diagram, there may be more than one DDR3 channel per socket (processor) and each channel may sometimes have sub-channels as well.
  • Lower memory capacity DDR3 SDRAM (Like 1 GB) might be more cost effective (per GB) than higher capacity DDR3 SDRAM (Like 8 GB) but when more lower capacity DDR3 SDRAM modules are used, the total memory capacity achievable by the server and the future expandability is limited.
  • The DDR3 DIMM has a maximum rating of speed that it can run (MT/s or MHz). The common speeds at which these modules run are 800 Mhz, 1066 Mhz & 1333 Mhz. Mixed DIMM speeds can be installed but not recommended as all of them will run at the speed of the slowest populated DIMM.
  • Fewer high capacity DIMM’s may give a better performance than many low capacity DIMM’s populated to achieve the same memory capacity. But this is subjective to other factors as well.
  • If there is a triple channel Processor RAM (like the one shown in the above diagram), it might be better to populate it with DIMM modules in the multiples of three (3,6,9….) – filling up individual banks, for best performance. Similarly for dual channel, it is multiples of two (2,4,6…..).
  • With multiple processors, it may be better to balance memory across processors (instead of filling up all the memory modules with the first processor) – preferably of same size and type.

excITingIP.com

You could stay up to date on the various computer networking & related IT technologies by subscribing to this blog with your email address in the sidebar box that says, ‘Get email updates when new articles are published’