Cooling Energy Consumption Investigation of Data Center IT Room with Vertical Placed Server

Abstract

As energy consumption by cooling data center IT equipment can be over 40% of total energy consumption, efficient cooling for large data centers is essential for reducing operation costs. Modern data centers are complex systems involving IT facilities, power system, cooling and ventilation systems. In our previous work, literature study was made to investigate available data center energy consumption models; and energy consumption models for data center IT room with distributed air flow control were developed. In this paper, the models are further extended and developed to cover the combined distributed air flow control and vertical placed servers in raised floor ventilation system. Simulation of the three types of ventilation systems with Even load, Idle server and Uneven load scenarios showed that significant cooling energy consumed by a traditional ventilation system can be saved by applying the proposed new concept and method.

Full text

1876-6102 © 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of the scientific committee of the 8th International Conference on Applied Energy.
doi: 10.1016/j.egypro.2017.03.581
Energy Procedia 105 ( 2017 ) 2047 – 2052
ScienceDirect
The 8
th
International Conference on Applied Energy – ICAE2016
Cooling Energy Consumption Investigation of Data Center IT
Room with Vertical Placed Server
X. Zhang
a
*
, T. Lindberg
b
, N. Xiong
c
, V. Vyatkin
d,e
, A. Mousavi
d
a
Department of Power Device, ABB AB, Corporate Research, Vasteras, Sweden
b
Department of Engineering and Physics, Karlstad University, Sweden
c
School of Innovation, Design and Engineering, Malardalen University, Vasteras, Sweden
d
Department of Computer Science, Electrical and Space Engineering, Lulea University of Technology, Sweden
e
Aalto University, Helsinki, Finland
Abstract
As energy consumption by cooling data center IT equipment can be over 40 % of total energy consumption, efficient
cooling for large data centers is essential for reducing operation costs. Modern data centers are complex systems
involving IT facilities, power system, cooling and ventilation systems. In our previous work, literature study was made
to investigate available data center energy consumption models; and energy consumption models for data center IT
room with distributed air flow control were developed. In this paper, the models are further extended and developed to
cover the combined distributed air flow control and vertical placed servers in raised floor ventilation system. Simulation
of the three types of ventilation systems with Even load,Idle server and Uneven load scenarios showed that significant
cooling energy consumed by a traditional ventilation system can be saved by applying the proposed new concept and
method.
© 2016 The Authors. Published by Elsevier Ltd.
Selection and/or peer-review under responsibility of ICAE
Keywords: data center; server rack; air flow; energy efficiency
1. Introduction
With rapid growth of large data centers worldwide, data centers become energy intensive processes
accounting for over 1% of the world’s electricity usage [1]. Large data centers with capacity up to 120 MW
have been built in recent years. Energy efficiency becomes even more important for these data centers. 
ǤInvestigation showed that
energy consumption by cooling data center IT equipment is between 30% and 55% of the total energy
* Xiaojing Zhang. Tel.: +46 21 323062
E-mail address: xiaojing.zhang@se.abb.com
Available online at www.sciencedirect.com
© 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of the scientific committee of the 8th International Conference on Applied Energy.

2048 X. Zhang et al. / Energy Procedia 105 ( 2017 ) 2047 – 2052
consumption. Cooling and ventilation system consumes on average 40% of the total energy consumption
in a data center [2].
A comprehensive data center power consumption model describing IT room, computer room air
handling (CRAH), data center ventilation and cooling characteristics was developed to cover traditional
raised floor ventilation system and the distributed air flow system [3]. The distributed air flow system
showed advantages when IT load or cooling load varies for each server rack. The simulation results showed
that up to 16% of power consumption reduction can be achieved by applying the distributed air flow control
for the data centers with idle server and uneven IT load. Literature study of energy consumption models for
different subsystem components was also made in our previous work with intention of using models to
build a powerful simulation tool and to estimate and simulate different operation scenarios as decision
support for data center design and operation. This paper focuses on modelling air flow with the vertical
placed server and evaluates operation benefits through simulation and comparison between traditional
designs of air flow and the proposed scheme of distributed air flow system combined with vertical place
server.
2. Data center distributed air flow cooling with vertical placed server
In the conventional data center with hot aisle/cold aisle, cold air generated by the cooling system is
supplied through a plenum under the floor and perforated air flow panels. The cold air flows up horizontally
entering the tiny spaces between the servers from one side of the servers and leaving from another side.
Higher flow pressure drop, cold and warm air mixture on the upper side of racks are the main disadvantages.
Most data centers operate under varying and uneven IT loads resulting in different cooling loads over
different racks. Hot spots cannot be avoided when cold air is uniformly supplied to all server racks. To meet
the cooling needs of individual server racks without local overcooling, it requires that cold air being
distributed on-demand across local cooling loads. Figure 1 shows a data center with uneven server
utilization. The concept of distributed air flow control is to divide a data center in pre-defined zones and
different amounts of air flow are supplied to these zones across the data center based on local cooling loads.
It requires special ventilation system design under the raised floor to distribute cooling air through
ventilation ducks and dampers. By this solution, the hot spots and overcooling can be reduced or even
eliminated. A significant energy saving can also be achieved.
Fig. 1. Data center with uneven server utilization
In the vertical placed sever concept, the traditional hot aisle/cold aisle air supply showed in Figure 2 is
replaced by air supply directly from the bottom of each server rack and it is illustrated in Figure 3. The
concept was described in detail in our previous work [2][3]. The distributed air flow control can be applied
in a traditional raised-floor data center or air flow control system with or without vertically placed servers.

X. Zhang et al. / Energy Procedia 105 ( 2017 ) 2047 – 2052 2049
Fig. 2. Traditional hot aisle/cold aisle air supply.
Fig. 3. Air supply with vertical placed server rack.
3. Model for the vertical placed server rack
The data center power consumption models previous developed covered the power consumptions of
servers, racks, CRAH, chiller, cooling tower, UPS and PDU. Comparing to the traditional hot aisle/cold
aisle ventilation system, models to calculate system pressure drop and flow rates are required due to changes
of ventilation system setup. CRAH fan power consumption is affected by ventilation system pressure drop.
For standard type of racks, the pressure drop over the CRAH units was earlier developed [4][5]. In a
standard server rack, servers are stacked on top of each other in parallel. The pressure drops over front
doors, rear doors and servers contribute to the change in pressure over a server rack. For the vertical placed
server rack showed in Fig. 3, the pressure drop models are further developed in this study. The setup consists
of two separate compartments with individual air supply. Each compartment has three rows of racks. Air
streams flow up and warm air is dissipated to hot aisles on both sides between server rows. Since the air
flow enters the bottom of the compartments, the flow rate will be reduced for each row further up. Because
of this change in flow rate, the pressure drop needs to be determined for each row. With three-row and two-
compartment (lower and upper in Fig. 3) design, pressure drops can be described by equations (1) to (6),
οܲ௟௢௪௘௥ǡଵൌܥ௦௘௥௩௘௥
ଶܸሶ௦௘௥௩௘௥
ଶൌܥ௦௘௥௩௘௥
ଶቌ௏ሶభ
ಿೞ೐ೝೡ೐ೝೝೌ೎ೖ
ಿೝ೚ೢೝೌ೎ೖ ቍଶ
(1)
οܲ௟௢௪௘௥ǡଶൌܥ௦௘௥௩௘௥
ଶቌ௏ሶమ
ಿೞ೐ೝೡ೐ೝೝೌ೎ೖ
ಿೝ೚ೢೝೌ೎ೖ ቍଶ
(2)

2050 X. Zhang et al. / Energy Procedia 105 ( 2017 ) 2047 – 2052
οܲ௟௢௪௘௥ǡଷൌܥ௦௘௥௩௘௥
ଶ൭௏ሶయ
ಿೞ೐ೝೡ೐ೝ̴ೝೌ೎ೖ
ಿೝ೚ೢ̴ೝೌ೎ೖ ൱ଶ൅ܥ௕௘௡ௗ
ଶܸሶଷଶ (3)
οܲ௨௣௣௘௥ǡଵൌܥ௦௘௥௩௘௥
ଶቌ௏ሶభ
ಿೞ೐ೝೡ೐ೝೝೌ೎ೖ
ಿೝ೚ೢೝೌ೎ೖ ቍଶ൅ܥௗ௜௦௧
ଶܸሶଵଶ (4)
οܲ௨௣௣௘௥ǡଶൌܥ௦௘௥௩௘௥
ଶቌ௏ሶమ
ಿೞ೐ೝೡ೐ೝೝೌ೎ೖ
ಿೝ೚ೢೝೌ೎ೖ ቍଶ
(5)
οܲ௨௣௣௘௥ǡଷൌܥ௦௘௥௩௘௥
ଶቌ௏ሶయ
ಿೞ೐ೝೡ೐ೝೝೌ೎ೖ
ಿೝ೚ೢೝೌ೎ೖ ቍଶ (6)
where ܸሶ1, ܸʹǡ
ሶܸሶ͵ are the flow rates through the first, second and third rows of each ܸሶcompartment,
respectively. Cserver is coefficient for server pressure lossǤܸሶ௦௘௥௩௘௥ is the flow arte through the servers in the
first compartment. Cbend is coefficient for the pressure loss caused by the roof of the lower compartment,
which forces the flow to bend off its path and leave through the sides. Cdist is the pressure loss coefficient
of the distributor that distributes the air flow equally in the bottom of the upper compartment. Nserver_rack is
the number of servers in each rack. Nrow_rack is the number of racks in each row. In traditional server racks,
supply air flow rate from the CRAH units is based on the racks with the maximum heat load. For zones
with lower utilization, the air bypass increases since the servers need less cooling. Track_in or Track_out can
be used as the set-point parameter, the mixed flow of rack flow and air bypass can be expressed as,
ܶ௥௔௖௞̴௜௡̴௬ ൌூି௃௏ሶೝೌ೎ೖ̴೤்ೝೌ೎ೖ̴೚ೠ೟̴ೞ೐೟
ଵି௃௏ሶೝೌ೎ೖ೤ା௄ (7)
ܫൌ߳஼ோ஺ுಹಶܥ௠௜௡ܶ஼ோ஺ுಹమೀ೔೙
߳஼ோ஺ுಹಶܥ௠௜௡െܥ௔௜௥
ܬൌ ܰ௥௔௖௞
ܰ஼ோ஺ு൫ͳെߜ௟௘௔௞௔௚௘൯ܸሶ஼ோ஺ு
ܭൌ ܥ௔௜௥
߳஼ோ஺ு̴ுாܥ௠௜௡െܥ௔௜௥
where ߳஼ோ஺ுಹಶ is heat exchanger effectiveness, Cmin and Cair are the lower heat capacity rate and the heat
capacity rate of the air flow respectively,
ܶ஼ோ஺ுಹమೀ೔೙is the temperature of the BCW water entering the
CRAH heat exchanger and ߜ௟௘௔௞௔௚௘ is leakage flow fraction. The server fans are assumed to keep the rack
outlet temperature constant at the set-point value.
ܸሶ௥௔௖௞ ൌொሶೝೌ೎ೖ
ఘೌ೔ೝ௖೛ǡೌ೔ೝ൫்ೝೌ೎ೖ̴೚ೠ೟ି்ೝೌ೎ೖ̴೔೙൯ (8)
The rack flow rate for vertical server racks can be determined directly from the CRAH flow rate since
these supply just the amount of air needed by the racks.
ܸሶ஼ோ஺ு̴௬ ൌேೝೌ೎ೖ௏ሶೝೌ೎ೖ̴೤
ே಴ೃಲಹ൫ଵିఋ೗೐ೌೖೌ೒೐൯ (9)

X. Zhang et al. / Energy Procedia 105 ( 2017 ) 2047 – 2052 2051
where ܳሶ௥௔௖௞ is the heat load of one rack. ߩ௔௜௥ and ܥ݌ǡ௔௜௥ are density and the specific heat of the air.
Nrack and NCRAH are the number of racks and CRAHs.
4. Simulation and Results
A case study of a data center with 208 racks is simulated. Each CRAH has capacity of 452 kW. Cooling
design is with redundancy of N+1. All servers were assumed to be identical, making heat load and cooling
load differences dependent only on the different server utilizations. 13 zones are defined and each rack
contains 42 servers. As the same as our previous distributed air flow investigation, the simulation was made
in MATLAB and covered three scenarios, Even load,Idle server and Uneven load. Both racks and CRAH
units are assumed to operate on the principle of keeping a constant level for the outlet temperature of the
topmost row in a rack compartment, since this is the hottest and thus critical temperature measured with
available sensors. Fig. 4 shows the effect of the rack outlet temperature on the total cooling power
consumption for all three scenarios and with three air supply systems, namely traditional air supply, the
distributed air flow control and the distributed air flow control with vertical placed server rack. When the
server IT load are evenly distributed as shown in Fig. 4 (a), there is no benefit to apply only the distributed
air flow control. From Fig. 4 (b) and 4 (c), it is evident that the distributed air flow control saves energy
and the combined distributed air flow control with vertical placed server rack saves even more energy,
which is mainly ǤBecause cooling air is supplied on demand for
individual zones; cooling air bypass flow is eliminated and vertical cooling air flow has much lower
pressure drop than traditional air supply. Owing to the vertical placed server with vertical air flow through
servers it is favourable to heat transfer with ǡ
Ǥ
Ǥ
Fig. 4. Total cooling power consumption as a function of the rack outlet temperature for traditional air supply, distributed air flow
control and distributed air flow control with vertical placed server; (a) Even load, (b) Idle servers, (c) Uneven load

2052 X. Zhang et al. / Energy Procedia 105 ( 2017 ) 2047 – 2052
Fig. 5 shows the total cooling power consumption for three ventilation systems and three scenarios, Even
load,Idle server and Uneven load with rack outlet temperature set point of 37°C. A base sever utilization
of 0.85 is applied for all three scenarios. In Idle server and Uneven load, 9 and 7 out of the 13 zones have
the basic utilization respectively. The rest of the servers are either idle or with defined utilizations [4].
Depending on system setup, over 29% of cooling energy can be saved with the proposed solution. The
current study is one of the sub-work packages in our modelling and simulation effort. It is limited with
available data center operation measurement and data. There is on-going project for Swedish national data
center test facility in North of Sweden. The next phase of development is to test and verify these results in
this facility.
Fig. 5. Total cooling power consumption for all scenarios.
5. Conclusion
This paper has presented simulation results for the distributed air flow control with the vertical placed
server in comparison with traditional data center ventilation and air flow design. Previous developed data
center energy consumption models have been extended to cover the vertical placed server design. The
distributed air flow control with vertical servers showed the lowest power consumption in all scenarios.
References
[1] J. G. Koomey. Growth in data center electricity use 2005 to 2010.Analytics Press, Oakland, 2011.
[2] Z. Song, X. Zhang, C. Eriksson, Data center energy and cost saving evaluation. The 7th International Confernece on Apllied
Energy, Taipei, Taiwan, 2015.
[3] X. Zhang, T. Lindberg, K. Svensson, V. Vyatkin, A. Mousavi, 7th International Confernece on Computer Modeling and
Simulation, Brisbane, Australia, 2016.
[4] T. Van Giang, D. Vincent, and B. Seddik. Data center energy consumption simulator from the servers to their cooling system.
In PowerTech (POW- ERTECH), 2013 IEEE Grenoble. IEEE, June 2013.
[5] M. Iyengar and R. Schmidt. Analytical modeling for thermodynamic characterization of data center cooling systems. Journal
of Electronic Packaging, 101, June 2009.
Xiaojing Zhang received Ph.D. degree in energy technology in 1997 at Royal Institute of
Technology, Stockholm, Sweden. He joined ABB in 1998 and his career included 18 years of
industrial experiences in process industry and datacenter infrastructure management. Dr. Zhang is
currently Principal Scientist at ABB Corporate Research, Sweden and Adjunct Professor at Lulea
University of Technology, Sweden.