Active e-mail system SMTP protocol monitoring algorithm

Abstract

E-mail system is one of the most ubiquitous Internet-based applications today. It enables users to send and receive e-mail messages among each other within and from outside of the local area network. E-mail system is used every day in almost all organizations as a communication tool between managers, employees, customers, and partners for better information flow and conduct business which requires communication with people outside the organization, or from different geographical locations. The e-mail system offers a fast, reliable and easy solution for such communication. SMTP (simple mail transfer protocol) is a transportation protocol used to transfer e-mail messages over the Internet. All e-mail servers use the SMTP to send e-mails from one e-mail server to another. SMTP is also used to send e-mail messages from e-mail clients to e-mail servers. In this paper, the SMTP e-mail system protocol will be briefly explained. New active monitoring algorithm architecture is also proposed to improve the current e-mail system protocol functions and detect the SMTP protocol failure during the process of sending e-mail messages.

Full text

Proceedings of IC-BNMT2009
ACTIVE E-MAIL SYSTEM SMTP PROTOCOL
MONITORING ALGORITHM
R. Sureswaran, Hussein Al Bazar, O. Abouabdalla,
Ahmad M.Manasrah, Homam El-Taj
National Advanced IPv6 Centre (NAv6)
Universiti Sains Malaysia
Penang, Malaysia.
{sures, hussein,omar,ahmad,homam}@nav6.org
Abstract
E-mail system is one of the most ubiquitous
Internet-based applications today. It enables users
to send and receive E-mail messages among each
other within and from outside of the local area
network. E-mail system is used every day in almost
all organizations as a communication tool between
managers, employees, customers, and partners for
better information flow and conduct business which
requires communication with people outside the
organization, or from different geographical
locations. The E-mail system offers a fast, reliable
and easy solution for such communication. SMTP
(Simple Mail Transfer Protocol) is a transportation
protocol used to transfer E-mail messages over the
Internet. All E-mail servers use the SMTP to send
E-mails from one E-mail server to another. SMTP
is also used to send E-mail messages from E-mail
clients to E-mail servers. In this paper, the SMTP
E-mail system protocol will be briefly explained.
New Active monitoring algorithm architecture is
also proposed to improve the current E-mail system
protocol functions and detect the SMTP protocol
failure during the process of sending E-mail
messages.
Keywords: E-mail server, SMTP, Active
monitoring.
1 Introduction
E-mail system has many protocols, each of which
has a specific function to accomplish the process of
sending or receiving an E-mail message. The
commonly used E-mail system protocols are SMTP
protocol for sending E-mail messages and POP3 or
IMAP4 protocol for retrieving E-mail messages. E-
mail system consists of two important components
which are E-mail client and E-mail server. The E-
mail client has software that is responsible of
providing user interface view to the E-mail system,
manage E-mail messages’ functions such as
compose, send, receive and read E-mail messages.
This software is named Mail User Agent (MUA).
Users use MUA such as Outlook Express, Pine,
Eudora and Mozilla to compose their E-mail
messages, specify the sender’s and recipient’s E-
mail addresses and send the E-mail messages to the
E-mail server. MUA which is in the SMTP client
side is used to communicate between the client and
the local E-mail server without having connection
to the Internet [8,9]. Once the TCP connection is
created between the E-mail client and E-mail server,
the E-mail message is sent to the sender’s E-mail
server using the SMTP protocol. Using the same
protocol, the E-mail server sends the E-mail
message to another E-mail server or directly to the
recipients’ E-mail server using the E-mail gateway
[3]. The SMTP protocol is a client/server procedure
which starts when the client in the local network
wants to send an E-mail message to an address
outside the local network using the local E-mail
server, or when an E-mail messages is transferred
among the hosts in the same network. This
procedure commences by establishing a TCP
connection between the E-mail client and E-mail
server on port 25. Then, it performs a set of
commands dialogue till the E-mail message is
transferred to the sender’s E-mail server and quits
the communication with the E- mail server when
the client does not have any other new messages to
be sent [1,2,3].
Active network monitoring technique is one of the
network monitoring techniques that works by
sending packets from the monitoring agent to the
server or application to measure the network
performance and detect the network protocols’
failures. In this technique, sending a small number
of packets on the network traffic can be used to
acquire the required information. On the other hand,
sending a large number of packets on the network
traffic sometimes leads to the creation of extra load
on the network traffic. The active technique gives
accurate information about the services on the
network [4,5].
This paper is organized as follows: In the next
section, the E-mail system sending protocol
procedure is reviewed; Section III explains the

___________________________________
978-1-4244-4591-2/09/$25.00 ©2009 IEEE
Authorized licensed use limited to: Vimal Jyothi Engineering College. Downloaded on August 16,2010 at 08:46:58 UTC from IEEE Xplore. Restrictions apply.

proposed active SMTP protocol monitoring
algorithms. In section IV the proposed monitoring
algorithm is implemented in real-working
environment. Finally the conclusion is presented.
2 E-mail system protocols procedures
In this section, the client/server commands dialogue
of the SMTP protocol procedure and the set of
commands used to access the sending or retrieving
session are explained in brief.
SMTP is an E-mail system transferring protocol,
which is responsible for delivering the E-mail
messages from the sender to the recipient. SMTP
protocol is a client/server procedure, where the
client establishes two-way transmission channel
with the local E-mail server, followed by
performing client specific commands and finally
transfers the E-mail message to the local E-mail
server. Once the E-mail server receives the client’s
E-mail message, it takes responsible for delivering
the E-mail message to the recipient’s E-mail server.
This is performed by starting the routing process of
the E-mail message over the Internet network till
the sender’s E-mail message is delivered to the
recipient's E-mail server.
DNS system and TCP protocol are used in the
sending procedure, where DNS is used to lookup
for the domain name belonging to the recipient’s
address. Then, the domain name is resolved to an IP
address and it is used to map the E-mail message
from the sender’s E-mail server to the recipient’s E-
mail server. The client has to establish TCP
connection on port 25 with the E-mail server before
sending the E-mail message. It uses this connection
to process the operation of transferring the E-mail
message data. The processes of establishing
connection, transferring E-mail message data and
terminating the connection are the main procedures
of SMTP. However, the SMTP client/server
procedures start when the client in the local
network wants to send an E-mail message to an
address outside the local network using the local E-
mail server, or when the E-mail message is
transferred among the hosts in the same network
SMTP procedure starts at the client’s side. The
procedure starts with the client using DNS to
lookup if the address of the E-mail server is known.
This is followed by establishing a new TCP
connection using the standard three-way handshake
between the client and the E-mail server on port 25.
This connection is used to transfer the control
commands and E-mail message data to the SMTP
E-mail server. Once the TCP connection is
established; a greeting 220 reply codes will be sent
from the E-mail server to the client. This greeting
message is used to identify the domain name of the
server, version of SMTP E-mail server software
and some additional information. The sender cannot
send any command to the E-mail server until the
greeting message is received. Once the client
receives the server’s greeting, an acknowledgment
will be sent back to the server by sending using
HELO or EHLO commands. This greeting message
contains the client's identity and E-mail server
confirms back using 250 (OK) reply codes.. After
that, the E-mail client performs MAIL, RCPT,
DATA and QUIT commands to accomplish the
sending procedure. Figure 1 shows the illustration
of the SMTP client/server procedure.
Figure 1. SMTP Client/Server Procedure.
In this section, not all the commands dialogue
protocol procedure which is performed after the
access request and the E-mail server’s greeting
received at the client side are explained in detail.
For full explanation of SMTP protocol, refer to the
RFCs references [1].
3 Proposed monitoring algorithm
In this section an active monitoring algorithm to
monitor the E-mail system SMTP protocol is
proposed. The main idea that came from the active
technique is to monitor the services availability and
protocol functionality by using the services itself.
Thus, the SMTP protocol is monitored and tested
by the protocol itself.
As mentioned in section II, the procedure of SMTP
protocol starts by creating a TCP connection
session on port 25, followed by starting the
commands dialogue procedure between the E-mail
client and E-mail server to achieve the sending
process of the E-mail messages. This monitoring
algorithm tests two major parts of the SMTP
protocol on the subject of the SMTP procedure. The

Authorized licensed use limited to: Vimal Jyothi Engineering College. Downloaded on August 16,2010 at 08:46:58 UTC from IEEE Xplore. Restrictions apply.

first part tests the E-mail server’s ability to accept
an incoming TCP connection establishment request
from the E-mail clients to the E-mail server on port
25, starting with the handshake procedure and
establish connection between the E-mail client and
E-mail server on that port. The second part of this
monitoring algorithm is testing the E-mail server’s
ability to start the SMTP commands dialogue
procedure with the E-mail client and respond
correctly without any error response.
This monitoring algorithm starts by generating
SMTP request session from the monitoring agent to
the Email server on port 25, requesting the E-mail
server to establish TCP connection on that port. The
monitoring agent monitors the E-mail server’s
response. In normal cases, this response contains an
acknowledgment from the E-mail server which is
the 220 reply codes. This response indicates that the
E-mail server is ready to receive E-mail message
from the E-mail client and sends it to the specific
destination address. If the E-mail server fails to
send the 220 reply codes or send an error response,
then the monitoring agent will generate a warning
message to the network administrator which
indicates that the TCP connection on port 25 among
the E-mail server is unavailable. Otherwise, the
monitoring agent will continue to execute this
monitoring algorithm and start testing the second
case of this algorithm.
Once the TCP connection is created on port 25, the
second part of this algorithm can be achieved by
generating the HELO or EHLO command from the
monitoring agent to the E-mail server and monitors
the E-mail server’s response. In normal cases, this
response contains an acknowledgment from the E-
mail server which is the 250 OK reply codes. This
response indicates that the E-mail server is ready to
receive the E-mail message information and
continue the commands dialogue session. If the E-
mail server fails to send the 250 OK reply codes or
send an error response, then the monitoring agent
will generate a warning message to the network
administrator indicating that the SMTP protocol has
a problem.
The error response from the E-mail server for the
HELO or EHLO command is also sent with the
warning message. Otherwise; the monitoring
system generates the QUIT command and the E-
mail server terminates the TCP connection and
sends 221 OK reply codes to the monitoring agent.
The time required to accomplish this monitoring
algorithm test is calculated and stored in the
monitoring system database. Algorithm 1:
describes the SMTP protocol monitoring algorithm.
Algorithm1: SMTP Protocol Monitoring.
Inputs: SMTP Request Session, SMTP
Commands.
Output: SMTP Protocol Status.
Method:
Start
1- Generate SMTP session request from the
monitoring agent to the SMTP port 25.
2- monitor the E-mail server respond where:
- If the E-mail server responds using 220 reply
code, then go to step number 3.
- If the E-mail server responds using Error reply
code, then generate a warning message to inform
about the TCP connection problem on port 25.
3- Generate HELO/EHLO command from the
monitoring agent to the E-mail server.
4- Monitor the E-mail server respond where:
- If the E-mail server responds using 250 reply
code, then go to step number 5.
- If the E-mail server responds using Error reply
code, then generate a warning message to inform
about the SMTP protocol problem.
5- Generate QUIT command to close the SMTP
protocol session on port 25.
6- Calculate the time, which is required for the
TCP connection establishment, send the
commands, receive the responds and store it in
the monitoring system database.
End
4 Monitoring and results
In this section, the real-working environment
monitoring results for the proposed E-mail system
protocols monitoring algorithms are presented.
Plug-in JAVA software is developed to monitor one
of the E-mail servers in the National Advanced
IPv6 Center (NAv6). Every 15 seconds the
monitoring protocols algorithm is executed
automatically in order to calculate the E-mail
server’s respond time in the form of milliseconds,
generate protocol chart behavior of the SMTP
protocol and select the minimum E-mail server
respond time, maximum E-mail server respond time,
as well as the average E-mail server respond time.
Moreover, the proposed monitoring results are
obtained from 12 hours of continues monitoring,
starting from 10:00AM, and stopped at 10:00PM.
For the SMTP protocol, the minimum respond time
for an E-mail server to accomplish the process of
creating TCP connection session, receive E-mail
server greeting message, send HELO command,
and terminate the connection is 16 MS. On the
other hand, the maximum E-mail server’s respond
time is 828 MS. During the test period, some
anomalous results were met which are more than 10
seconds per test. In the test, the anomalous results
are ignored, and the average response time for 12
hours monitoring of 2880 active tests is calculated,

Authorized licensed use limited to: Vimal Jyothi Engineering College. Downloaded on August 16,2010 at 08:46:58 UTC from IEEE Xplore. Restrictions apply.

which is 50 MS. The monitoring algorithm did not
detect any protocol failure during the monitoring
time, whereas in other monitoring the proposed
algorithm improved its efficiency, and accuracy in
detecting all the E-mail system protocol error
respond, delay respond, and failures to accomplish
the monitoring design. To accomplish the
monitoring processes, the E-mail server’s respond
time on port 25 is ranged between 16 to 900 MS,
where in some cases there are abnormal behaviors
which are due to E-mail server load or delay
problems. Figure 2 depicts the 12 hours SMTP
protocol monitoring behavior.
Figure 2. SMTP protocol monitoring behavior.
This monitoring algorithm is able to detect any
problem that occurs during the monitoring period.
The problem can be in the TCP establishing
procedure, where the connection on port 25 is not
available. In this case the E-mail server sends back
an error message, for example, “Could not connect
to the SMTP server name”. The commands
dialogue procedure and the responds from the E-
mail server’s end can be monitored and detect any
problem in the protocol if there’s any. Table1
describes the possible E-mail server responds for
each command in this monitoring algorithm.
Table 1 SMTP E-mail server replies.
Command Reply Description
Connect TCP 220 Service ready
421 Service not available
HELO/EHLO 250 Requested action okay
500 Command unrecognized
501 Syntax error
504 Command parameter not
implemented
521 Does not accept mail
421 Service not available
550 Not implemented
QUIT 221 Service closing
transmission channel
500 Command unrecognized
5 Conclusion
E-mail system uses the transaction protocols to
send and retrieve E-mail messages over the local
network and Internet networks. This paper
proposed an active E-mail system SMTP protocol
monitoring algorithm using the active monitoring
technique, to provide more efficient tools which are
able to monitor the E-mail system protocol in real-
working environment and detect any problems
related to the SMTP protocol. The paper also tested
out the proposed algorithms and presented the
output of the experiments on that algorithm.
References
[1] J. Klensin, “Simple Mail Transfer Protocol”,
RFC 2821, April 2001.
[2] M. H. Haggag "ENHANCED DELIVERY
THROUGH A SMART SMTP CLIENT".
The International Journal on Intelligent
Cooperative Information, VOL. 4, NO. 1,
January 2004.
[3] P. Tzerefos, C. Smythe, I. Stergiou, and S.
Cvetkovic, “A Comparative Study of Simple
Mail Transfer Protocol (SMTP), Post Office
Protocol (POP) and X.400 Electronic Mail
Protocols”, Proceedings of the IEEE 1997
22nd Conference on Local Computer
Networks - LCN, pp 545-554, 199.
[4] G. Bartlett, J. Heidemann, C. Papadopoulos,
“Understanding Passive and Active Service
Discovery”, 7th ACM SIGCOMM conference
of Internet Measurement Conference. VOL 7,
2007.
[5] MOHYUDDIN, A. & DOWLAND, P. S.
(2007) The Art of Network Monitoring
Advances in Network & Communication
Engineering 4. University of Plymouth UK.
[6] Giencke, P. (1995). The future of email or
when will Grandma be on the net? Paper
presented at the Electro/95 International.
Professional Program Proceedings.
[7] ROMAN, H. T. (2007) You’ve Got Mail!
Understanding How E-mails Works. Tech
Directions, 66, 2.
[8] Halsall, F. (2005). Computer Networking and
the Internet (Fifth ed.): Addison-Wesley.
[9] Lam, H.-y. (2007). A Learning Approach to
SPAM Detection Based on Social Network.
The Hong Kong University of Science and
Technology, Hong Kong.

Authorized licensed use limited to: Vimal Jyothi Engineering College. Downloaded on August 16,2010 at 08:46:58 UTC from IEEE Xplore. Restrictions apply.