Enhanced IPTV Services through Time
Lourdes Beloqui Yuste
Discipline of Information Technology
Discipline of Information Technology
Abstract—User interactivity and customisation are key features
that must be promoted to help distinguish IPTV from satel-
lite/aerial/cable TV alternatives. Furthermore, IPTV providers
also have to compete with Internet TV by providing extra features
to justify the cost. In this paper, we present a number of scenarios
whereby IPTV providers can achieve this through the use of
synchronised time. Integrating and tightly synchronising multiple
user-selected media streams in Real-Time that are logically and
temporally related potentially adds signiﬁcant extra value to any
IPTV platform. It also raises signiﬁcant research challenges. We
outline the prototype developed for a TV/Text Feed scenario and
present some details of ongoing and future work.
Index Terms—IPTV, RSS, Media Synchronisation.
The difﬁculty in synchronising multiple media streams
over an IP Network is derived ﬁrstly from the potentially
separated locations of the sources; secondly, the issue
of where to perform the synchronisation, and thirdly the
technical challenges of displaying multiple synchronised
ﬂows to the ﬁnal user. In an IPTV platform the operator
often has control over all media streams on the network so
the ﬁrst synchronisation challenge can be managed through
speciﬁcation of media content. In Fig. 1 different multimedia
sources connected to the same Network Time Protocol (NTP)
Server are synchronised at source ready to send to the client
through an IP Network.
We envisage that IPTV beneﬁts such as superior image
quality, safe well-managed network and known viewers ,
can be combined with the advantages of Internet TV such as
worldwide access and unlimited media choice .
We envisage that a number of scenarios where Real-Time
integration and synchronisation of user-selected media streams
that are both logically and temporally related, could generate
signiﬁcant end-user interest especially in sports related events.
To enable IPTV to present multiple media streams that are
logically and temporally related, precision time stamping at
source and synchronisation at delivery are key requirements.
Examples from the sporting world include: in the Soccer
World Cup where group of games are played simultaneously
users can select to be informed about one or more games
in Real-Time. Media streams might include related video
streams, video stream with Internet Radio stream or a
Fig. 1. Different multimedia sources (TV, Radio and Feed) to be synchronised
at source via NTP Server or equivalent
combined Real Simple Syndication (RSS)/Atom Feed with
All of these scenarios, merging multiple video streams,
TV/Feed and TV/Radio, present a range of common research
challenges; we identify and partially examine these through
implementation of a TV/RSS prototype and indicate our plans
to expand the work.
The remainder this paper is structured as follows: Section
II describes the different scenarios, section III outlines the
TV/Feed design issues, section IV details timing issues
for both video and text Feeds, section V describes the
implementation, section VI outlines our research plans, and
section VII concludes the paper.
II. SC ENA RI OS
A. Multiple Video Streams
This scenario presents a mosaic of video/media streams of
a live World Cup Football match originating from different
content providers. For example a video stream of a match plus
another two video streams from the two different sets of fans in
DIFF ERE NCE S: S ERVE R-S IDE V ER SUS CL IE NT-SID E
Server-side Synchronisation Client-side Synchronisation
Maximises clients bandwidth: only
one multimedia stream is delivered
Requires more client bandwidth:
two multimedia streams delivered
Server cannot provide all different
client’s option in large systems
Unlimited options to choose every
option in the STB
System not scalable System scalable
Increase server complexity/cost Increase STB complexity/cost
Server overload Client overload
their home countries. This mosaic of the three different media
streams comprising the live match feed, and video feeds of
the two home-based fans in their respective countries, allows
the end-user to see the simultaneous reaction to a match event.
Typically the audio channel should be the one related to the
TV channel transmitting the sport event.
There are different types of multimedia synchronisation
described in . This case of video and text feed can be con-
sidered as inter-media synchronisation because audio, video
and Feed textual information are synchronised.
The user selects a sporting event as well as a related
Feed which has the information that the user is interested
in. Presenting betting odds/prices via a Feed in Real-Time on
a live streamed horse racing event is one example. Another
would be Formula 1 racing whereby a Feed gives Real-Time
information on lap times in one Feed and inter-car distances
This could be used to facilitate Real-Time betting where a
Feed can show all the odds related to the sport event displayed
on TV. The alignment of the video and Real-Time odds as well
as the exact time when a client execute a bet is critical.
In this scenario a soccer fan watches his/her team in the
World Cup, from a foreign country via IPTV, but wishes to
listen to his/her national audio commentary on the match. The
radio could either be delivered via IPTV or Internet TV. In
this case, the IPTV provider must remove the embedded audio
from the video and replace it with a precisely aligned user-
chosen audio stream.
IPTV provides a concrete number of Radio channels. How-
ever Internet Radio widens the client’s choice although it adds
more technical challenges due to the use of the public IP
III. TV/FEE D DES IG N ISS UES
In the second scenario presented above, logically related TV
and RSS Feeds, though separately sourced, can be selected by
the user to have his/her own personalised TV. They choose
which TV channel and the type of information they want to
receive via a Feed.
Regardless of scenario, in terms of scalability, a key issue to
MAI N DIFFE REN CE S: RSS 2.0/ATOM
RSS 2.0 Atom
rssChannel <pubDate>atomFeed <updated>
— atomEntry <published>
rssItem <pubDate>atomEntry <updated>
Date format follows: obsolete
RFC822 (Second level granularity)
Date format follows: RFC3339
compatible with ISO 8601 (Better
than second level granularity)
be implemented is location of synchronisation, server-side or
client-side. This, in turn, will impact on the potential for mul-
ticasting which will have implications for bandwidth as well
as server/client complexity and Set-Top Box (STB) cost. Both
options for deployment were implemented; the integration and
synchronisation facility running on the server and running on
the client. Table 1 establishes the main differences between
the two options.
IV. TIMING DETA IL
A key requirement is for media content to be time stamped
at source with a universal time source. NTP is the most
widely used mechanism that allows synchronisation of dif-
ferent sources to better that 1 msec on LANs and less than
10msec on WANs  when properly implemented.
An IPTV provider often has full control of media sources
on its own well-managed private network and can therefore
ensure that media content is adequately time stamped. In our
scenario, this refers to Feed and Video time stamps.
A. RSS/Atom Timing
There are two main Feed formats: RSS  and Atom .
Although 80% of feeds use RSS , it is a frozen standard
and no changes can be made . Atom has been adopted
as IETF Proposed Standard and enhances the RSS format
and it could provide future improvements including making
compulsory milliseconds accuracy in the time formats. In table
2 we present the main differences between elements/tags in the
two standards and the date format standards they use.
RSS 2.0 is based on a tag Extensible Mark-up Language
(XML) format. In the RSS Channel, following RSS 2.0
speciﬁcation , the tags related to time are optional. The
<pubDate>tag indicates the time when the channel has
been published; <ttl>indicates the period in minutes that the
client waits until the channel is reloaded from the server, and
<lastBuildDate>indicates when the channel’s content was
last modiﬁed. The individual items also may have <pubDate>,
different from above, which details the time and date the item
has to be displayed. All of them follow the RFC822  format
which includes the date, day of the week and time in seconds,
Greenwich Meridian Time (GMT). Second level granularity is
insufﬁcient for our application scenario where synchronisation
Fig. 2. Example time standards used by RSS (RFC822) and Atom (RFC3339)
Fig. 3. RSS 2.0 and Atom time related tags used as time stamps for
of the order of millisecond is required.
Like RSS, Atom is based on XML but adds new features.
Atom adds the namespace structure so all elements belong to
one deﬁned namespace . Tags and time format are different
In Atom the tag <published>in an entry is not obligatory
and it indicates the time the entry has been created. In Atom
the tag <updated>is compulsory for the feed and the entries;
it indicates the time when the information has been updated.
The main difference is that Atom uses RFC3339  which
is compatible with ISO 8601  and speciﬁes granularity
better than seconds but it does not specify how many digits.
Again, for multimedia synchronisation purposes, particularly
scenario three, we would recommend a minimum of 3 digits
to have milliseconds in order to have acceptable inter-media
synchronisation. In Fig. 2 an example of both formats, RFC822
and RFC3339, can be found.
For our purpose all time related tags would be compulsory
in the RSS/Atom Feed and the time granularity required would
be dependent on the application. This could be of the order of
<100 milliseconds to meet the synchronisation requirements
in the scenarios presented such as Real-Time race betting
or Real-Time race track information in Formula 1 racing.
Fig. 3 shows a general example of the distribution in an RSS
Channel/Atom Feed of the time-related tags inside the Feed
B. MPEG2 Timing
In the three possible scenarios the TV channel sent via the
network from the TV streamer to the STB will typically be
in MPEG2 Transport Stream (TS) format . In MPEG2
TS there are a number of different timestamps embedded in
the packets: Presentation Time Stamp (PTS), Decoding Time
Stamp (DTS) and Program Clock Reference (PCR). DTS and
PTS perform the inter-media synchronisation between audio-
video while PCR is used by the decoder to perform Clock
Recovery functions . The PCR (27MHz) is located in the
Fig. 4. MPEG2 TS Header with all time stamps ﬁelds and related fags
Fig. 5. MPEG2 PLL High Level Diagram where PCR and STC are compared
to assure that encoder and decoder’s frequency is 27MHz 
Transport Stream Adaptation Field and DTS and PTS (90KHz)
are located at the Transport Packet Payload inside the Package
Elementary Stream (PES) Packet Header. Fig. 4 shows where
the time stamps are located in a TS packet. Not all TS packets
have the Adaptation Field and PTS and DTS are not always
present. The PTS DTS ﬂags indicate its presence in the PES
Packet Header. If DTS is missing then DTS=PTS. DTS can
never be found on its own.
It is important that encoder and decoder clock run at the
same frequency, 27MHz in MPEG2. The Phase-Locked Loop
(PLL) is responsible for the clock recovery functions at the
decoder to guarantee that both run at the same frequency. PLL
compares the encoder’s PCR and the decoder’s System Time
Clock (STC) and applies clock recovery functions to assure
that both run at 27MHz. Fig. 5 shows an MPEG2 PLL High
Level Diagram where decoder compares PCR with STC and
applies the clock recovery functions .
The STB decoder takes care of the synchronisation be-
tween DTS, PTS and PCR. In our system of video/text
synchronisation has to synchronise the PTS from the video
with the <pubDate>/<updated>time stamp from the RSS
channel/Atom Feed. This is done either client or server-side
as discussed in section 5.
There are three pieces of hardware involved: the TV
Streamer, the Feed Server and the STB. Both, the video stream
and RSS data are stored on the TV/RSS server. Both must
be time stamped from Coordinated Universal Time (UTC)
sources, such as a NTP server. Synchronising the STB to a
NTP Server would also facilitate a mechanism to indicate
how old the information is as well as facilitating clock skew
removal via an alternative mechanism of clock recovery .
Fig. 6. Ralationship time stamps between RTP and RTCP protocols and the
Video PES Header 
C. Supported Protocols
Real-Time Protocol (RTP) is used to transport the MPEG2
TS over User Datagram Protocol (UDP) and Real-Time Con-
trol Protocol (RTCP) is used to report back on Quality of
Service (QoS) of the communications.
UDP  is a transaction oriented transport protocol which
does not guarantee the delivery of streams and does not
guarantee duplicate protection. Therefore, it assumes no re-
transmission of lost packets which is suited to video Real-Time
characteristics. Sending MPEG2 TS on RTP packets adds extra
features broadly explained in  with the only trade-off being
a small increase of the payload.
There is a speciﬁc RTP payload to carry MPEG1/MPEG2
video that provides extra information about the video content.
Its functions are to increase compatibility between MPEG2
systems and to ensure that MPEG2 media streams are com-
patible with any media streams encapsulated in RTP .
This RTP format to send MPEG1/MPEG2 video streams
relates the time stamps of the video/audio PTS with the one
in RTP. RTCP associates RTP time stamps to the NTP based
clock time .
When properly implemented, NTP can deliver millisecond
level synchronisation. In Fig. 6 we can see the relationship
between all time stamps in the protocols involved in the RTP,
RTCP and PES headers.
In the case of the Feeds they are typically delivered using
Hypertext Transfer Protocol (HTTP) over Transmission Con-
trol Protocol (TCP). TCP is a connection oriented transport
protocol which provides retransmission of lost packets and
congestion control , therefore it is the appropriate protocol
for the normal delivery of Feeds where the content is essential
and its delivery must be guaranteed to the client. In the
example of sending Real-Time betting information via a Feed
we must guarantee the client gets all the information available.
If retransmission of a packet is required, this impacts on Real-
Time delivery and it must be compensated by the system.
If the Feed information is not critical it could then be
delivered by UDP and RTP using the same protocols as
MPEG2 TS. Fig. 7 shows the protocol stack with the two
possible groups of protocols to deliver Feeds in the transport
and application layer.
Fig. 7. Differences between different transport protocols (UDP/TCP) and
application protocols (HTTP/RTP/RTCP)
V. IM PL EM EN TATION
A. STB User Interface
A web interface allows the user to select different multi-
media options making the system fully personalised. There
could be, for example, TV channels, Radio Channels or
Feeds. In many cases, these multimedia choices will not be
logically or temporally related and therefore synchronisation
is not required. As an example, a ﬁlm and a Feed about a
football match do not require synchronising because the only
purpose is to keep the viewer informed of a sports event whilst
watching a ﬁlm, therefore a delay of a few of seconds won’t
make any difference to the viewer.
In our prototype the user selects video and RSS Feeds to be
aligned and can select where to perform the synchronisation,
server-side or client-side (STB) alignment.
B. STB Alignment Option
In this scenario TV and RSS are streamed independently
to the STB so every client requires the complexity to syn-
chronise the TV channel with the selected RSS Feed. Server
scalability is therefore less of an issue as multicast-support
is more feasible and bandwidth requirements at server are
lessened. However, the STB must meet the Real-Time inte-
gration processing requirements. The STB buffers and time
aligns (synchronises) the channels for display. A convenient
mechanism to implement this could be through an extension
of the existing subtitles facility as described brieﬂy below.
C. Server-side Option
The other method whereby the media streams are inte-
grated/aligned on the server greatly reduces STB complexity
which is a key advantage but presents scalability challenges.
This relates to both server bandwidth required to service
individual client needs due to reduced capacity for multicasting
and complexity on the server to process requests. On client-
side, it also requires less bandwidth which, though of relevance
here, is more pertinent when other synchronisation scenarios
such as Multiple Video streams and/or Audio Substitution are
Fig. 8. High Level Diagram of the prototype. The external elements: the
database Server and Sky RSS Feed are not connected to the NTP Server but
the system’s Web, TV and RSS Server are connected along with the client
D. RSS Feed
The prototype had two possible type sources for the RSS
Feed, ﬁrst a choice from Sky server, with no time stamps,
and secondly an RSS Feed stored in the same server as the
TV streamer was located, with time stamps. The idea was to
develop a system that could use an internal and external RSS
Feeds to show the full potential of the idea.
As previously mentioned the time stamps in RSS are not
compulsory and in the internal RSS feed we included them in
the ﬁle items with milliseconds level granularity.
Using a real Sky RSS Feed has the drawback of not having
the <pubDate>tags in the items implemented. The system
works around this inconvenience by adding the time stamps
to the items to synchronise the display in the video.
The Database server is external to the system and it stores
information about the client TV requirements. The Sky RSS
Feed Server is also external with no possibility of modifying
the content of the RSS Feed. Fig. 8 shows the full high level
diagram with all the system’s components and its connection
with the NTP Server.
The two approaches of server-side, client-side synchronisa-
tion were developed with the media player VideoLan (VLC)
. It is used to stream the MPEG2 TS stream in the server
and is used in the client-side as a decoder.
The embedded subtitle mechanism within VLC provides
a suitable mechanism to integrate the RSS Feed within the
TV stream. The level of alignment required for good subtitle
implementation is much less than 1 sec and therefore ﬁtted our
requirements. Essentially our implementation converts RSS
Fig. 9. Transformation time stamps RSS Feed into subtitle SRT ﬁle before
displaying video/audio synchronised with the subtitles in the client’s decoder
Feeds into subtitles format and embeds this within a TV
In the client-side implementation the RSS Feed was trans-
formed into the subtitles ﬁle SubRip format which allows html
text formatting with millisecond timing accuracy. In the server-
side alignment the subtitles ﬁle was embedded to the MPEG2
video and streamed to the client. In the client-side alignment
the RSS ﬁle is read from the server and transformed into
subtitles in the client. Fig. 9 shows how the RSS Feed format is
transformed into a subtitle in SubRip format before displaying
the video at the client.
VI. CU RR EN T STATUS AND FUTURE WOR K
Both the STB and the Server options have been imple-
mented and are currently being tested. The system has been
client-side. The user-interfce has been developed with Java
Server Pages (JSP). VLC is the media player chosen as a TV
streamer at the server-side, and as a decoder at the client-side.
The prototype only accepts RSS standard as a Feed. A new
model that also uses Atom has to be implemented. In the
prototype we will evaluate UDP and TCP based RSS/Atom
delivery mechanism and the effects it has on the multimedia
To date our research has focused on MPEG2 and we plan
to extend this to MPEG4 in developing a mosaic of multiple
The different format companies use to deliver Internet
Radio, in context of the TV/Radio scenario, will be another
consideration. Internet Radio uses different systems such
as MPEG Audio Layer 3 (MP3), Ogg Vorbis, Windows
Multimedia Audio (WMA), RealAudio and High-Efﬁciency
Advanced Audio Coding (HE-accPlus) among others. It
is also important to analyse the different protocols used
to deliver these formats. Some use RTP/TCP and others
In this abstract, we present an overview of a research
plan to synchronise multiple media streams within an IPTV
environment. We outline a number of scenarios and detail one
in particular whereby a TV and RSS Feed are synchronised.
We summarise the implementation and the key technical
issues involved including scalability design trade-off. In our
ongoing work, we are examining these trade-offs in detail
and extending the implementation scope to the other scenarios
Thanks to SolanoTech, our industrial partner for support
Thanks to the Irish Research Council for Science, Engineer-
ing and Technology (IRCSET) for the Enterprise Partnership
REF ER EN CE S
 J. Maisonneuve, M. Deschanel, J. Heiles, H. Liu, R. Sharpe, Y. Wu.
An overview of IPTV standards Development, in: IEEE Transactions on
Broadcasting vol.55, no3, June 2009 pp.315-328.
 F. Boronat, J. Lloret, M. Garcia. Multimedia group and inter-stream
synchronisation techniques: A comparative study, Information Systems
34, 2009, pp 108-131.
 Internet Engineering Task Force. RFC1129 Internet Time Synchronisa-
tion: The Network Time Protocol. October 1989.
 Berkam Center. RSS 2.0 at Harvard Law (2003). Available at:
http://cyber.law.harvard.edu/rss/rss.html [Accessed: 02 December 2009].
 Internet Engineering Task Force. RFC4287. The Atom Syndication For-
mat. December 2005.
 H. Wittenbrink. RSS and Atom, Pack Publishing Ltd, Birminhgam. 2005.
 Internet engineering Task Force RFC0822. Standard for the format of
ARPA Network Text Messages. August 1982.
 Internet Engineering Task Force. RFC 3339 Date and Time on the
Internet: Timestamps. July 2002.
 ISO 8601. Data Elements and Interchange Formats Information Inter-
change Representation of Dates and Times 2000.
 ISO/IEC 13818-1. Information Technology Generic coding of moving
pictures and associated audio: Systems Recommendation H.222 (2000E).
 H. Melvin, L. Murphy. Synchronisation of Internet Multimedia Streams:
Some Issues and Solutions, Work In Progress Proceedings of the Real
Time Systems Symposium, RTSS 2005, Miami, Dec. 2005.
 Internet Engineering Task Force. RFC768 User Datagram Protocol.
 J. Goldberg. RTP/UDP/MPEG2 TS as a means of transmission for
IPTV Streams. Telecommunication Standardization Sector. Focus Group
on IPTV. IPTV-ID-0087. July 2006.
 Internet Engineering Task Force. RFC2250 RTP Payload Format for
MPEG1/MPEG2 Video. January 1998.
 Internet Engineering Task Force. RFC793 Transmission Control Proto-
col. September 1981.
 VideoLAN VLC media player. Client Software Open Source Code.
www.videolan.org [Accessed: 06 April 2010].