Content uploaded by Emanuele Guidotti
Author content
All content in this area was uploaded by Emanuele Guidotti on Nov 10, 2019
Content may be subject to copyright.
RBLDATALICENSE:
R INTERFACE TO BLOOMBERG DATA LICENSE
A PREPRINT
Emanuele Guidotti∗
Department of Finance
University of Neuchâtel
Switzerland
ABS TRAC T
As the technology infrastructure increases in sophistication, Bloomberg released the Data License
service, the platform that delivers Bloomberg’s datasets. For the first time, users can directly program
against Bloomberg’s comprehensive historical data, servicing the growing needs of data science. The
RblDataLicense
package aims at providing a user-friendly R interface to the Data License service.
As a prerequisite, a valid Data License from Bloomberg is needed. This software and its author are in
no way affiliated, endorsed, or approved by "Bloomberg" or any of its affiliates. "Bloomberg" is a
registered trademark.
Keywords Data acquisition, Bloomberg, R
1 Introduction
Bloomberg is a major global provider of financial news and information, delivering high quality data to more than
300.000 trading desks. The company is very well known for the "Bloomberg Terminal", a computer software system
that enables professionals in the financial service sector and other industries to monitor and analyze real-time financial
market data and place trades on the electronic trading platform. The
Rblpapi
package by Armstrong, Eddelbuettel,
and Laing (2018) provides a direct interface between R and the Bloomberg
Terminal
via the official C++ API. A
valid Bloomberg license and installation is required and the package can only be used on the same desktop where the
terminal is installed. Any data fetched using the API must remain on the same terminal. The user must be logged into
the terminal to retrieve data, or the Rblpapi will fail to connect to the Bloomberg backend.
All these limitations make the terminal unattractive to researchers and professionals wishing to build a data warehouse,
automating tasks, or dealing with cloud computing. As the technology infrastructure increases in sophistication,
Bloomberg released the Data License service, the platform that delivers Bloomberg’s datasets. For the first time, users
can directly program against Bloomberg’s comprehensive historical data, servicing the growing needs of data science.
The RblDataLicense package provides a user-friendly R interface to Bloomberg Data License.
2 Prerequisites
As a prerequisite, a valid Data License from Bloomberg is needed together with the corresponding SFTP credentials
and whitelisting of the IP from which accessing the service. The credentials consist of the username and password
needed to authenticate requests. As an additional security measure, all requests are blocked by default and are only
accepted if coming from a whitelisted IP address. The user needs to inform Bloomberg of the IP addresses from which
accessing the service. Only correctly authenticated requests from whitelisted IP will work.
The Bloomberg Data License Service is billed on a pay-per-use model, based on the amount of instruments and data
categories requested. The RblDataLicense package implements several checks to protect the user against unintended
∗
emanuele.guidotti@unine.ch
RblDataLicense:
R Interface to Bloomberg Data License A PREPRINT
programming errors and to help keeping costs under control. In any case, to avoid incurring unexpected expenses, the
user should carefully read the Bloomberg contract before accessing the Data License Service with this or any other
software.
RblDataLicense
is built on top of
RCurl
by Lang and the CRAN team (2019), a wrapper for libcurl. The package
establishes SFTP connections to Bloomberg Data License. On some Linux systems, this may not work out of the box,
as libcurl does not natively support SFTP. In that case, the user needs to compile curl with SFTP support first.
3 Software and usage
As the very first step, the credentials received from Bloomberg to access the SFTP server must be provided to the
RblConnect
function. The function checks for the SFTP connection to be successful and stores the credentials in the
current R session. Then, all the other functions of the package will be unlocked and ready to use.
# These are dummy credentials.
# Replace with the credentials received from Bloomberg
RblConnect(user = 'dl000000', pw = '0000000000000000')
The user should now prepare a request file according to the Bloomberg’s documentation, containing the instructions for
the data to retrieve. This file has to be uploaded to the SFTP server the user owning a valid License has been granted
access to. Upon receipt, Bloomberg generates the corresponding response file containing the data. This process can
take up to several minutes, depending on the kind of request. Once available, the user can download the response file
from the SFTP server and parse it to import the data in R. The
RblDataLicense
package implements the following
routine and functions to simplify the process.
Building the request file:
RblRequestBuilder()
Uploading the request file:
RblUpload()
Downloading the response file:
RblDownload()
Parsing the response file:
RblParse()
RblQuery()
3.1 Building the request file
The
RblRequestBuilder
function generates the request file to query Bloomberg according to the Data License
documentation. The user must provide the following arguments:
•
header
: named vector containing at least the parameters
–
FIRMNAME
: the DL user, can be retrieved by calling
RblUser()
–
PROGRAMNAME
: the Bloomberg program to run, usually getdata or gethistory
•
identifiers
: the ID(s) of the instrument(s) to retrieve
•
fields
: the field(s) of the instrument(s) to request
The following code snippet builds a minimal request file to download the daily closing prices of EURO STOXX Index
from 2005-01-01 to 2015-12-31. Refer to Bloomberg Documentation for details.
# RblRequestBuilder(header, fields, identifiers, overrides = c())
RblRequest <-
RblRequestBuilder(
header = c(
FIRMNAME = RblUser(),
PROGRAMNAME = 'gethistory',
DATERANGE = '20050101|20151231'),
fields = c('PX_LAST'),
identifiers = c('SXXE Index')
)
3.2 Uploading the request file
The
RblUpload
function uploads a request file to the SFTP server. The request file can be user-defined following
the Bloomberg Data License documentation or generated with the
RblRequestBuilder
function. A response file
2
RblDataLicense:
R Interface to Bloomberg Data License A PREPRINT
containing the data will be generated by Bloomberg and made available at the SFTP server. The following code snippet
uploads the
RblRequest
generated in the previous example.
# RblUpload(RblRequest, filename = format(Sys.time(),"%m%d%H%M%S"))
req <- RblUpload(RblRequest)
Note that the
RblRequest
argument is the character string representing the request itself, not the path to the file
containing the request. In order to use a local file, the user should import its content in R as a chacracter string and pass
the string in the
RblRequest
argument. The
RblUpload
function returns a list containing:
•the name of the request file that has been uploaded to Bloomberg
•the name of the corresponding response file containing the data
3.3 Downloading the response file
The
RblDownload
function downloads a generic file from Bloomberg. The user can run
RblFiles()
to list the files
available at Bloomberg. The following code snippet downloads the response file corresponding to the request uploaded
in the previous example.
# RblDownload(file, frequency = 60, timeout = 3600)
out <- RblDownload(req$out)
The response file may take several minutes before being available at Bloomberg. If the file is not available yet, the
function waits until the response file is there or the timeout is reached.
RblDownload
returns the path to the local copy
of the remote file.
3.4 Parsing the response file
The
RblParse
function provides a simple parser to import local Bloomberg response files in a structured R format.
Depending on the
PROGRAMNAME
used by the file, data are imported as
data.frame
or
xts
objects, eventually loaded
to
env
if
auto.assign=TRUE
to mimic the behaviour of the
quantmod
package by Ryan and Ulrich (2018). The
following code snippet imports in R the data downloaded in the previous example.
# RblParse(file, auto.assign = FALSE, env = parent.frame())
data <- RblParse(out)
3.5 Putting it all together
The
RblQuery
function provides a high level interface to Bloomberg Data License, automating the routine explained
above: building the request file - uploading the request file - downloading the response file - parsing the response file.
The final result produced by running all the examples shown before can be achieved with the following single command:
# Run RblConnect first
data <- RblQuery(identifiers = 'SXXE Index',
fields = 'PX_LAST',
from = '2005-01-01',
to = '2015-12-31')
The function is supposed to work out-of-the-box but at the same time it provides an extensive set of arguments to
customize requests and achieve a great degree of flexibility.
Main arguments:
•
identifiers
: vector of Bloomberg identifiers.
E.g.
c("SXXE Index", "SX5E Index")
•
fields
: vector of Bloomberg fields.
E.g.
c("PX_LAST", "PX_CLOSE", "PX_OPEN", "PX_HIGH", "PX_LOW")
•
from
: date or string (format
YYYY-MM-DD
). Start time for the ’gethistory’ request. If not provided, a ’getdata’
request will be made.
3
RblDataLicense:
R Interface to Bloomberg Data License A PREPRINT
•
to
: date or string (format
YYYY-MM-DD
). End time for the ’gethistory’ request. Ignored if
from
is not provided
Safety arguments:
•
category
: vector of Data License categories to enable. Each DL category is billed separately. The user
should check the DL license carefully before enabling additional categories.
E.g.
c("SECMASTER", "PRICING", "FUNDAMENTALS")
.
•
limit
: integer. Prevent requesting data for more than this amout of identifiers. This is done to help the user
keeping the cost under control. Data License is billing based on the amout of instruments requested.
•
split
: integer. Maximum number of identifiers to process at once. Requests are split to avoid memory leaks.
Flexibility arguments:
•
add_headers
: named vector of Bloomberg additional headers.
E.g.
c(PROGRAMFLAG = "oneshot")
•
overrides
: named vector of Bloomberg overrides.
E.g.
c("END_DT" = "20100101")
•
filename
: string. Name assigned to the remote file(s).
Download arguments:
•
frequency
: the polling frequency to check if the response file is available at Bloomberg.
•
timeout
: the timeout in seconds.
4 Summary
Bloomberg is a major global provider of financial news and information, delivering high quality data to more than
300.000 trading desks. As the technology infrastructure increases in sophistication Bloomberg released the Data
License service, the platform that delivers Bloomberg’s datasets. For the first time, users can directly program against
Bloomberg’s comprehensive historical data, servicing the growing needs of data science. The
RblDataLicense
package
provides a user-friendly R interface to Bloomberg
Data License
, implementing the following routine and functions to
simplify the data acquisition process.
Building the request file:
RblRequestBuilder()
Uploading the request file:
RblUpload()
Downloading the response file:
RblDownload()
Parsing the response file:
RblParse()
RblQuery()
5 Acknowledgements
The
RblDataLicense
was developed at Algo Finance Sagl, software house start-up developing financial algorithms for
the asset management industry. The Swiss company is gratefully acknowledged for open-sourcing the software.
6 Legal
All code of the
RblDataLicense
is released under the GNU GPL-3. This software and its author are in no way affiliated,
endorsed, or approved by "Bloomberg" or any of its affiliates. "Bloomberg" is a registered trademark. All trademarks
and registered trademarks are the property of their respective owners.
References
Whit Armstrong, Dirk Eddelbuettel, and John Laing. Rblpapi: R Interface to ’Bloomberg’, 2018. URL
https:
//CRAN.R-project.org/package=Rblpapi
. R package version 0.3.8.
Duncan Temple Lang and the CRAN team. RCurl: General Network (HTTP/FTP/...) Client Interface for R, 2019. URL
https://CRAN.R-project.org/package=RCurl
. R package version 1.95-4.12.
Jeffrey A. Ryan and Joshua M. Ulrich. quantmod: Quantitative Financial Modelling Framework, 2018. URL
https://CRAN.R-project.org/package=quantmod
. R package version 0.4-13.
4
