Development of an FPGA-Based Data Acquisition Module for Small Animal PET

Abstract

We report on the design of a data acquisition (DAQ) module for a small animal PET camera developed at our institutes. During the design an important guideline was to develop a system which is built up from strictly identical DAQ modules, and which has no built-in hardware limitation on the maximum number of modules. The developed DAQ module comprises of an LSO scintillator crystal block, a position sensitive PMT, analog signal conditioning circuits, a digitizer, an field programmable gate array (FPGA) for digital signal processing, and a communication module through which the collected data are sent to a cluster of computers for postprocessing and storage. Instead of implementing hardware coincidence detection between the modules, we attach a precise time stamp to each event in our design, and the coincidence is determined by the data collecting computers during postprocessing. The digital CFD algorithm implemented in the FPGA gives a time resolution of 2 ns FWHM for real detector signals

Full text

Page 1

Development of an FPGA-based Data Acquisition
module for Small Animal PET
Gy. Hegyesi, J. Imrek, G. Kalinka, J. Moln´ ar, D. Nov´ ak, J. V´ egh
Institute of Nuclear Research of the Hungarian Academy of Sciences, Debrecen, Hungary
L. Balkay, M. Emri, G. Moln´ ar, L. Tr´ on
University Medical School of Debrecen, PET Center, University of Debrecen, Debrecen, Hungary
I. Bagam´ ery, T. B¨ ukki, S. R´ ozsa
MEDISO Ltd, Budapest, Hungary
Zs. Szab´ o
Institute of Experimental Physics, University of Debrecen, Debrecen, Hungary
A. Kerek
Royal Institute of Technology, Stockholm, Sweden
Abstract—We report on the design of a DAQ module for a
small animal PET camera developed at our institutes. During the
design an important guideline was to develop a system which is
built up from strictly identical DAQ modules, and which has no
built-in hardware limitation on the maximum number of modules.
The developed DAQ module comprises of an LSO scintillator
crystal block, a position sensitive PMT, analog signal conditioning
circuits, a digitizer, an FPGA for digital signal processing and
a communication module through which the collected data is
sent to a cluster of computers for post processing and storage.
Instead of implementing hardware coincidence detection between
the modules we attach a precise time-stamp to each event in our
design, and the coincidence is determined by the data collecting
computers during the post processing. The digital CFD algorithm
implemented in the FPGA gives a time resolution of 2 to 3 ns
FWHM for real detector signals.
I. INTRODUCTION
Laboratory experiments on small animals are one of the
most suitable ways of testing the newly developed PET radio-
pharmaceuticals in vivo. Conventional PET cameras, however,
are not suitable for small animal experiments as their 3D
resolution (about 4 to 5 mm) is much worse than it is minimally
required (about 2 mm). To meet such demands special small
PET cameras (miniPETs) are developed that can be used for
investigating small animals, like mice or rats.
Our contribution will show the design of the detector mod-
ules developed at our institutes, which is used in such miniPET
cameras, and will describe the initial test results of the system.
II. THE DESIGN
During the design of the data acquisition (DAQ) module we
had scalability in mind. We wanted to design a module that
can be multiplied without modifications to form a PET ring,
and which has no built-in hardware limitation on the maximum
Fig. 1. Picture of the detector block.
number of modules. Our DAQ module consists of a scintillator
crystal block, a position sensitive PMT (PSPMT), analog signal
conditioning circuits, a digitizer, a digital signal processor and
a communication module, through which it sends the collected
data to a cluster of computers for post processing and storage.
In contrast to most of the small animal PET systems [1]
[2] we chose not to implement hardware coincidence detection
between the modules. As a PET camera obviously can not
function without determining coincidences between incoming
gamma photons, we attach a precise time stamp to each and
every event and the coincidence is determined during post pro-
cessing by the data collecting computers. This method increases
0-7803-8701-5/04/$20.00 (C) 2004 IEEE

Page 2

pulse recognition
time stamp (digital CFD)
energy calculation
base line restoration
local clock
F
I
F
O
FADC
FPGA
BallyRiff module
N
I
C
PIC
communication module
HV
PSPMT
X+
X−
Y+
Y−
hw control (RS232)
50 MHz CLK
HW_ENABLE
10 MBit Ethernet
DAQ module
PSPMT
fast preamp
scintillator block
analog frontend
Fig. 2.Block Diagram of a DAQ module.
the amount of data that needs to be transferred between the
detector blocks and the data collecting computers. However,
it reduces the complexity of the digital hardware design and
provides enormous flexibility for data analysis: since all data
are transferred from the detector blocks to the data collecting
computers different selection criteria can be applied to the
collected data - even off-line, long after the data acquisition.
When not using hardware coincidence detection only a
limited number of global signal need to be distributed to the
DAQ modules. A 50 MHz clock drives the local clocks on the
DAQ modules and serves as a time base for the time-stamp
generation, while an enable signal ensures the synchronization
between the local times and starts/stops data collection. A slow
control network sets the different parameters of the modules
(e.g. the value of the high voltage). Fig. 1 shows a picture
of an assembled DAQ module, while Fig. 2 shows its block
diagram.
III. THE LSO CRYSTAL ARRAY AND ITS OPTICAL
COUPLING
In small animal PET cameras gamma photons are usually
detected by a block of scintillating material [3]. In our case the
scintillator block is constructed of an 8 x 8 array of individual
LSO crystals. The size of an individual crystal is 2 mm x 2 mm
x 10 mm, cut to appropriate size by OPTILAB Ltd [4].
All surfaces of the crystals are mechanically polished, as
initial tests showed that this surface treatment gives the best
light collection performance compared to other ones [5]. A
180 µm reflective (so-called LUMIRROR [6]) film is placed
between and around the crystal elements in order to optically
isolate them and to increase the light collection efficiency.
This film is highly non-transparent and diffusively reflective,
which is, from the aspects of light collection, found to be better
than other foils with a mirror-like reflection [7]. The whole
crystal array is also wrapped two-fold into LUMIRROR film
to further improve its mechanical stability (see Fig. 3).
When developing a detector module for a miniPET, good
optical coupling between the scintillator block and the PSPMT
is an important issue. During the development phase we used a
Fig. 3.Individual scintillator crystals and the assembled scintillator block.
soft and transparent silicone film with a thickness of 0.2 mm to
achieve reliable but temporary coupling. However, as the initial
tests are over, we intend to replace the soft layer with a hard
optical glue to improve the mechanical fixture of the crystal
elements and to assure the long term stability of their original
(calibrated) position.
During the test phase we found that the detector blocks
have good energy resolution, which can be further improved by
calibrating the individual crystals (see Fig. 4). We are convinced
that the main reasons of the non-uniformity is the varying
efficiency caused by the silicone film, but PSPMT tubes are
also known to have some non-uniformity in energy over their
sensitive surface [8].
IV. POSITION SENSITIVE DETECTOR AND ANALOG SIGNAL
CONDITIONING
A HAMAMATSU PSPMT is attached to each crystal block
for the amplification and the read-out of the light signals. The
R8520-00-C12 PSPMT [9] with cross-plate anode was selected
0-7803-8701-5/04/$20.00 (C) 2004 IEEE

Page 3

0 5001000 1500 20002500300035004000
0
25
50
75
100
Energy (channel number)
Number of events
(Selected energy spectra)
1000
2000
3000
4000
Number of events (Full energy spectra)
Fig. 4.
crystals of it (colored curves).
Energy spectrum of a crystal block (black curve) and four individual
because of its compactness and because of its geometry per-
fectly matches our scintillator block (size, rectangular window
geometry). The PSPMT produces two pairs of signals (X+,
X−and Y+, Y−, so called corner signals), which are related
to the X and Y coordinates of the impact point of the incoming
gamma photon on the surface of the PSPMT. These signals also
carry energy and time information.
The analog electronic board that we built includes a compact
high-voltage supply, which can be controlled via serial line, a
voltage divider that drives the dynodes of the PSPMT, and the
analog circuits that do the necessary signal conditioning of the
four signals.
V. DIGITAL READOUT ELECTRONICS
Digitizer boards manufactured by NALLATECH [10] are
used to digitize and process the corner signals of the PSPMTs.
There are four 125 MHz ADCs on the BallyRiff digitizer board
along with one XILINX Virtex FPGA (Field Programmable
Gate Array). The corner signals of the PSPMT are connected
to the free-running ADCs and their digital output is fed into
the FPGA.
The FPGA performs numerous digital signal processing
tasks. For each incoming photon its energy, impact position
and time of arrival has to be determined [11].
The energy of the incoming photon can be calculated by
summing up the X+and X−or the Y+and Y−signals. A
cut on the minimal energy of the detected gammas is applied to
filter out those events that do not correspond to the annihilation
of a positron-electron pair.
The X and Y coordinates of the hit can be determined from
the corner signals using the Anger formula:
X =X+− X−
X++ X−,Y =Y+− Y−
Y++ Y−
(1)
A color coded flood-field image is shown in Fig. 5. The
calculated positions of the crystal needles on the surface of the
X position
Y position
50 100150200250
50
100
150
200
250
50
100
150
200
250
300
Fig. 5.
crystals. This image was created using a68Ge source with an activity of
about 1 millicurie, the data collection took approximately 15 minutes. The
different peaks corresponding to the individual scintillator crystals are clearly
distinguishable. The colored circles show those crystals the energy spectrum
of which is plotted in Fig. 4.
Position resolution of the detector blocks and alignment of individual
PSPMT line up very well. This makes it easy to separate the
regions that correspond to the individual crystals, thus enabling
event size reduction (see Section VII below).
As described above, no hardware coincidence detection is
used, therefore a high resolution time stamp has to be attached
to each event. Although the output signal from the last dynode
of the PSPMT has excellent timing properties and could be
used for timing [12], the use of this signal would unnecessarily
complicate the design. According to our measurements suffi-
ciently good time resolution can be achieved by extracting the
timing information from the four position-related signals. We
implemented a digital CFD algorithm in the FPGA [13] which
gives a time resolution of 2 to 3 ns FWHM for real detector
signals.
The digital signal processing algorithms (signal recognition,
peak-finding, position calculation, time-stamp generation) are
written in VHSIC Hardware Description Language (VHDL).
VI. COMMUNICATION MODULE
The energies, the X and Y coordinates and the time stamps
are stored in an output buffer (a FIFO) inside the FPGA, which
is read by the communication module. The communication
module is a microcontroller based intelligent device with a
10 MBit Ethernet interface. The module is responsible for the
communication with the data collecting computers (servers) and
the master computer (see Section VII below).
Ethernet based IP technology was chosen as the carrier
of the collected data, because of its numerous advantages: it
is sufficiently fast, well tested, scalable, easy to implement
and debug, cheap, and there are many standard hardware and
software components available. User Datagram Protocol (UDP)
0-7803-8701-5/04/$20.00 (C) 2004 IEEE

Page 4

is used on top of IP with a proprietary protocol developed
by us. While Transmission Control Protocol (TCP) would be
more reliable than UDP, it is more complicated and its more
demanding resource usage (CPU, memory, software) is not
justified in our case. Accidental and rare loss of a UDP packet is
acceptable in favor of speed and simplicity. However, to ensure
correct data analysis a sequence number is assigned to each
UDP packet, thus the number of lost packets and lost events
can be tracked.
The collected data is transferred from the FPGA to the
communication module on an 8 bit wide data bus, where they
are wrapped into UDP packets and sent to the appropriate
data collecting computer. The module can keep a minimum
time gap between consequent UDP packages - thus limiting
the output event rate and output bandwidth usage - to reduce
the load on the communication lines and on the data collecting
computers. It can also limit the maximum time gap between
two UDP packages to help in the discovery of detector failures
and the misbehaviors. It also has features to support debugging
and testing of communication links (e.g. event generators,
configurable packet sizes).
VII. DATA COLLECTION
The full data collecting and processing system consists of
the detector blocks (the clients), the data collecting computers
(the servers) and a dedicated computer which manages the data
collection (the master).
After power on the master discovers the various clients and
servers on the network using a Dynamic Host Configuration
Protocol (DHCP) like protocol developed by us. A so called
“communication channel” is built up between each client or
server and the master. The control channel is used to control the
various entities: the master can start and stop the data collection,
configure the clients and the servers for various data collection
modes (e.g. single spectrum collection on each detector for
energy calibration, data collection in list mode with coincidence
detection on the data collecting computers), investigate their
state or reset them. This dynamic discovery and configuration
of devices and the independence of the DAQ modules makes
reconfiguration, expansion or reduction of the PET ring easy,
and the whole design suitable for highly experimental and
research projects. The commercially available Ethernet technol-
ogy (Gigabit Ethernet) and the increasing computational power
of Personal Computers provide enough reserves for growth in
the number of DAQ modules in the system.
The data collection is based on “time slices”. A time slice
is a period of time with a given (configurable) length (see Fig.
6). Using time slices the event size can be reduced, because the
arrival time of the incident gammas is measured and represented
as the elapsed time from the beginning of the actual timeslice,
not as an absolute time.
In the present setup when data is collected for image recon-
struction the events belonging to one time slice are sent from all
DAQ modules to one data collecting computer. This computer
sorts the events by time, and detects possible coincidences
based on the time stamps and the geometry configuration. Only
120
TS_TOGGLE
HW_ENABLE
CLK
TIMESLICE NO
Fig. 6.
length of the time slices can be modified between measurements. The time
slices are numbered, starting with zero at the beginning of a measurement.
The measurement is started and stopped by a global HW ENABLE signal.
The FPGA notifies the communication module on time slice changes using the
TS TOGGLE signal, so the communication module can send further incoming
data to the next data collecting computer.
The time slices. The data collection is based on time slices. The
the coinciding events are forwarded to the master for storing
and visualization.
For each event the time of arrival (measured from the
beginning of the time slice, 4 bytes) and the peak value of
the corner signals (4 x 2 bytes) are sent to the data collecting
computers via the Ethernet link. With this configuration the
achievable sustained event rate is about 10000 events/sec with
no dead time at all. By eliminating the redundancy in the time
stamps and replacing the peak values with a crystal index the
number of transferred events can easily be increased by 2.5-
to 4-fold. The main limiting factors are the capabilities of the
microcontroller, and the 10 MBit Ethernet media. These can
be eliminated by simply replacing the communication module
with a commercially available one that has better capabilities
[14].
VIII. CONCLUSION
We described a DAQ module for a small animal PET
system. To increase the flexibility of the design no hardware
coincidence is used, but a precise time stamp is attached to
each incoming gamma photon for the coincidence detection.
The time stamp is created using a digital CFD algorithm, which
shows sufficiently good time resolution as tested on real-life
signals. The spatial resolution of the module is also proven
to be excellent. The Ethernet based readout system works
according to our expectations, and the used communication
techniques (scalable hardware, dynamic, “plug’n’play” like pro-
tocols) along with the independent nature of the DAQ modules
make expansion and reconfiguration of the data collecting
system easy. Thus the resulting miniPET camera is scalable and
suitable for experimental projects and commercial purposes.
ACKNOWLEDGMENT
The authors would like to thank NKFP (Nemzeti Kutat´ asi
´ es Fejleszt´ esi Alapprogram, National Fund for Research and
Development) for the support of this research project (project
no: NKFP-1A/0010/2002).
REFERENCES
[1] T. K. Lewellen, C. M. Laymon, R. S. Miyaoka, M. Janes, B. Park, K. Lee,
and P. E. Kinahan, “Development of a data acquisition system for the
MiCES small animal PET scanner.”
0-7803-8701-5/04/$20.00 (C) 2004 IEEE

Page 5

[2] J. L. Humm, A. Rosenfeld, and A. D. Guerra, “From PET detectors to
PET scanners,” European Journal of Nuclear and Molecular Imaging,
vol. 30, pp. 1574–1597, Nov. 2003.
[3] S. R. Cherry, Y. Shao, R. W. Silverman, K. Meadors, S. Siegel,
A. Chatziioannou, J. W. Young, W. F. Jones, J. C. Moyers, D. Newport,
A. Boutefnouchet, T. H. Farquhar, M. Anderaco, M. J. Paulus, D. M.
Binkley, R. Nutt, and M. E. Phelps, “MicroPET: A high resoulution PET
scanner for imaging small animals,” IEEE Trans. Nucl. Sci., vol. 44, pp.
1161–1165, June 1997.
[4] OPTILAB Ltd. Sulyok u. 2., H-1031 Budapest, Hungary.
[5] S. R. Cherry, Y. Shao, M. P. Tornai, S. Siegel, A. R. Ricci, and M. E.
Phelps, “Collection of scintillation light from small BGO crystals,” IEEE
Trans. Nucl. Sci., vol. 42, pp. 1058–1063, Aug. 1995.
[6] Toray Industries, Ehime, Japan. [Online]. Available: http://www.toray.
com/
[7] 3MTMRadiant Ligth Film. [Online]. Available: http://www.3m.com/
lightmanagement
[8] N. Belcaria, M. Camardaa, A. D. Guerraa, D. Herberta, A. Mottaa,
A. Vaianoa, G. D. Domenicob, and G. Zavattinib, “Detector development
for a novel positron emission mammography scanner based on YAP:Ce
crystals,” Nuclear Instruments and Methods in Physics Research A, vol.
525, pp. 258–262, 2004.
[9] Hamamatsu Photonics K.K., Electron Tube Center. [Online]. Available:
http://www.hamamatsu.com
[10] Nallatech. BallyRiff:4 channel,
capture DIMETMmodule. [Online]. Available: http://www.nallatech.
com/solutions/products/embedded systems/dime/modules/ballyriff/
[11] M. S. Musrock, J. W. Young, J. C. Moyers, J. E. Breeding, M. E. Casey,
J. M. Rochelle, D. M. Binkley, and B. K. Swann, “Performance charac-
teristics of a new generation of processing circuits for PET applications,”
IEEE Trans. Nucl. Sci., pp. 974–978, Aug. 2003.
[12] B. K. Swann, J. M. Rochelle, D. M. Binkley, B. S. Puckett, B. J. Blalock,
S. C. Terry, J. C. Moyers, J. W. Young, M. E. Casey, M. S. Musrock,
and J. E. Breeding, “A custom mixed signal CMOS integrated circuit for
high performance PET tomograph front-end applications,” IEEE Trans.
Nucl. Sci., vol. 50, pp. 909–914, Aug. 2003.
[13] M. Streun, G. Brandenburg, H. Larue, E. Zimmermann, K. Ziemons, and
H. Halling, “Coincidence detection by digital processing of free-running
sampled pulses,” NIM A, vol. 487, pp. 530–534, 2002.
[14] DIMM-PC R ?/520-IE.
products/pdproductdetail.cfm?keyProduct=38548&kps=1822&kpc=236
125 MSPS, 12bitanalogue
[Online].Available: http://www.kontron.com/
0-7803-8701-5/04/$20.00 (C) 2004 IEEE