ArticlePDF Available

Abstract and Figures

This paper introduces the concept of QR images, an automatic method to embed QR codes into color images with bounded probability of detection error. These embeddings are compatible with standard decoding applications and can be applied to any color image with full area coverage. The QR information bits are encoded into the luminance values of the image, taking advantage of the immunity of QR readers against local luminance disturbances. To mitigate the visual distortion of the QR image, the algorithm utilizes halftoning masks for the selection of modified pixels and nonlinear programming techniques to locally optimize luminance levels. A tractable model for the probability of error is developed and models of the human visual system are considered in the quality metric used to optimize the luminance levels of the QR image. To minimize the processing time, the optimization techniques proposed to consider the mechanics of a common binarization method and are designed to be amenable for parallel implementations. Experimental results show the graceful degradation of the decoding rate and the perceptual quality as a function the embedding parameters. A visual comparison between the proposed and existing methods is presented.
Content may be subject to copyright.
IEEE TRANSACTIONS ON IMAGE PROCESSING 1
QR Images: Optimized Image Embedding
in QR Codes
Gonzalo J. Garateguy, Member, IEEE, Gonzalo R. Arce, Fellow, IEEE, Daniel L. Lau, Senior Member, IEEE,
Ofelia P. Villarreal, Member, IEEE
Abstract—This paper introduces the concept of QR images,
an automatic method to embed QR codes into color images with
bounded probability of detection error. These embeddings are
compatible with standard decoding applications and can be ap-
plied to any color image with full area coverage. QR information
bits are encoded into the luminance values of the image, taking
advantage of the immunity of QR readers against local luminance
disturbances. To mitigate the visual distortion of the QR image,
the algorithm utilizes halftoning masks for the selection of
modified pixels and nonlinear programming techniques to locally
optimize luminance levels. A tractable model for the probability
of error is developed and models of the human visual system are
considered in the quality metric used to optimize the luminance
levels of the QR image. In order to minimize the processing
time, the optimization techniques proposed take into account the
mechanics of a common binarization method and are designed to
be amenable for parallel implementations. Experimental results
show the graceful degradation of the decoding rate and the
perceptual quality as a function the embedding parameters. A
visual comparison between the proposed and existing methods is
presented.
I. INT RO DUC TI ON
Quick response (QR) codes [1], [2] have rapidly emerged
as a widely used inventory tracking and identification method
in transport, manufacturing, and retail industries [3]. Their
popularity is due to the proliferation of smart phones, capable
of decoding and accessing on line resources as well as its high
storage capacity and speed of decoding. QR codes are used in
a variety of applications, such as accessing websites, download
personal card information, post information to social networks,
initiate phone calls, reproduce videos or open text documents.
This versatility makes them a valuable tool in any industry
that seeks to engage mobile users from printed materials.
Not surprisingly QR codes have been widely adopted in the
marketing and publicity industry thanks to the advantage they
provide in tracking the performance of publicity campaigns.
An important problem of QR codes is its impact on the
aesthetics of publicity designs. The square shapes and limited
color tolerance, severely impairs their integration into billboard
designs or printed materials. This challenge has generated
great interest for algorithms capable of embedding QR codes
into images without loosing decoding robustness. There have
been several efforts to improve the appearance of such em-
beddings [4]–[15] which can be classified in two categories,
methods that modify the luminance or color of image pixels
and methods that replace QR modules.
Copyright (c) 2013 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must be
obtained from the IEEE by sending a request to pubs-permissions@ieee.org
Fig. 1: a) Substitutions of QR modules by a logo, b) QArt code
designed using the method in [4], c) luminance modification
algorithm [5], J= 0.4149 d) proposed algorithm, J= 0.3989.
The distortion metric Jis defined in (16)
The methods presented in [6], [7] base the strategy on
finding the best group of QR modules to substitute by the
image or logo in the QR code. Other methods take advantage
of unused modules in the padding regions [8]–[10] to introduce
the image without affecting the decoding robustness. In [7]
the impact of replacing code modules with image pixels was
studied. The authors concluded that to retain high rate of
decodability, the ratio between image and code area should
be approximately proportional to the correction capacity of
the code. It was also found that superimposing images over
finder or alignment patterns severely decrease the probability
of correct decoding. As a consequence it is common for logos
or images to be located at the center of the code for these
methods as depicted in Fig. 1(a). In general these approaches
do not take advantage of the codeword generation process and
this imposes restrictions in the location of modified modules.
This problem was addressed by recently developed techniques
IEEE TRANSACTIONS ON IMAGE PROCESSING 2
[4], [11], [12] which manipulate the Reed Solomon encoding
procedure to maximize the area coverage without reducing the
correction capacity.
The second category of embedding algorithms is based on
the modification of the pixel’s luminance. The approach in [5]
chooses central pixels of each module to modify its luminance
since this is the area usually sampled by the decoder. This
approach provides an adequate trade off between robustness
and visual distortion but the number of center pixels modified
is in general a large proportion of the module area and creates
undesirable low pass artifacts (see Fig. 1(c)). A method which
implements a similar idea is LogoQ [13]. In [16] a method
to introduce colors into QR codes was proposed with the
goal of increasing its data capacity and a detailed study of
the interference between different color layers was presented.
Two embedding methods which are related to the method
presented here are [14] and [15]. In [14] the author chooses the
modified pixels in the code based on the average luminance of
image pixels and two user defined thresholds that regulate the
reliability of the embedding. QR blocks are still visible but
the visual impact is reduced with respect to [5]. The method
presented in [15] generates a binary embedding by subdividing
each QR module in a set of 3×3pixels and setting the
central pixel to the value of the QR code while the remaining
pixels are selected to minimize the probability of error and
generate a halftone of the image. In this case the probability of
error is evaluated empirically using a large database of random
embeddings.
The main challenge of any embedding method is the fact
that they should be decodable by standard applications. The
embedding of image pixels introduce changes in the lumi-
nance of the code, distorting the binarization thresholds and
thus increasing the probability of detection error. The second
challenge concerns the problem of using the entire area of
the code in which the image or logo is to be embedded. This
cannot be achieved by simply replacing information modules
with the desired image since the number of modules that can
be replaced is at most proportional to the correction capacity
of the code [7]. A good embedding method, should minimize
the number of corrupted modules and use the greatest possible
area while keeping visual fidelity to the original image.
This paper aims at the above cited goals by introducing QR
images, an optimization based approach for embedding color
images into QR codes. The algorithm proposed is based on the
selection of a set of modified pixels using a halftoning mask.
The concentration of pixels and its corresponding luminance
are optimized to minimize a visual distortion metric subject
to a constraint in the probability of error. This algorithm can
be applied to any color image and QR code with full area
coverage and bounded probability of error. A novel contribu-
tion of this paper is the use of halftone masks to distribute the
modified pixels and the introduction of a probabilistic model
to predict the distortion generated by the embedded image.
This allows to control the trade off between image quality
and decoding robustness by setting a few parameters such
as the number of modified pixels at the center of each QR
module and the maximum allowed probability of error. Figure
1(d) depicts the results of embedding a QR code using the
method proposed in this paper which is visually more pleasant
than the embedding in Figure 1(d) and presents a lower visual
distortion according to the metric defined in (16).
Sections II and III presents the structure and decoding
procedures for QR codes since they are central to understand
the possible modification strategies. Section IV presents the
concept of QR images and the luminance manipulation algo-
rithm and Section V describes the probability of error models
as a function of the embedding parameters. Finally Section VI
and Section VIII present the optimization algorithm and the
embedding results of applying the algorithm to several images
and logos.
II. CH ARAC TER IS T IC S O F QR CO DE S
The patterns and structures inside a QR code have well
defined functions which include symbol alignment, sampling
grid determination, and error correction. The information is
encoded in square black and white modules of several pixels
wide. Finder patterns play a central role in the speed and
success of decoding and are located in three corners of the
symbol. Figure 2 shows the three main regions in the QR
symbol structure: function pattern region, encoding region and
the quiet zone which is a guard region located on the outside
of the symbol.
1) Function Pattern Region: Finder and alignment struc-
tures are essential to locate, rotate and align the QR code.
The former ones are designed to have the same ratio of black
and white pixels when intersected by a line at any angle,
allowing to easily detect rotated or inverted codes. Alignment
patterns are used to determine the sampling grids from which
codewords are extracted and they are easily identifiable as
concentric square structures evenly distributed along the code
area.
2) Encoding Region: The code area delimited by finder
patterns is denoted as the encoding region, where data, parity
modules and decoding information is stored. This area is
divided into codewords consisting of blocks of 8QR modules.
Two dimensional shapes of these codewords depend on the
version of the code and are designed to optimize area coverage.
Fig. 2: QR code regions with the location of finder and
alignment patterns highlighted in red
IEEE TRANSACTIONS ON IMAGE PROCESSING 3
3) Data Capacity and Error correction: Different types of
QR codes defined in the standard [1], are identified by their
version and error correction level. The version of the QR code
determines its size and goes from 21×21 modules for version 1
up to 177×177 for version 40. QR codes use Reed Solom code
for error correction and there are 4types of error correction
L, M, Q and Hthat allow to correct up to 7%,15%,20% and
30% of codewords in error respectively.
III. DEC OD ING AL GO RIT HM
After acquiring the image and calculating its luminance
from the RGB components, the decoding process continues
with three basic stages: binarization, detection, and decoding
of the bit stream. In the binarization stage, the gray scale
image captured by the camera is segmented into black and
white pixels. This binary image is used to determine the QR
modules centers and the sampling grid from which the code-
words are extracted. After this process, detected codewords are
corrected using the Reed Solomon algorithm and then decoded
according to its format and version.
A. Threshold Calculation for Binarization
A salient feature of QR codes which plays a central role
in their decoding speed is the use of binarization as the first
step in the decoding process. Binary images are obtained by
thresholding the gray scale image as
IB[i, j] = 1if Yi,j > ti,j
0if Yi,j ti,j (1)
where Yis the image captured by the camera, ti,j is the
threshold assigned to pixel [i, j ]and IB[i, j]is the binary
output. Determining the values of the thresholds ti,j for an
optimal binarization is a challenging task due to the variability
in lighting conditions. The QR code standard does not define
specific binarization methods and these are usually selected
making a trade off between speed and quality. Thresholds can
be calculated globally using all the pixels in the image or lo-
cally considering only reduced windows. Global segmentation
methods as [17], [18] have been tried and proved ineffective
since they require high level of contrast in the acquired image.
Local thresholding strategies on the other hand are better
suited to illumination variations and have been successfully
applied to QR code binarization [19], [20]. Adaptive methods
such as the one presented in [21] show better binarization
accuracy but at the cost of increased computational complexity
which is not always possible to satisfy in embedded devices.
B. Mean Block Binarization Method
One of the most popular libraries for QR code generation
and reading is the open source Zxing library [22]. The
thresholds used in the binarization functions of this library,
are calculated through a hybrid method that use the average
luminance in a set of overlapping square windows. The
captured image is divided into non-overlapping blocks Bm,n
of 8×8pixels and then the average luminance in overlapping
sub windows of 5×5blocks is calculated according to
Tm,n =1
25 ×64
p=m+2
X
p=m2
q=n+2
X
q=n2X
(k,l)Bp,q
Y[k, l].(2)
Fig. 3: a) Image of a QR module showing the length of the
module and the center region in pixels. b) Diagram of a module
showing possible sampling points
The averages calculated for each block Bm,n are assigned
to the pixels in the block as ti,j =Tm,n for [i, j]Bm,n .
Segmentation performance is inferior to the method presented
in [19] but this method has the advantage of being computa-
tionally less expensive. This is the technique assumed in the
following sections to develop the probability of error model
and the QR embedding algorithm.
C. Sampling Grid and Probability of Sampling Error
Once the binary image IBis obtained, codewords are
extracted by sampling on a grid estimated using finder and
alignment patterns. The points in this grid are generated by
drawing parallel lines between the estimated centers of finder
and alignment patterns and the spacing between lines is set
to the estimated width of a QR module Wa. For larger code
sizes, multiple sampling grids are used to compensate for local
geometric distortions.
In order to safely detect the binary value, the luminance
around the center of the module should be clearly defined. If
we define a region of size da×dapixels centered in the QR
module, the probability of sampling outside this region can be
obtained by assuming a Gaussian distribution of the sampling
point around the center and integrating outside the region. In
the following section we model the sampling distribution as a
Gaussian with σ=Wa/4. The probability of sampling error
denoted by pscan be precomputed for different sizes of Wa
and dato be used in the embedding algorithm.
IV. AES T HE T IC E NHA NC E ME NT OF QR CO DES
The QR code embedding technique introduced here, en-
codes the QR code value on the luminance values of the image
in such a way that the average luminance is increased for light
regions in the code and decreased for dark regions. In fact any
embedding algorithm tailored for a standard decoder must be
a variation of this type since the binarization thresholds are
usually calculated as local averages of pixels luminance. The
parameters of the modification consist of selecting the location
of modified pixel and its luminance or to keep the QR module
pixels unchanged. In this paper we consider the last possibility
as a particular case in which the concentration of modified
pixels in a region is close to 1. The embedding method
proposed consists of two components. The first is the use
of halftoning techniques for the selection of modified pixels
allowing to break and reduce the coarse square structures
IEEE TRANSACTIONS ON IMAGE PROCESSING 4
of the code. The second component is the modification of
the luminance levels to minimize the image distortion. This
modification parameters are optimized independently in local
windows as described in Section VI in order to accelerate the
embedding and leverage the correlation between the luminance
of the image and the code.
A. Halftoning Techniques
The method proposed to select modified pixels is based on
halftoning techniques in order to minimize the appearance of
blocks while preserving high frequency details. If modified
pixels are randomly but uniformly distributed in space, the
visual impact of the embedding is minimized since these
patterns concentrate most of their energy at higher frequencies
where the human visual system is less sensitive. This effect
is commonly used in digital halftoning [23] where different
algorithms to generate even distributions of points with par-
ticular spectral properties have been proposed. Examples of
such algorithms are, error diffusion [24], [25], blue noise
masks [26]–[28], green noise masks [26], or direct binary
search [29]. Error diffusion has a long history in the printing
industry where it has been used for the last 30 years; however,
being a recursive algorithm, the processing time increase
considerably for very large images. Blue and green noise
masking techniques in contrast, generate high frequency binary
patterns by thresholding a carefully designed multilevel array.
These patterns can be computed in parallel which greatly
increase the speed of generation. Many algorithms to design
halftoning masks were presented in the literature such as,
Void and Cluster [30], direct binary search [29], green noise
masks [26], [31], blue noise multitone dithering [32] and
techniques based on centroidal voronoi tessellations [28]. Each
of these techniques have different computational complexities,
but since the mask design process is performed offline the
speed of pattern generation does not change.
B. Pixel Selection
Ideally, only the pixels at the center of the QR module are
relevant for a correct decoding, however, due to errors in the
determination of the sampling grid, adjacent pixels play an
important role in the decoding process. To account for this
we make a distinction between the pixels in the QR module.
A square of size da×dais always selected for modification
and the remaining pixels in the module are selected using
a halftone mask. The size of the square of center pixels
is selected by the user and regulates the robustness of the
embedding but also affects its visual quality. The distribution
of modified pixels in non central pixels is generated by
thresholding a blue or green noise mask to generate a binary
pattern with a concentration of dots of pc. To simplify the
notation we denote by Ipcthe binary pattern generated by the
halftoning mask and by Ma mask that is 1on the set of
central pixels and 0otherwise.
C. Luminance Modification
After selecting the pixels, its luminance is modified to one
of four possible levels α, β, αc, βc. The luminance of the
embedded image Yout
i,j at (i, j)is selected as a function of
the QR code value qi,j and the luminance of the original
image Yi,j as described in equation (3). This transformation
changes the luminance of the pixels that are selected according
to the halftone distribution and keep the remaining pixels in
the image unchanged. The pixels at the center of the QR
module are assigned different luminance levels, since they
play a central role in the detection of binary values when the
sampling accuracy is high.
Yout
i,j =
βif Mi,j = 0 , qi,j = 1, Ipc,(i,j )= 1
αif Mi,j = 0 , qi,j = 0, Ipc,(i,j )= 1
βcif Mi,j = 1 , qi,j = 1
αcif Mi,j = 1 , qi,j = 0
Yi,j otherwise.
(3)
D. Color Optimization
Since the goal is to embed the codes into color images,
we need to establish a rule to select the corresponding
color vector for each modified luminance since there is an
associated subspace of colors in the RGB space for each
luminance value. To determine the best possible color that
fulfills the luminance constraint, it is necessary to measure
the color differences into a perceptually uniform color space
[33]. The H SL color space is selected because it involves
simpler computations than other color spaces while it is still
capable of representing perceptual difference using euclidean
metrics. To obtain the RGB values of a modified pixel
given the luminance target lt[0,1], the original color
is transformed into the H SL color space and then the L
component is optimized keeping Sand Hfixed until reaching
the desired luminance Y=lt. The relationship between the
luminance defined as Y= 0.2989R+ 0.5870G+ 0.114B
and the L=min(R,G,B)+max(R,G,B )
2component in the HSL
color space is a piecewise, linear and monotone function
Y=f(L). If the weight luminance vector is defined as
w= [0.298,0.587,0.1140]T, then f(L)is given by
Y=f(L) = wTT1(H, S, L),(4)
where (R, G, B) = T1(H, S, L)is the backward transfor-
mation from HSL to RGB color spaces. The optimal value
of Lis obtained from the luminance target Yout =ltas the
solution of
L=argmin
L
|f(L)lt|.(5)
Once the optimal value Lis determined, the new RGB
components of the pixels are obtained by using the forward
transformation between the H SL and RGB color spaces
(R, G, B) = T(H, S, L). Summarizing the different steps,
the transformations to obtain the RGB value of a modified
Algorithm 1 Color Modification for Luminance Target
Require: (R, G, B)pixel and target value of Yout =lt
(H, S, L)T(R, G, B )
find L=argmin
L
|f(L)lt|
(R, G, B)T1(H , S, L)
return (R, G, B)
IEEE TRANSACTIONS ON IMAGE PROCESSING 5
pixel, given the target luminance value ltis described in
algorithm 1.
V. PROBAB IL ITY O F E RRO R M OD ELS
Pixels of the embedded code are a combination of pixels
from the image and the QR code as described in (3). This
mixture distorts the binarization thresholds with respect to the
black and white QR code, and in general increases the proba-
bility of binarization error. The problem can be minimized
by optimizing the luminance parameters, but this presents
a trade off between decoding robustness and image quality.
Efficient optimization methods require a tractable model for
the probability of error defined in local neighbourhoods to
leverage the parallel nature of the problem. The probability
model presented here is based on reasonable assumptions
and yields an expression for the probability of error that
is smooth and continuous and can be easily integrated into
different optimization algorithms. This model considers two
regimes, which correspond to the precise sampling of QR
modules centers and the case when all the pixels in the module
are indistinguishable. Two independent probability models are
constructed and then combined to yield a general model.
A. Probability of Binarization Error Model
The probability of binarization error is defined as the
probability of sampling the incorrect binary value at any pixel
in the QR module. This probability is influenced by different
factors such as the local distribution of luminance values in
the image, the distribution of pixels in the QR code and the
parameters of the luminance transformation. To formulate a
model of the probability of error, consider the detected binary
value at pixel (i, j). The probability of error at this location
is given by
PBerr =P(decide qi,j = 1|qi,j = 0)p0(6)
+P(decide qi,j = 0|qi,j = 1)p1.
where p0=P(qi,j = 0) and p1=P(qi,j = 1) are the
probabilities of having a black or white pixel in the code
respectively. The detector makes a binary decision on the value
of the code qi,j based on the threshold ti,j and the embedding
luminance Yout
i,j . Taking this into account, the probabilities of
miss detection in (6) can be expressed as
P(decide qi,j = 1|qi,j = 0) = P(Yout
i,j > ti.j |qi,j = 0) (7)
P(decide qi,j = 0|qi,j = 1) = P(Yout
i,j < ti.j |qi,j = 1).
The gray scale image Yis modeled as a bi-dimensional ran-
dom field, and the thresholds as correlated random variables.
The exact calculation of the probabilities in (7) involves the
knowledge of the joint distribution of the image and threshold
values which in practice can be quite challenging to calculate.
Instead of using this general relation a simplified model is
proposed here, assuming independence of the components of
the embedding luminance Yout
i,j . The value of Yout
i,j at location
(i, j)can be decomposed as the combination of several random
variables that are related to the luminance parameters of the
transformation. In this model we don’t differentiate between
Fig. 4: Probability of binarization error (PBE) for a randomly
selected window as a function of αfor fixed βand pc. The
variance of the noise used was ση= 0.1.
central and non-central pixels and consider that the contribu-
tion of central pixels to the threshold is minimal.
The expression for the output image embedding under these
assumptions is given by
Youy
i,j = [βqi,j +α(1 qi,j )]Ipc,(i,j)+Yi,j (1 Ipc,(i,j)).(8)
The first component of this expression is the binary random
variable Ipc,(i,j)that selects the pixels being modified and
models the stochastic nature of the blue binary pattern. Despite
the fact that the location of modified pixels are correlated
by the binary pattern constraints, it is reasonable to model
them as independent Bernoulli random variables given that
a sufficiently large window is considered. Furthermore, in-
dependence is also assumed between QR code values, qi,j
and luminance values Yi,j in the original image. Finally the
variables Yi,j are modeled as independent random variables
following a local distribution based on the size and location of
the window in the image. Combining these assumptions and
the expression for the binarization thresholds, it is possible
to derive (see Appendix A) a model for the probability
distribution of the thresholds given by
P(ti,j =t) = Pn
k=0 f(ttk)n
kpk
1pnk
0.(9)
where n=pc(64 ×25)is the total number of modified
pixels in the the window of blocks defined in (2), tk=
h
25×64 +(nk)α
25×64 iare the possible outcomes from the sum
of nmodified luminance pixels βqi,j +α(1 qi,j )and fis
the convolution of fY(x)corresponding to the mean value of
unmodified pixels and fn(x)corresponding to the distribution
of the Gaussian noise at the detector. This model is central
for the tractability of the decision probabilities in (7) and
yields a smooth and continuous expression for the probability
of binarization error given by
PBerr =pcPn
k=0 wk(p0F(αtk)p1F(βtk))
+ (1 pc)Pn
k=0 wk(p0p1)F(Yi,j tk) + p1
(10)
where the coefficients wk=n
kpk
1pnk
0are introduced for
compactness. Figure 4 depicts the agreement between the
IEEE TRANSACTIONS ON IMAGE PROCESSING 6
probability model and the empirical probability of binarization
error calculated by adding Gaussian noise with the appropriate
variance to the embedding and then calculating the probability
of error by counting the number of discrepancies between the
QR code and the binarized embedding.
B. Probability of Detection Error
If we assume that the pixels at the center of each module can
be sampled accurately, then the probability of error is given
by
PDerr =P(decide qc= 1|qc= 0)p0+
P(decide qc= 0|qc= 1)p1(11)
where qcis the value of the QR code module at its center and
p0=P(qc= 1) ,p1=P(qc= 1) are the probabilities of
having the corresponding QR module inside the local window.
There is a slight difference with respect to the expression in
(6) since here we consider the probability of detecting the
incorrect value but only at the center position of the module.
This is a much relaxed requirement than the one in (6) since
only the mean value of non-central pixels is important. This
naturally leads to a different probability model where the miss
detection probabilities in (11) reduce to
P(decide qc= 1|qc= 0) = P(αc> t)(12)
P(decide qc= 0|qc= 1) = P(βc< t).
Considering the luminance transformation in (3) and using
the four luminance levels. The binarization thresholds at
the detector are given by t=µ+b+ηwhere µ=
αnα+βnβ+αcnαc+βcnβc
Nand bis the mean value of unmodified
pixels in the window. The number of pixels in each modified
luminance level is denoted by, nα, nβ, nαc, nβcand Nis
the total number of pixels. By the considerations made in
section IV-C the variable bcan be regarded as a Gaussian
random variable with mean µb=E[Y1,j ](1pc)and variance
σ2
b=V ar[Yi,j ]
N(1 pc)independent of the noise η. Therefore
the probability distribution of tis also Gaussian with mean
µt=µ+µband variance σ2
t=σ2
η+σ2
band its cdf
can be expressed as a function of the Gaussian normal cdf,
P(t < x) = Φ xµt
σt. Finding the probability of detection
error then reduces to combining this result with the expressions
in (7) and (11) yielding
PDerr = Φ αcµt
σtp0Φβcµt
σtp1+p1.(13)
C. Global Probability of Error
The previous probability models define two clear regimes
in the detection of binary values, however, there are cases in
in which only a faction of the sampled pixels lie inside the
center regions with modified luminance αcor βc. However,
since the probability of sampling error can be precomputed
for a given module (Wa) and center size da, it is possible
to find the global probability of error by conditioning on the
event of successful sampling of the center region. The global
probability of error is then given by
Perr =PBerr ps+PDerr(1 ps).(14)
VI. QUA L IT Y MET RIC S
Based on the probability models developed, is possible to
accurately determine the probability of error for a given set
of luminance parameters and guarantee the decodability of
the embedding. However, it is necessary to define appropriate
metrics to quantify also its visual quality. In spite of the
complexities of the human visual system (HVS) a largely
used quality metric to measure image similarity is the mean
squared error (MSE) since it has many desirable mathematical
properties such as convexity and differentiability, however it
does not always correlates with visual quality [34]. There have
been efforts to overcome the limitations of the MSE such
as the SSIM index [35] which explicitly considers different
luminance, contrast and structure measures. For the case of
halftones the most commonly used metric is based on the
MSE between two filtered versions of the original image and
the halftone where the filter usually models the low pass
characteristic of the HVS. One of the most used models
for the HVS was proposed by N¨as¨anen [36] and consist
of an exponential model H(fr) = ekfrwhere fris the
radial frequency in cycles/degree. The corresponding discrete
impulse response considering the viewing distance Din inches
and the resolution Rin dots/inch is given by
h[m, n] = k
2π
1
k
2π+ (180m
πRD )2+ ( 180n
πRD )23/2(15)
where the constant kis determined by fitting empirical data
to the model [36].
To evaluate the similarity between the image and the em-
bedding, we propose a variation of the metric presented in
[37] originally introduced to design structure aware halftones.
The rationale for this selection is that blocks of the QR code
should be less visible when viewed from a distance where
the low pass characteristics dominate but at the same time the
embedding should preserver important details when viewed
at closer distances. This metric considers both the halftone
visibility and image structure and allows to select the trade
off between both by adjusting two weighting parameters. The
general form of this metric is given by
J(Yout, Y ) = wlG(Yout , Y ) + wh(1 M S SIM (Yout , Y )).
(16)
where the term G(Yout , Y )gives more importance to the low
pass error between the embedding and the image while the
term 1MSSIM(Yout, Y )measures the structure dissimi-
larity considering high frequency components and the overall
structure of the image.
The weighting factors selected for all the simulations in this
paper are wl= 1 and wh= 1/2and its determination was
based on several experiments with different images from which
the best combination of parameters was selected. To make a
faithful prediction of tone similarity the term Gincludes a
model of the HVS and the difference between the images in
calculated in the HSL color space since color similarity is an
important component of the perceived similarity. The general
expression for Gis then given by
G(Yout, Y ) = 1
N||h(Lout L)||F(17)
IEEE TRANSACTIONS ON IMAGE PROCESSING 7
Fig. 5: Different stages of the QR embedding procedure. The inputs are the original image, the QR code, the halftone mask,
the masks used for image prioritization and global value of Pmax and da. These images are divided in local windows and then
optimized independently and in parallel. Finally the results are interpolated to combine them in the final result.
where Lout and Lare the lightness components of the em-
bedding and original image in the HSL color space and Nis
the number of pixels in the window. Only the Lcomponent is
considered here because Sand Hare unchanged in the color
optimization. The filter hused in all the experiments is the one
defined in (15) with parameters D= 9 inches and R= 150
dpi. Considering all the requirements, the final expression for
the distortion metric used in the optimizations is then given
by
J(α, β, αc, βc, pc) = 1
N||h(Lout L)||F
+1
2(1 MSSIM(Yout, Y )) (18)
VII. PARA ME T ER O PTI MIZ ATION
In order to take advantage of local correlations between the
luminance of the image and the values of the QR code, the
optimization of the transformation parameters is performed
independently using local overlapping windows. In our im-
plementation we choose to set the size of the windows to
40 ×40 pixels centered around each image block of 8×8
pixels. This respects the proportions of the window defined in
(2) and based on our experimental tests, yields a good trade off
between robustness and quality for typical scanning distances.
The optimization of luminance parameters in local windows
has other advantages besides leveraging the correlation be-
tween image and code luminance. For example it is possible
to specify regions of the image which require higher visual
quality or higher decoding robustness and set different proba-
bility bounds for each case.
A block diagram of the optimization process is depicted in
Fig. 5. After the initial subdivision of the image, the code
and the different masks, the optimization of each window is
performed in parallel. The combination of the global lumi-
nance parameters is performed by low pass filtering the array
of solutions and interpolating to match the size of the original
image. This global map is then applied to the original image
to obtain the QR embedding.
The local optimizations are formulated based on the quality
metrics defined in (18) and the probability of error model in
(14) as argmin
α,β,αcc,pc
Ji(α, β, αc, βc, pc)
subject to Perr < Pmax,i
(19)
The input parameters to the optimizations are the global
parameter Pmax which determines the maximum admissible
probability of error and the size of the center region in pixels,
da. The parameter Pmax,i in the local optimization (19) is
determined as a function of the local features of the image.
In particular here, we define a mask MPwhich has value
2in regions where the image presents an edge, 1in the
function pattern regions and 0elsewhere. The value of Pmax,i
is calculated as
Pmax,i = (1 mean(MP,i))Pmax (20)
where MP,i is the corresponding patch of MPin the local
window.
The edges are detected using a canny edge detector and
lowpass filtering to generate the edge regions. This is a
common practice in other embedding algorithms such as [15]
or [12]. The values of Pmax,i can be distributed based on a
probability of error budget consistent with the user preferences
and type of error correction provided by the QR code (i.e.
L,M,Q or H).
A. Non-linear Optimization Methods
The optimization problem in Eqn. (19) is solved by means
of a logarithmic barrier method [38]. The solutions are found
by solving the sequence of problems
argmin
α,β,αcc,pc,s
J(α, β, αc, βc, pc) + µlog(s)
subject to Perr (α, pc)Pmax +s= 0 (21)
for decreasing values of the regularization parameter µ. The
advantage of this approach is that the starting points can be
infeasible at the beginning and as the regularization parameter
is decreased the sequence of solutions converges towards a
feasible local minimum of the original problem in Eqn. (19).
The solver used is based on a sequential quadratic programing
with trust region method presented in [39] and implemented
by the f mincon function of MATLAB [40].
VIII . IMAGE A ND LO GO E M BE DDI NG S
This Section presents the result of embedding different
images into a QR code and the experimental decoding rates
IEEE TRANSACTIONS ON IMAGE PROCESSING 8
Fig. 6: (Top row) Image embeddings for a) da= 2 pixels (J=0.4), b) da= 3 pixels (J=0.4053), c) da= 4 pixels (J=0.4194).
The QR modules in this example had a size of Wa= 8 pixels, and the global probability of error used was Pmax = 0.2. A
noise level of ση= 0.1was assumed. (Bottom row) Images acquired using a Google Nexus 7 tablet at 12in from the target.
The images were printed in a resolution of 150dpi
.
obtained with mobile devices in order to evaluate the effec-
tiveness of the embedding method on a real setting.
A. Decoding Robustness
To determine the robustness of the embedding against non
modeled disturbances such as blur and uneven illumination we
conducted an scanning experiment. The experiment consisted
of scanning the embeddings in Fig 6 from a printed version
in which each QR pixels was render at 150 dpi and also
from a laptop screen with a smart phone and a tablet. The
embeddings generated had different sizes of the central region
corresponding to da= 2 ,da= 3 and da= 4 pixels
with a corresponding module size of Wa= 8 pixels and
the halftone mask used was a green noise mask [41] to
minimize the dot gain effect introduced by the printer. The
devices used to scan the codes were a smart phone Samsung
Galaxy S I897 and a Google Nexus 7 tablet and the app
used was Barcode Scanner 1which is based on the Zxing
Library. The recognition times where in general in the order
of 5 to 10 seconds for the embeddings in Fig 6(a) and Fig
6(b) and in the order of 1 to 2 seconds for the embedding
of Fig 6(c) comparable to the method in [5]. The effective
scanning distance depends on the resolution of the device,
1https://play.google.com/store/apps/details?id=com.google.zxing.client.
android
and in general is necessary to have multiple image pixels
per QR module pixel for a successful decoding. This is in
line with the assumption that QR pixels can be individually
resolved, used to develop the probability models. Examples
of the images captured by the tablet are depicted in Figures
6(d), 6(e) and 6(f). These pictures were taken at an average
distance of 9inches from the target with approximately 10
pixels per QR module. The embeddings with smaller center
regions where more susceptible to error and the scanning
distance limited to 1 or 2 inches around the distance in which
a QR module pixel is represented by a pixel in the captured
image. For images with larger central regions such as 6(c). The
range of successful scanning is extended since the assumption
about correct sampling of the centers applies to a broader
range of conditions. One possible explanation for the reduced
robustness of the embedding is 6(a) and Fig 6(b) is that the
noise level used to design the embedding underestimates the
contributions of poor illumination or camera blur.
The values of Pmax selected should be smaller than the
corresponding decoding robustness of the code in order to have
a successful decoding. In the case of Fig 6, the code used had a
correction level H(30% correction capacity) and Pmax = 0.2.
It is important to note that this embedding method always
reduces the robustness of the code, however, this reduction
can be precisely controlled by changing the value of the Pmax
parameter and the center size da. Smaller center sizes imply a
IEEE TRANSACTIONS ON IMAGE PROCESSING 9
Fig. 7: Comparison of the embedding of different images and logos into a QR code. a) Embedding of an image for a center
region size of da= 2 pixels and a global probability of error of Pmax = 0.2using a blue noise mask b) Embedding of a logo
for da= 1 and Pmax = 0.15 using a clustered dot mask aligned with the centers of the QR code modules. c) Embedding of
a logo for da= 3 and Pmax = 0.15 using a blue noise mask. The embedding in d) and e) where generated using the method
in [5] while the embedding in f) corresponds to [14]
.
unifrom tone change similar to the results in [14] while larger
center regions yield similar results to the method in [5]. The
main advantage here is that the user can select which is best in
each case given more versatility for the integration in different
designs.
B. Visual Quality of the Embeddings
Figure 7 depicts the results of embedding a QR code into
different images and logos. Each embedding is compared with
their corresponding counterparts using the methods in [5] or
[14]. Different center sizes and halftone masks are used for
each embedding. For an objective evaluation of the quality
different metrics are compared in table I, including the metric
J(18) used in the optimization. An interesting feature of
the proposed embedding method is the possibility to generate
embeddings with different textures by changing the halftone
mask as in Fig. 7(b) where a clustered dot mask aligned with
the center of the QR modules was used generating pseudo
random cluster of dots. This feature also allows to solve
the problem of dot gain generated when the embeddings are
printed simply by using appropriate halftoning masks.
IX. DI SCU SS ION A N D CON CLU SI ON
A method to embedded QR codes into color images was
presented, allowing to automatically generate embeddings
Fig 7 (a) (d) (b) (e) (c) (f)
J(18) 0.3446 0.4171 0.3794 0.3950 0.4508 0.9091
MSSIM 0.5505 0.3716 0.5590 0.4793 0.3792 0.0802
MSE 0.1954 0.2254 0.2892 0.3091 0.2505 0.6059
TABLE I: The MSE and MSSIM are calculated over the
luminance of the embedding an the original image. The MSE
presented here is normalized by the number of pixels in the
image, M SE =1
N||(Yout Y)||F.
with limited probability of detection error for a given noise
power. This method shares different elements with previous
methods such as [14], [5] and [15] since it allows to optimize
the luminance values of individual pixels in the image at
locations selected via halftoning techniques and by setting the
centers of the QR modules to specific values. The method
presented in [14] change the luminance of modified pixels
into two luminance levels analogous to the αand βused
here. However these levels are global and the selection of
its optimal values is based on the assumption that the image
is uniformly distributed and that the binarization threshold
is fixed. The method developed here is much more general
and includes these as particular cases. We make an accurate
approximation of the image statistics using its local mean and
variance and develop a model for the probability of error
using a common binarization algorithm widely used in real
IEEE TRANSACTIONS ON IMAGE PROCESSING 10
applications. The proposed method also introduces a novel
technique to distribute modified pixels based on halftoning
methods which help to reduce the visual impact of the
modification. The embedding of halftones into QR codes was
proposed in [15] where the location of binary pixels in the QR
modules is optimized to maximize visual quality and decoding
robustness. This method however is only suitable to generate
binary halftones and no modification for the embedding of
color or multilevel halftones was proposed.
In addition to the use of halftones to diminish the visual
distortion of the embedding, the method presented here defines
a quality metric which considers color, tone and structural
similarity used to select the optimal luminance of modified
pixels. To fully leverage the characteristics of QR decoders,
central pixels in the QR modules play an important role in
our algorithms similarly to [5] and [15]. However in contrast
to these methods which fix the ratio between central and non
central pixels, the method proposed here allows to choose the
number of central pixels and then optimize the location and
luminance of modified pixels to achieve particular error limits.
In a similar way as in [15] or [12] different regions of the
image are prioritized by using local probability limits Pmax,i.
These bounds can be used to select the codewords in error
allowing to distribute the errors spatially to regions where the
image presents non relevant features.
The main limitation of this method is the assumption that
single pixels in the QR code can be resolved in the acquired
image and that they are only corrupted by Gaussian noise.
This is a simple yet powerful model but it adds a constraint
on the minimum resolution of the image. Different trials using
mobile devices and printed codes however have shown that for
common scanning distances the models are indeed applicable
and the decoding rates are reasonable in comparison with sim-
ilar methods while having a noticeable improvement in visual
quality with respect to the metric defined. An important factor
for the success of the decoding is the correct estimation of the
variance of the noise to accommodate different illuminations
or camera blur. The values of σηused in this paper are between
0.1and 0.2an were selected by experimentation with different
images and standard fluorescent illumination in an office
environment. It is important to note that the embedding method
might fail to decode if the reading application uses a different
binarization method than the one assumed here, however since
the Zxing library is widely adopted this is not a severe limita-
tion. Finally as a consequence of the elaborate models for the
visual quality and the probability of error the implementation
of the method is significantly more computationally intensive
than other methods. The present implementation in Matlab
takes 30 minutes to optimize a color image of size 350 ×350
pixels, but more efficient implementations that take advantage
of parallelism and especial processors like GPUs might reduce
the optimization time to practical levels.
APP EN D IX A
PROBAB IL ITY O F BINARIZ ATI ON E RROR
If we assume that all pixels in the window are indistin-
guishable then we can treat them as non-central pixels and
assume that they can only take two independent values αor
β. The binarization threshold at location (i, j)is in general
given by (2) and it can be divided in three components
ti,j =ai,j +bi,j +ηi,j with
ai,j =1
25×64 Pp,q P(k,l)Bp,q
(k,l)∈C
βqk,l +α(1 qk,l )
bi,j =1
25×64 Pp,q P(k,l)Bp,q
(k,l)6∈C
Y[k, l].
(22)
Given the set of modified pixels these components are indepen-
dent and is possible to evaluate the distribution of the threshold
as the convolution of the distributions of its components. The
component ai,j can be modeled as a binomial distribution
assuming independence of the code pixels given by
P(ai,j =x) = Pn
k=0 δxh
25×64 +(nk)α
25×64 in
kpk
1pnk
0
(23)
where kis the number of pixels with modified luminance βin
the local window and nis the total number of modified pixels.
Since the variables Yi,j are assumed to be independent and
identically distributed, the distribution of bi,j can be approx-
imated by a Gaussian distribution denoted by P(bi,j =y) =
fY(y)with mean µy=E[Yk,l](25×64n)
25×64 =E[Yk,l](1pc)and
variance σ2
y=V ar[Yk,l ]1pc
25×64 . Combining these expressions
and using the fact that ai,j , bi,j and ηi,j are independent give
C, the probability distribution of the threshold ti,j is given by
P(ti,j =t) = Pn
k=0(fYfn)(ttk)n
kpk
1pnk
0
=Pn
k=0 f(ttk)n
kpk
1pnk
0
(24)
where tk=h
25×64 +(nk)α
25×64 iare the possible outcomes
of the sum of nmodified pixels. The distribution fη(n)
corresponds to the Gaussian noise and f=fYfnis the
convolution of the average image values and noise distribu-
tions. Since Yand nare independent and Gaussian, fis also
Gaussian with mean µ=µYand variance σ=pσ2
Y+σ2
n.
Finally the conditional probabilities on (7) are obtained by
conditioning on the set of modified pixels Ipcas follows
P(Yout
i,j > ti.j |qi,j = 0) = P(α > ti.j )pc
+P(Yi,j > ti.j )(1 pc)(25)
and P(Yout
i,j < ti.j |qi,j = 1) = P(β < ti.j )pc
+P(Yi,j < ti.j )(1 pc).(26)
Using (25), (26) and substituting in (6) we obtain the proba-
bility of binarization error
Perr =pcPn
k=0 wk(p0F(αtk)p1F(βtk))
+(1 pc)Pn
k=0 wk(p0p1)F(Yi,j tk) + p1
where Fis the cumulative distribution function of fand
wk=n
kpk
1pnk
0. The probability of error depends on the
distribution of the image, the QR code, the transformation
parameters α, β, pcand the noise power level ση.
IEEE TRANSACTIONS ON IMAGE PROCESSING 11
REF ERE NC ES
[1] ISO, “Information technology Automatic identification and data capture
techniques Bar code symbology QR Code,” International Organization
for Standardization, Geneva, Switzerland, ISO/IEC 18004:2000.
[2] M. Hara, M. Watabe, T. Nojiri, T. Nagaya, and Y. Uchiyama, “Optically
readable two-dimensional code and method and apparatus using the
same,” Patent US 5 726 435, 03, 1995.
[3] T. J. Soon, “Qr Code,” Synthesis Journal,Information Technology and
Standards Committee Singapour, vol. 2008, no. 3, pp. 59 – 78, 2008.
[4] R. Cox. (2012, April) Qart codes. [Online]. Available: http:
//research.swtch.com/qart
[5] Visualead company. [Online]. Available: http://www.visualead.com/
[6] S. Ono, K. Morinaga, and S. Nakayama, “Two-dimensional barcode
decoration based on real-coded genetic algorithm,” in Evolutionary
Computation, 2008. CEC 2008.(IEEE World Congress on Computational
Intelligence). IEEE Congress on. IEEE, 2008, pp. 1068–1073.
[7] D. Samretwit and T. Wakahara, “Measurement of reading characteristics
of multiplexed image in qr code,” in Intelligent Networking and Col-
laborative Systems (INCoS), 2011 Third International Conference on,
2011, pp. 552–557.
[8] T. Wakahara and N. Yamamoto, “Image processing of 2-dimensional
barcode,” in Network-Based Information Systems (NBiS), 2011 14th
International Conference on, 2011, pp. 484–490.
[9] S. Oouchi, “Program, information storage medium, two-dimensional
code generation system, image generation system and printed mate-
rial,” Patent., March 2007, uS 7950589 http://www.google.com/patents/
US7950589.
[10] M. Hagiwara, “Qrjam,” http://qrjam.jp/, 2013.
[11] K. Fujita, M. Kuribayashi, and M. Morii, “Expansion of image dis-
playable area in design qr code and its applications,” 2011.
[12] Y. Lin, Y. Chang, and J. Wu, “Appearance-based qr code beautifier,
Multimedia, IEEE Transactions on, vol. 15, no. 8, pp. 2198–2207, 2013.
[13] D. W. Inc., “Logoq,” http://www.qrcode.com/en/codes/logoq.html, 2013.
[14] Z. Baharav and R. Kakarala, “Visually significant qr codes: Image
blending and statistical analysis,” in Multimedia and Expo (ICME), 2013
IEEE International Conference on, 2013, pp. 1–6.
[15] H.-K. Chu, C.-S. Chang, R.-R. Lee, and N. J. Mitra, “Halftone qr codes,”
ACM Transactions on Graphics, 2013, to appear.
[16] O. Bulan, H. Blasinski, and G. Sharma, “Color qr codes: Increased
capacity via per-channel data encoding and interference cancellation,”
in Color and Imaging Conference, vol. 2011, no. 1. Society for Imaging
Science and Technology, 2011, pp. 156–159.
[17] N. Otsu, “A threshold selection method from gray-level histograms,”
Automatica, vol. 11, no. 285-296, pp. 23–27, 1975.
[18] A. Brink and N. Pendock, “Minimum cross-entropy threshold selection,”
Pattern Recognition, vol. 29, no. 1, pp. 179–188, 1996.
[19] J. Sauvola and M. Pietik¨ainen, “Adaptive document image binarization,”
Pattern Recognition, vol. 33, no. 2, pp. 225–236, 2000.
[20] J. Zhou, Y. Liu, and P. Li, “Research on binarization of qr code image,”
in Multimedia Technology (ICMT), 2010 International Conference on,
2010, pp. 1–4.
[21] H. Yang, A. Kot, and X. Jiang, “Binarization of low-quality barcode
images captured by mobile phones using local window of adaptive
location and size,Image Processing, IEEE Transactions on, vol. 21,
no. 1, pp. 418–425, 2012.
[22] S. Owen. (2012) zxing, multi-format 1d/2d barcode image processing
library with clients for android, java and c++. [Online]. Available:
https://code.google.com/p/zxing/
[23] D. L. Lau and G. R. Arce, Modern Digital Halftoning. New York, NY,
USA: Marcel Dekker, Inc., 2001.
[24] R. Floyd and L. Steinberg, “An adaptive technique for spatial grayscale,”
in Proc. Soc. Inf. Disp, vol. 17, 1976, pp. 78–84.
[25] R. L. Stevenson and G. R. Arce, “Binary display of hexagonally sampled
continuous-tone images,JOSA A, vol. 2, no. 7, pp. 1009–1013, 1985.
[26] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Digital halftoning by
means of green-noise masks,” JOSA A, vol. 16, no. 7, pp. 1575–1586,
1999.
[27] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Green-noise digital
halftoning,Proceedings of the IEEE, vol. 86, no. 12, pp. 2424–2444,
1998.
[28] G. J. Garateguy, G. R. Arce, and D. L. Lau, “Voronoi tessellated halftone
masks,” in Image Processing (ICIP), 2010 17th IEEE International
Conference on. IEEE, 2010, pp. 529–532.
[29] M. Analoui and J. P. Allebach, “Model-based halftoning using direct
binary search,” in SPIE/IS&T 1992 Symposium on Electronic Imaging:
Science and Technology. International Society for Optics and Photonics,
1992, pp. 96–108.
[30] R. A. Ulichney, “Void-and-cluster method for dither array generation,” in
IS&T/SPIE’s Symposium on Electronic Imaging: Science and Technol-
ogy. International Society for Optics and Photonics, 1993, pp. 332–343.
[31] D. L. Lau, R. Ulichney, and G. R. Arce, “Blue and green noise halftoning
models,” Signal Processing Magazine, IEEE, vol. 20, no. 4, pp. 28–38,
2003.
[32] J. B. Rodriguez, G. R. Arce, and D. L. Lau, “Blue-noise multitone
dithering,Image Processing, IEEE Transactions on, vol. 17, no. 8, pp.
1368–1382, 2008.
[33] H. J. Trussell, E. Saber, and M. Vrhel, “Color image processing: Basics
and special issue overview,” 2005.
[34] Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it?
a new look at signal fidelity measures,” Signal Processing Magazine,
IEEE, vol. 26, no. 1, pp. 98–117, 2009.
[35] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image
quality assessment: From error visibility to structural similarity,” Image
Processing, IEEE Transactions on, vol. 13, no. 4, pp. 600–612, 2004.
[36] R. Nasanen, “Visibility of halftone dot textures,” Systems, Man and
Cybernetics, IEEE Transactions on, no. 6, pp. 920–924, 1984.
[37] W.-M. Pang, Y. Qu, T.-T. Wong, D. Cohen-Or, and P.-A. Heng,
“Structure-aware halftoning,” ACM Transactions on Graphics (TOG),
vol. 27, no. 3, p. 89, 2008.
[38] D. P. Bertsekas, “Nonlinear programming,” 1999.
[39] R. H. Byrd, J. C. Gilbert, and J. Nocedal, “A trust region method based
on interior point techniques for nonlinear programming,Mathematical
Programming, vol. 89, no. 1, pp. 149–185, 2000.
[40] Mathworks. fmincon reference page documentation. [Online]. Available:
http://www.mathworks.com/help/optim/ug/fmincon.html
[41] D. L. Lau, G. R. Arce, and N. C. Gallagher, “Digital color halftoning
with generalized error diffusion and multichannel green-noise masks,”
Image Processing, IEEE Transactions on, vol. 9, no. 5, pp. 923–935,
2000.
Gonzalo J. Garateguy (S06) received the B.E.E.
degree from the Catholic University of Uruguay in
2007 and the PhD. degree from the University of
Delaware in 2014. He is currently working as a
Signal Processing Quality Engineer at the Math-
works. His research interest include signal and image
processing, halftoning and convex optimization.
Gonzalo R. Arce (F00) received the M.S. and
Ph.D. degrees in electrical engineering from Purdue
University, West Lafayette. He is the Charles Black
Evans Professor of Electrical and Computer Engi-
neering at the University of Delaware, Newark. He
held the 2010 Fulbright-Nokia Distinguished Chair
in Information and Communications Technologies
at Aalto University, Helsinki, Finland. His research
interests include statistical signal processing and
computational imaging. He is coauthor of the texts
Nonlinear Signal Processing (Wiley, 2004), Modern
Digital Halftoning (CRC Press, 2008), and Computational Lithography (Wiley,
2010). Dr. Arce has served as Associate Editor of several journals of the IEEE,
OSA, and SPIE.
Daniel L. Lau (SM09) received the B.Sc. degree
with highest distinction from Purdue University,
West Lafayette, IN, in 1995, and the Ph.D. degree
in electrical engineering from the University of
Delaware, Newark, in 1999. He is currently an As-
sociate Professor with the Department of Electrical
and Computer Engineering, University of Kentucky,
Lexington, where his early research interest focused
on digital printing and halftoning. He is the author
of Modern Digital Halftoning. His current research
interests include 3-D measurements by means of
structured light, with his contributions having been featured in numerous
machine vision trade magazines, as well as the creation of a start-up
corporation, i.e., Seikowave Inc. Aside from his research, he is also an avid
digital photographer primarily interested in lenticular photography. He has
published papers in the PROCEEDINGS OF THE IEEE.
... QR codes were first invented for tracking vehicles and parts during the manufacturing process, recently with the rapid development of mobile Internet, these have been widely applied to many different fields. This has resulted in an increased research interest in the technology necessary for embellishing QR codes [2], [3], [4], [5], [9], [10], [11], [12], [13], [14], [15], [16]. Currently, the existing foundation algorithms of generating embellished QR codes can be classified into four groups: embedding icons [17], replacing colors [18], changing the module shape [3], [14], and adjusting codewords [2]. ...
... Subsequently, Garateguy et al. [9] proposed a new algorithm using halftone, which was based on the selection of a set of modified pixels using a halftoning mask and a probabilistic model predicting the distortion generated by the embedded image. Although it can enhance the visual quality by sacrificing some scanning robustness, the generated images still contain much image noise. ...
... However, when converting the color space, the range of color expression is always limited, so simply adjusting luminance may not meet the expected requirements in some cases. In addition, luminance adjustment is usually transformed into an optimization problem, which increases the cost of calculation [9]. ...
Preprint
QR codes are usually scanned in different environments, so they must be robust to variations in illumination, scale, coverage, and camera angles. Aesthetic QR codes improve the visual quality, but subtle changes in their appearance may cause scanning failure. In this paper, a new method to generate scanning-robust aesthetic QR codes is proposed, which is based on a module-based scanning probability estimation model that can effectively balance the tradeoff between visual quality and scanning robustness. Our method locally adjusts the luminance of each module by estimating the probability of successful sampling. The approach adopts the hierarchical, coarse-to-fine strategy to enhance the visual quality of aesthetic QR codes, which sequentially generate the following three codes: a binary aesthetic QR code, a grayscale aesthetic QR code, and the final color aesthetic QR code. Our approach also can be used to create QR codes with different visual styles by adjusting some initialization parameters. User surveys and decoding experiments were adopted for evaluating our method compared with state-of-the-art algorithms, which indicates that the proposed approach has excellent performance in terms of both visual quality and scanning robustness.
... Quick Response (QR) codes, due to their capability to store a substantial amount of data and their ease of accessibility through basic camera devices, have become an exceedingly widespread medium for the representation of information in the digital era [13,47,3,4,36]. With the wide application of QR codes in social context, there has been increasing needs for customizing QR codes to include personal identity and aesthetic allure. However, such needs cannot be fulfilled by the dull appearance of common QR codes, which contain only black and white modules. ...
... With the widespread application of QR codes across diverse fields, related technologies are also developing at a rapid pace. While techniques employing image transformation [5,6,13,46] and style transferring [36,47] can partially retain facial features, their perceptual quality and aesthetic adaptability are limited. On the other hand, generative model-based approaches [11,43] can produce QR code images of superior quality and diversity, yet they pose challenges in controlling the generated content, particularly in preserving human facial characteristics. ...
... Achieving a harmonious balance between strict facial ID preservation and diverse customized background styles within a unified pipeline presents a notable challenge. Methods reliant on style transfer [36,47] often yield facial textures that appear unnatural, while those based on image transfer [5,6,13,46] may introduce visible artifacts in the facial region; (2) Conflict between face ID and QR code pattern. While prior generative model-based techniques [43] have demonstrated the ability to control the QR code pattern using QR blueprints, they have struggled to exclude these patterns from the facial region, resulting in unnatural shadows and undesirable artifacts. ...
Preprint
Full-text available
Existing methods to generate aesthetic QR codes, such as image and style transfer techniques, tend to compromise either the visual appeal or the scannability of QR codes when they incorporate human face identity. Addressing these imperfections, we present Face2QR-a novel pipeline specifically designed for generating personalized QR codes that harmoniously blend aesthetics, face identity, and scannability. Our pipeline introduces three innovative components. First, the ID-refined QR integration (IDQR) seamlessly intertwines the background styling with face ID, utilizing a unified Stable Diffusion (SD)-based framework with control networks. Second, the ID-aware QR ReShuffle (IDRS) effectively rectifies the conflicts between face IDs and QR patterns, rearranging QR modules to maintain the integrity of facial features without compromising scannability. Lastly, the ID-preserved Scannability Enhancement (IDSE) markedly boosts scanning robustness through latent code optimization, striking a delicate balance between face ID, aesthetic quality and QR functionality. In comprehensive experiments, Face2QR demonstrates remarkable performance, outperforming existing approaches, particularly in preserving facial recognition features within custom QR code designs. Codes are available at \href\href{https://github.com/cavosamir/Face2QR}{\text{this URL link}}.
... A standard QR code consists of a square grid of black and white square blocks that reveals no useful information to human eyes. Some QR code generation methods [6][7][8][9][10][11][12] were proposed for constructing QR codes with visually meaningful patterns in order to provide discriminative information to human beings. ...
... Most aesthetic QR codes [8][9][10][11][12] in the literature were designed on the basis that the module value is determined by a one-third centroid region (i.e., Ω = 1/3) of a module block. Experimental results demonstrated that the generated QR codes performed well both in terms of the quality of the visual elements and decoding robustness of the QR codes. ...
Article
Full-text available
A novel dual-message QR code is proposed for carrying two individual messages that can be read by standard QR code readers: one from a close range and the other from a large distance. By exploring the module value determining the rule of typical QR code readers, we designed two-state module blocks that can be recognized as different module values through changing the distance from which the QR code is scanned, and applied them to construct the proposed dual-message QR code. Experiments were conducted to test the readability of the two messages within a dual-message QR code, with the results demonstrating the high feasibility of the proposed method. The dual-message QR code can be applied for designing creative applications. For example, an interactive wedding card that can access the growing film of the groom and that of the bride interchangeably, which bring the viewers a higher-quality experience.
... Jika sebuah QR Code tidak terdapat zona ini, maka ada kemungkinan QR code sulit terbaca.Penggunaan QR code secara umum melalui ponsel yang dilengkapi dengan aplikasi pembaca QR code (i.e QR scanner, QR Code reader). Pola dan struktur dalam QR code memiliki fungsi yang terdefinisi dimana terdiri dari garis symbol, sampling grid, dan koreksi error[20]. Prosesnya informasi di encoding dalam finder pattern. Awalnya, data di encode berdasarkan mode dan bit stream. ...
Article
Full-text available
Medical record play important role in order to improve hospital healthcare quality. Medical record must provide proper, fast, and relevant data. Current medical records documentation is manual based in mean the documents are available in notes and document maps. There is no integration among the document, and it can caused a misunderstanding among hospital stakeholder such as doctor or nurse and also duplicated data. QR code is one of alternative to handle completeness file, searching document, and store all document. The combination between QR code, web and mobile platform can handle the problem faced by hospital stakeholder. In testing phase, the proposed system can handle medical record problem in RS. St. Elisabeth. Average of respondent, more than 50% state that sistem is good to handle tracking record, completeness file, and store.
Article
The prevalence of the Internet of Things (IoT) has led to the widespread adoption of 2D barcodes as a means of offline-to-online communication. Whereas, 2D barcodes are not ideal for publicity materials, due to their space-consuming nature. Recent works have proposed 2D image barcodes that contain invisible codes or hyperlinks to transmit hidden information from offline to online. However, these methods undermine the purpose of the codes being invisible, due to the the requirement of markers to locate them. The conference version of this work has presents a novel imperceptible information embedding framework for display or print-camera scenarios, which includes not only hiding and recvoery but also locating and correcting. With the assistance of learned invisible markers, hidden codes can be rendered truly imperceptible. A highly effective multi-stage training scheme is proposed to achieve high visual fidelity and retrieval resiliency, wherein information is concealed in a sub-region rather than the entire image. However, our conference version does not address the optimal sub-region for hiding, which is crucial when dealing with local region concealment problems. In this paper extension, we consider human perceptual characteristics and introduce an optimal hiding region recommendation algorithm that comprehensively incorporates Just Noticeable Difference (JND) and visual saliency factors into consideration. Extensive experiments demonstrate superior visual quality and robustness compared to state-of-the-art methods. With the assistance of our proposed hiding region recommendation algorithm, concealed information becomes even less visible than the results of our conference version without compromising robustness.
Article
Full-text available
The article reveals the features of the graphic language of coded information by identifying, systematizing, and combining QR-codes and AR-codes at the present stage of graphic design development. The analysis of coded information in different types of graphic design is presented: advertising, environmental, motion, print, packaging, set prop, web, mobile app, art, and illustration graphic design. The role of a graphic designer in the development of digital codes of various types for various fields of application is described. It is proved that the coded information is a trigger between the online and offline worlds. The article also reveals the importance of using coded information for the formation, development, and improvement of the brand.
Article
Full-text available
Display field communication (DFC) is an unobtrusive display-to-camera technology that transmits data within the frequency domain of images, ensuring that the embedded data are hidden and do not disrupt the viewing experience. The display embeds data into image frames, while the receiver captures the display and extracts it. Two-dimensional DFC (2D-DFC) focuses on embedding data in the width and height of an image. This study explores two methods to minimize the error rate in 2D-DFC without affecting the quality of the displayed image. The orthogonal method embeds data in the orthogonal direction of an image. On the other hand, the diagonal embedding method strategically embeds the data in the diagonal direction. Experiments show the diagonal method maintains a higher peak signal-to-noise ratio and surpasses the orthogonal embedding method in terms of bit error rate. 2D-DFC is expected to have practical applications in digital signage, advertising and informational displays at airports and train stations, as well as at large-scale displays for events, sports arenas, and performance venues.
Article
Full-text available
Article
Full-text available
We introduce a novel technique for generating green-noise halftones—stochastic dither patterns composed of homogeneously distributed pixel clusters. Although techniques employing error diffusion have been proposed previously, the technique here employs a dither array referred to as a green-noise mask, which greatly reduces the computational complexity formerly associated with green noise. Compared with those generated with blue-noise masks, halftones generated with green-noise masks are less susceptible to printer distortions. Because green noise constitutes patterns with widely varying cluster sizes and shapes, the technique introduced here for constructing these green-noise masks is tunable; that is, it allows for specific printer traits, with small clusters reserved for printers with low distortion and large clusters reserved for printers with high distortion. Given that blue noise is a limiting case of green noise, this new technique can even create blue-noise masks.
Article
Objective methods for assessing perceptual image quality have traditionally attempted to quantify the visibility of errors between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a Structural Similarity Index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000. A MatLab implementation of the proposed algorithm is available online at http://www.cns.nyu.edu/~lcv/ssim/.
Conference Paper
This chapter deals with digital halftoning. In terms of photolithography, halftoning involved projecting light from the negative of a continuous-tone photograph, through a mesh screen, such as finely-woven silk, onto a photosensitive plate. Later versions of the halftoning process employed screens made of glass that were coated on one side by an opaque substance. Later still, the glass plate mesh was replaced altogether with a flexible piece of processed film, placed directly in contact with the unexposed lithographic film. This contact screen had direct control of the dot structure. The screen controlled the screen frequency, the dot shape, and the screen angle. This chapter discusses three types of halftonig—amplitude-modulated (AM) halftoning, frequency-modulated (FM) halftoning, and AM-FM hybrids. In AM halftoning, halftone screen has been replaced in digital printers with a raster image processor (RIP) that converts each pixel of the original image from an intermediate tone directly into a binary dot. In FM or stochastic screening techniques, dots of constant size but variably spaced according to tone, are available to digital printers. AM–FM hybrids produce dot clusters that vary according to tone in both their size and spacing.
Article
QR code is a popular form of barcode pattern that is ubiquitously used to tag information to products or for linking advertisements. While, on one hand, it is essential to keep the patterns machine-readable; on the other hand, even small changes to the patterns can easily render them unreadable. Hence, in absence of any computational support, such QR codes appear as random collections of black/white modules, and are often visually unpleasant. We propose an approach to produce high quality visual QR codes, which we call halftone QR codes, that are still machine-readable. First, we build a pattern readability function wherein we learn a probability distribution of what modules can be replaced by which other modules. Then, given a text tag, we express the input image in terms of the learned dictionary to encode the source text. We demonstrate that our approach produces high quality results on a range of inputs and under different distortion effects.
Conference Paper
We introduce the concept of green noise-the mid-frequency component of white noise-and its advantages over blue noise for digital halftoning. Unlike blue noise, which creates the illusion of continuous tone by spreading the minority pixels of a binary dither pattern as homogeneously as possible, green noise forms minority pixel clusters which are themselves distributed as homogeneously as possible. By clustering pixels, green noise patterns are less susceptible to image degradation from printer distortions such as dot-overlap (the overlapping of a printed dot with its nearest neighbors), and by adjusting the average number of pixels per cluster, green noise patterns are tunable to specific printer characteristics. Using both spectral and spatial statistics, are establish models for ideal green noise patterns
Conference Paper
QR codes are widely used as a means of conveying textual information, such as emails, hyperlinks, or phone numbers, through images that are interpreted using a smartphone camera. The codes take up valuable space in print media. The random appearance of QR codes not only detracts from the production values of the advertisement in which they appear, but the codes are also visually insignificant in the sense that a human cannot discern the vendor, brand, or purpose of the code just by looking at it, without the aid of scanning software. Though neither the aesthetics nor the visual significance of the code matter for scanning purposes, they do matter for advertising layout and, more importantly, can provide valuable brand distinction. In this paper, we show how the visually significant QR codes may be obtained by image blending. Unlike various ad-hoc methods that have been proposed by others, our method leaves completely intact the error correction budget of the code. Our method allows images as diverse as corporate logos and family photographs to be embedded in the code in full color. We provide a detailed statistical analysis of the method to show the effect of blending on error rates in noisy environments.
Article
Quick Response (QR) code is a widely used matrix bar code with the increasing population of smart phones. However, QR code usually consists of random textures which are not suitable for incorporating with other visual designs (e.g. name card and business advertisement poster). In order to overcome the shortcomings of noise-like looks of QR codes, we propose a systematic QR code beautification framework where the visual appearance of QR code is composed of visually meaningful patterns selected by users, and more importantly, the correctness of message decoding is kept intact. Our work makes QR code from machine decodable only (i.e. standardized random texture) to a personalized form with human visual pleasing appearance. We expect the proposed QR code beautifier can inspire more visual-pleasant mobile multimedia applications.
Article
Objective methods for assessing perceptual image quality traditionally attempted to quantify the visibility of errors (differences) between a distorted image and a reference image using a variety of known properties of the human visual system. Under the assumption that human visual perception is highly adapted for extracting structural information from a scene, we introduce an alternative complementary framework for quality assessment based on the degradation of structural information. As a specific example of this concept, we develop a Structural Similarity Index and demonstrate its promise through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000.