ArticlePDF Available

The Impact of Curviness on Four Different Image Sensor Forms and Structures

Article

The Impact of Curviness on Four Different Image Sensor Forms and Structures

Abstract and Figures

The arrangement and form of the image sensor have a fundamental effect on any further image processing operation and image visualization. In this paper, we present a software-based method to change the arrangement and form of pixel sensors that generate hexagonal pixel forms on a hexagonal grid. We evaluate four different image sensor forms and structures, including the proposed method. A set of 23 pairs of images; randomly chosen, from a database of 280 pairs of images are used in the evaluation. Each pair of images have the same semantic meaning and general appearance, the major difference between them being the sharp transitions in their contours. The curviness variation is estimated by effect of the first and second order gradient operations, Hessian matrix and critical points detection on the generated images; having different grid structures, different pixel forms and virtual increased of fill factor as three major properties of sensor characteristics. The results show that the grid structure and pixel form are the first and second most important properties. Several dissimilarity parameters are presented for curviness quantification in which using extremum point showed to achieve distinctive results. The results also show that the hexagonal image is the best image type for distinguishing the contours in the images.
Content may be subject to copyright.
sensors
Article
The Impact of Curviness on Four Different Image
Sensor Forms and Structures
Wei Wen * ID and Siamak Khatibi
Department of Technology and Aesthetics, Blekinge Tekniska Högskola, 37141 Karlskrona, Sweden;
siamak.khatibi@bth.se
*Correspondence: wei.wen@bth.se; Tel.: +46-734-223-636
Received: 15 December 2017; Accepted: 29 January 2018; Published: 1 February 2018
Abstract:
The arrangement and form of the image sensor have a fundamental effect on any further
image processing operation and image visualization. In this paper, we present a software-based
method to change the arrangement and form of pixel sensors that generate hexagonal pixel forms on a
hexagonal grid. We evaluate four different image sensor forms and structures, including the proposed
method. A set of 23 pairs of images; randomly chosen, from a database of 280 pairs of images are used
in the evaluation. Each pair of images have the same semantic meaning and general appearance, the
major difference between them being the sharp transitions in their contours. The curviness variation
is estimated by effect of the first and second order gradient operations, Hessian matrix and critical
points detection on the generated images; having different grid structures, different pixel forms
and virtual increased of fill factor as three major properties of sensor characteristics. The results
show that the grid structure and pixel form are the first and second most important properties.
Several dissimilarity parameters are presented for curviness quantification in which using extremum
point showed to achieve distinctive results. The results also show that the hexagonal image is the
best image type for distinguishing the contours in the images.
Keywords:
software-based; virtual; hexagonal image; grid structure; pixel form; fill factor; curviness
quantification; Hessian matrix; critical points
1. Introduction
The arrangement and form of photoreceptors vary from the fovea to the periphery of the retina.
This is a consequence of evolution which argues that the arrangement and form of camera pixel sensors
should be variable as well. However practical issues and history of camera development have made
us to use fixed arrangements and forms of pixel sensors. Our previous works [
1
,
2
] showed that despite
the limitations of hardware, it is possible to implement a software method to change the size of pixel
sensors. In this paper, we present a software-based method to change the arrangement and form of
pixel sensors.
The pixel sensor arrangement is often referred to as the grid structure. Most available cameras
have rectangular grid structures. Previous works [
3
,
4
] have shown the feasibility of converting the
rectangular to hexagonal grid structure by a half pixel shifting method (i.e., a software-based approach).
Generation of the hexagonal pixel form is generally achieved by interpolation of intensity values of
the rectangular pixel form. In this paper, we present a method, based on our previous works, for
maximizing the size of rectangular pixel forms, that generates a hexagonal pixel form on a hexagonal
grid. Each original rectangular pixel form is deformed to a hexagonal one using modelling of the
incident photons onto the senor surface. To the best of our knowledge, there is no previous method
which can offer hexagonal deformation of pixel form on a hexagonal grid.
The comparison of different grid structures or different pixel forms is a challenging task and
should be directed to a more specific task. Inasmuch as human vision is highly evolved to detect
Sensors 2018,18, 429; doi:10.3390/s18020429 www.mdpi.com/journal/sensors
Sensors 2018,18, 429 2 of 19
objects in a dynamic natural scene, the gradient computation as an elementary operation in object
detection becomes interesting and appropriate candidate for this specific task. We have focused our
investigation on the effect of the sharp transitions in the contour of objects which is estimated by first
and second order of gradient computation on the images generated by different grid structures and
different pixel forms. Two categories of images having curved versus linear edges of the same object in
a pair of images, are used to estimate the detectability of each of the four considered sensor structures
for curviness.
This paper is organized as follows: in Section 2, related research on hexagonal grid resampling
and form is explained. Then the four types of image generations are explained in Section 3. Sections 4
and 5present the methodology of curviness quantification and the experiment setup, respectively,
then the results are shown and discussed in Section 6. Finally, we summarize the work described in
this paper in Section 7.
2. Background
Due to a higher sampling efficiency, consistent connectivity and higher angular resolution of
hexagonal grids [
3
] in comparison to square grids, in the last four decades hexagonal grids have been
investigated in numerous methods and applications [
3
,
5
7
], which include image reconstruction [
5
],
edge detection [
7
,
8
], image segmentation [
9
], and motion estimation [
10
]. Different algorithms and
mathematical models have emerged in recent years to acquire hexagonal grids. For example, the
rectangular grid can be suppressed in rows and columns alternatively and be sub-sampled; i.e., by
a half-pixel shifting method [
11
]. In this way, a bigger hexagonal pixel is generated at the cost of
obtaining lower resolution in comparison to the original rectangular grid. In the method, the distance
between rows is changed by
3/
2 and the pixel shifting can be achieved e.g., by implementing
normalized convolution [
12
]. The significances of such a structure are the equidistant and 60 degrees
intersection of the sampling points. In Yabushita et al. [
5
], the pseudohexagonal elements are composed
of small square pixels with an aspect ratio of 12:14, which was later implemented by Jeevan et al.
with a different ratio of 9:8 [
13
]. In the spiral architecture of He et al. [
14
] four square pixels are
averaged and generate a hexagonal pixel. Based on the spiral architecture, a design procedure for the
development of hexagonal tri-directional derivative operators is present in [
15
], that can be applied
directly to hexagonal images and can be used to improve both the efficiency and accuracy with respect
to feature extraction on conventional intensity images. Although the architecture preserves the main
properties of object, it loses some degree of resolution, which has an impact, especially on the result of
edge detection applications [
8
]. Later this architecture was improved by Wu et al. [
16
], by mapping
the rectangular grids to hexagonal ones, processing images on hexagonal grids, and remapping the
results to the square grids. By processing images on a hexagonal grid less distortion was observed [
3
].
All above software-based methods have one major common property: they convert the rectangular
grid to the hexagonal one using a linear combination of rectangular pixels. The technique of resampling
digital images on this pseudohexagonal grid by using three interpolation kernels is proposed in [
17
]
and one blurring kernel have been demonstrated. Then a new spline based on a least-squares approach
is presented in [
18
], used for converting to a hexagonal lattice and has been demonstrated to achieve
better quality than traditional interpolative resampling. In the most recent research, Ref [
19
] introduced
a method to convert images from square to hexagonal lattices in the frequency domain using the
Fourier transform. However, in our approach, the rectangular pixels initiate a non-linear learning
model based on the photons incident onto the senor surface. The approach is based on our earlier
works [
20
,
21
] where the fill factor of an arbitrary image was estimated and used to obtain an enhanced
image, as captured by a 100% fill factor sensor.
Recently as image quality and preserving of the quality during operations such as translation,
rotation, and super resolution have become important issues to the research community, implementing
a hexagonal grid has been shown to be one of the alternative solutions [
12
,
22
]. The hexagonal grid
is our visual system solution for observing our complex environmental scenes. Believing that such
Sensors 2018,18, 429 3 of 19
natural scenes have had a great impact on the evolution of our visual system; i.e., in the creation of
hexagonal grid in the fovea of the retina, then the justified question is which features in a natural
scene and in its dynamical alteration cause such an impact. The pioneering work was done by Gestalt
psychologists and, more in detail, by Rubin [
23
], who first demonstrated that contours contain most
of the information related to object perception, like the shape, the color and the depth. In fact, by
investigating simple conditions like those used by Gestalt psychologists, mostly consisting of contours
only, Pinna et al. [
24
] demonstrated that the phenomenal complexity of the material attributes emerges
through appropriate manipulation of the contours. Bar et al. [
16
] showed in their psychological
study that our visual system prefers curved visual objects; i.e., the physical property of objects in a
scene, which is manifested in sharp transitions in the contour of objects, has a critical influence on
our perception of that object. Other studies such as [
25
] show the capacity of the hexagonal grid on
detection of the sharp transitions in contour of objects. In this paper, we implement the curviness;
i.e., the sharp transitions in contour of objects, as a comparison feature to evaluate four different
grid structures.
3. Image Generation
In this section, we explain the generation of hexagonal enriched, square enriched, haft pixel
shift enriched, and half pixel shift images from an original image which has a square pixel form on a
square grid.
3.1. Generation of the Hexagonal Enriched Image (Hex_E)
The hexagonal enriched image has a hexagonal pixel form on a hexagonal grid. The generation
process is similar to the resampling process in [
1
], which has three steps: projecting the original image
pixel intensities onto a grid of sub-pixels; estimating the values of subpixels at the resampling positions;
estimating each new hexagonal pixel intensity in a new hexagonal arrangement. The three steps are
elaborated in the following.
3.1.1. A Grid of Virtual Image Sensor Pixels Is Constructed
Each pixel is projected onto a grid of
L×L
square subpixels. By using the fill factor
FF
value, the
size of the active area is defined as S
×
S, where
S=L×FF
. The intensity value of every pixel in the
image sensor array is assigned to the virtual active area in the new grid. The intensities of subpixels
in the non-sensitive areas are assigned to be zero. An example of such sensor rearrangement on a
sub-pixel level is presented on the left in Figure 1, where there is a 3
×
3 pixel grid, and the light and
dark grey areas represent the active and non-active areas in each pixel. Assuming
L=
30 and the
active area is composed by 18
×
18 subpixels, and thereby the fill factor becomes 36% according to
the above equation, and the intensities of active areas are represented by different grey level values.
The size of the square subpixel grid for one pixel is examined from 20
×
20 to 40
×
40, the intensity
in the generated images show no further significant changes after the size is 30
×
30. Thus, in the
experiment, Lis set to 30.
3.1.2. The Second Step Is to Estimate the Values of Subpixels in the New Grid of Subpixels
Considering the statistical fluctuation of incident photons and their conversion to electrons on
the sensor, a local Gaussian model is estimated by maximum likelihood method from each certain
neighborhood area of pixels. Using each local model, a local noise source is generated and introduced
to each certain neighborhood. Then inspired by Monte Carlo simulation, all subpixels in each certain
neighborhood are estimated in an iteration process using the known pixel values (for sub-pixels in
the active area) or by linear polynomial reconstruction (for subpixels in non-sensitive area). In each
iteration step the number of subpixels of the active area in the actual pixel is varied from zero to
total number of subpixels of active area (i.e., the total sub-pixel number is defined by the fill factor).
By estimating the intensity values of the subpixels during the iteration process, a vector of intensity
Sensors 2018,18, 429 4 of 19
values for each subpixel is created from which the final subpixel value is optimally predicted using
Bayesian inference method and maximum likelihood of Gaussian distribution.
3.1.3. In the Third Step, the Subpixels Are Projected back to a Hexagonal Grid Shown as Red Grids on
the Right of Figure 1, Where the Distance between Each Two Hexagonal Pixels Is the Same
Then the subpixels in each hexagonal area are estimated with respect to the virtual increase of
the fill factor. The intensity value of a hexagonal pixel in the grid is the intensity value which has the
strongest contribution in the histogram of belonging subpixels. The corresponding intensity is divided
by the fill factor for removing the fill factor effect to obtain the hexagonal pixel intensity.
Sensors 2018, 18, x 4 of 19
3.1.3. In the Third Step, the Subpixels Are Projected back to a Hexagonal Grid Shown as Red Grids
on the Right of Figure 1, Where the Distance between Each Two Hexagonal Pixels Is the Same
Then the subpixels in each hexagonal area are estimated with respect to the virtual increase of
the fill factor. The intensity value of a hexagonal pixel in the grid is the intensity value which has the
strongest contribution in the histogram of belonging subpixels. The corresponding intensity is
divided by the fill factor for removing the fill factor effect to obtain the hexagonal pixel intensity.
Figure 1. From left to right: the sensor rearrangement onto the subpixel, the projection of the square
pixels onto the hexagonal grid by half pixel shifting method and the projection of the square pixels
onto the hexagonal grid in generation of hexagonal image.
3.2. Generation of the Square Enriched Image (SQ_E)
The estimated square images are generated by three steps where the two steps explained in 3.1.1
and 3.1.2 are followed by a third step as follows. The subpixels are projected back to the original
square grid shown as red grids on the left of Figure 1. The intensity value of each pixel in the square
grid is the intensity value which has the strongest contribution in the histogram of its belonging
subpixels. Then the corresponding intensity is divided by the fill factor to obtain the square pixel
intensity by virtual increase of fill factor to 100% as the work in [1].
3.3. Generation of the Half Pixel Shift Image (HS) and Half Pixel Shift Enriched Image (HS_E)
The hexagonal grid in the previous work [3,4] is mimicked by a half-pixel shift which is derived
from delaying sampling by half a pixel in the horizontal direction. The red grid, which is presented
in the middle of Figure 1, is the new pseudohexagonal sampling structure whose pixel form is still
square. The new pseudohexagonal grid is derived from a usual 2-D grid by shifting each even row a
half pixel to the right and leaving odd rows unattached, or of course any similar translation. The half
pixel shift image (HS) and half pixel shift enriched image (HS_E) are both generated from the original
image (SQ) and enriched image (SQ_E) on the square grid, respectively.
4. Curviness Quantification
The curviness is quantified by comparison of the sharp transitions in contour of all
correspondent objects in pair of images which have exact similar contents but two different contours;
namely straight or curved contour. We define an image which has only straight or only curved
contour as SC or CC image respectively. First and second order gradient operations are used in the
quantification on each original image (i.e., SQ image type) and its set of generated images (i.e., Hex_E,
SQ_E, HS_E, and HS image types). We elaborate these operations in following.
Figure 1.
From left to right: the sensor rearrangement onto the subpixel, the projection of the square
pixels onto the hexagonal grid by half pixel shifting method and the projection of the square pixels
onto the hexagonal grid in generation of hexagonal image.
3.2. Generation of the Square Enriched Image (SQ_E)
The estimated square images are generated by three steps where the two steps explained in 3.1.1
and 3.1.2 are followed by a third step as follows. The subpixels are projected back to the original square
grid shown as red grids on the left of Figure 1. The intensity value of each pixel in the square grid is
the intensity value which has the strongest contribution in the histogram of its belonging subpixels.
Then the corresponding intensity is divided by the fill factor to obtain the square pixel intensity by
virtual increase of fill factor to 100% as the work in [1].
3.3. Generation of the Half Pixel Shift Image (HS) and Half Pixel Shift Enriched Image (HS_E)
The hexagonal grid in the previous work [
3
,
4
] is mimicked by a half-pixel shift which is derived
from delaying sampling by half a pixel in the horizontal direction. The red grid, which is presented
in the middle of Figure 1, is the new pseudohexagonal sampling structure whose pixel form is still
square. The new pseudohexagonal grid is derived from a usual 2-D grid by shifting each even row a
half pixel to the right and leaving odd rows unattached, or of course any similar translation. The half
pixel shift image (HS) and half pixel shift enriched image (HS_E) are both generated from the original
image (SQ) and enriched image (SQ_E) on the square grid, respectively.
4. Curviness Quantification
The curviness is quantified by comparison of the sharp transitions in contour of all correspondent
objects in pair of images which have exact similar contents but two different contours; namely straight
or curved contour. We define an image which has only straight or only curved contour as SC or CC
image respectively. First and second order gradient operations are used in the quantification on each
original image (i.e., SQ image type) and its set of generated images (i.e., Hex_E, SQ_E, HS_E, and HS
image types). We elaborate these operations in following.
Sensors 2018,18, 429 5 of 19
4.1. Implementing a First Order Gradient Operation
The familiar first order gradient Jis defined as:
J(z)=J(z)Jz0(1)
where
J
represents the image,
z
and
z0
represent the positions of two adjacent pixels which have a
common border; i.e., a common side or corner border. The angle between orientation of adjacent pixels
and horizontal axis represents direction of the gradient. The first order gradient values of a pair of
images (i.e., even with different grids) can be compared by computing and analyzing the eigenvalues
and eigenvectors which are obtained by solving the expression
AλjIej=
0, where
A
is the 2
×
n
matrix with nnumber of first order gradient values of each of the images,
λ
and
e
are the eigenvalue
and the eigenvector respectively,
j
is the index number with value of 1 or 2, and
I
is the identity matrix.
In the comparison, when two images have same contents but different grids, the range of eigenvalues
from small to large values indicate the similarity to dissimilarity between grid structures. When two
images have the same content but different contours the curviness can be quantified by comparison of
the eigenvalues related to the images.
4.2. Implementing Hessian Matrix on SQ, and SQ_E Images
Analyzing the second order gradient operation in form of Hessian matrix computation has
an intuitive justification in the context of curvature quantification. The eigenvalues analysis of the
Hessian extracts the principle gradient directions in which the local second order structure of an image
is decomposed. This directly gives the direction of smallest curvature (along the contour) [
26
,
27
].
The Hessian matrix of:
H="Jxx Jxy
Jyx Jyy #(2)
is computed from convolution of the image
J
and gradients of the gaussian kernel
G=1
2πσ2ex2+y2
2σ2
as follows:
Jxx =Gxx J,Jyy =Gyy J,Jxy =Jyx =Gxy J,
where
Gxx
,
Gxy
and
Gyy
represent the gradient kernel on the horizontal, vertical and diagonal
directions, respectively.
The eigenvalues and eigenvectors of the Hessian matrix
H
are obtained by solving the expression
(Hλhs
jI)ehs
j=0, where λhs
jand ehs
jare the eigenvalue and the eigenvector, respectively, and jis the
index number with value of 1 or 2. The first eigenvector (the one whose corresponding eigenvalue
has the largest absolute value) is the direction of greatest curvature. The other eigenvector (always
orthogonal to the first one) is the direction of least curvature. The corresponding eigenvalues are
the respective amounts of these curvatures. Inspired by earlier work of Frangi et al. [
22
], three
measurement parameters are derived from eigenvalues and eigenvectors of Hessian matrix which
are used in comparison of the pair of images when they have the same content but different contours.
The parameters are:
P1=arctang ehs
2x
ehs
2y!, (3)
P2=log
1+ λhs
2
λhs
1!2
, (4)
P3=log1+λhs
12+λhs
22, (5)
Sensors 2018,18, 429 6 of 19
where
λhs
1
(the largest one) and
λhs
2
are the eigenvalues of the Hessian matrix and
ehs
1=hehs
1x,ehs
1yi
ehs
2=hehs
2x,ehs
2yi
are the related eigenvectors.
P1
,
P2
,
P3
measure the main orientation, the relation of
the two principal curvatures (i.e., each of which measures amount of curvature bending in different
directions), and second order structureness respectively. The dissimilarity of each pair of SC and
CC images (i.e., which are having square grid and the same content but different contours) are
measured by:
D(SC,CC)
Pj=var (PSC
j·PCC
j)
var (PSC
j)var (PCC
j)(6)
where
SC
and
CC
are a pair of images which have the same contents but different contours and the
pair can be the type of SQ or SQ_E images,
j
is the index number (which can have values of 1, 2, or 3),
Pj
is one of the three parameters according to the
j
,
PSC
j
and
PCC
j
are the measurement parameters
Pj
applied on the pair of images of SC and CC, respectively.
4.3. Implementing Second Order Operation to Detect Saddle and Extremum Points
The second order gradient operation has been used to detect spatial critical points; i.e., saddle
and extremum points. The number of critical points in an image depends on the contour shape. Thus,
a pair of images with the same content but different contours can be compared using the detected
critical points in each image. Generally, the critical points are detected when the gradient is zero.
This means the Hessian matrix can be used in this relation; the critical points are found by using
eigenvalues of the Hessian matrix on a square grid. However, on a square grid the zeros of gradient
will in general not coincide with the grid points, but lie somewhere in between them. Kuiiper [
28
]
showed that by converting the square grid to a hexagonal grid; implementing a half pixel shifting
method, it is possible to detect a more accurate number of critical points in a square grid-based image.
In his detection process based on hexagonal grida, each point has six neighbours. The sign of intensity
difference for each of these neighbours with respect to the point itself is determined which results in
its classification into four different points: regular, minimum or maximum (extremum), saddle, and
degenerated saddle point. We used this detection process not only on square based grid images, but
also on hexagonal grid based images to detect saddle and extremum points.
In relation to curvature quantification, pair of SQ, SQ_E, and Hex_E image types are compared in
relation to the detected critical points. The comparison measurement related to the saddle points is
defined as:
Rsaddle =1csd
asd ×csd
|Asd Bsd |×bsd
asd
,Asd Bsd (7)
where
Asd
and
Bsd
are two sets of
asd
and
bsd
saddle points in images A and B respectively (where
A and B are two types of images),
Csd =Asd Bsd
is the set of the saddle points that are on the
same position in the two images with
csd =|Csd |=|Asd Bsd |
points, and
|Asd Bsd |
is the number
of all points which are in sets of
Asd
and
Bsd
. Equation (7) is a normalized nonlinear dissimilarity
measurement function based on the common detected saddle points in the two images; see a typical
of such a function characteristic in top left of Figure 2. The comparison measurement related to the
extremum points is defined as:
Rextremum =1aex
bex ×cex
|Aex Bex |,Bex Aex (8)
where
Aex
and
Bex
are two sets of
aex
and
bex
extremum points where in images A and B, respectively
(where A and B are two types of images),
Cex =Aex Bex
is the set of the extremum points that are on
the same position in the two images with
cex =|Cex |=|Aex Bex |
points, and
|Aex Bex |
is all points
which are in sets of
Aex
and
Bex
. Equation (8) is a normalized nonlinear dissimilarity measurement
function based on the common detected extremum points; see a typical of such a function characteristic
Sensors 2018,18, 429 7 of 19
in top right of Figure 2. The comparison measurement related to the saddle points between two pairs
of images is defined as:
RPsaddl e =1csd
|Asd Bsd |×fsd
|Dsd Esd |×maxasd,bsd
minasd,bsd ×maxdsd,esd
minde x ,esd (9)
where
asd bsd dsd
and
esd
are the numbers of saddle points in sets of
Asd Bsd Dsd
and
Esd
in image A,
B, D and E, respectively. The two pairs of images are (A, D) and (B, E). Each pair of images has the
same type and different type from another pair. The images A and B are the SC images where the
images D and E are CC images. The
Csd =Asd Bsd
and
Fsd =Dsd Esd
are the sets of the saddle
points that are on the same position in each related two images with
cex =|Cex |
and
fex =|Fex|
points
respectively. Equation (9) is a normalized 2D nonlinear dissimilarity measurement function based
on the common detected saddle points in each pair of the images; see a typical of such a function
characteristic in bottom left of Figure 2. The comparison measurement related to the extremum points
between two pairs of images is defined as:
RPextremum =1cex
|Aex Bex |×fex
|Dex Eex |×maxaex ,bex
minae x ,bex ×maxdex ,eex
minde x ,eex (10)
where
aex bex dex
and
eex
are the numbers of extremum points in sets of
Aex Bex Dex
and
Eex
in image
A, B, D and E, respectively. The two pairs of images are (A, D) and (B, E). Each pair of images has the
same type and different type from another pair. The images A and B are the SC images where the
images D and E are CC images. The
Cex =Aex Bex
and
Fe x =Dex Eex
are the sets of the extremum
points that are on the same position in each two related images with
cex =|Cex |
and
fex =|Fex|
points.
Equation (10) is a normalized 2D nonlinear dissimilarity measurement function based on the common
detected saddle points in each pair of the images; see a typical of such a function characteristic in
bottom right of Figure 2.
Figure 2.
Typical characteristics of
Rsd(Csd )
in Equation (7),
Rex (Cex )
in Equation (8),
RPsd(Csd ,Fsd)
in Equation (9),
RPex(Cex ,Fex)
in Equation (10) are shown in top left, top right, bottom left, and bottom
right respectively.
Csd
,
Cex
,
Fsd
, and
Fex
are sets of possible values of
csd
,
cex
,
fsd
, and
fex
, respectively.
Sensors 2018,18, 429 8 of 19
5. Experimental Setup
An image dataset from [
29
] is used for our experiments. The database was used earlier to
investigate human visual preference of curved versus linear edges to find the physical elements in
a visual stimulus which cause like or dislike of objects. The database is composed of 280 pairs of
images, each pair of images have the same semantic meaning and general appearance. We found 51
image pairs had contour curvature differences additionally to the semantic meaning; i.e., straight to
curved line. Then each pair of images with the same semantic content is defined as straight contour
(SC) or curved contour (CC) images. Each of the images has the same resolution, 256
×
256, and in
Uint8 format. Figure 3shows twenty-three pairs of images which are randomly selected from the
51 pairs of images and used for the experiment. The images of the database are generated by computer
graphic tools (i.e., without natural noise). However, they are compressed as jpg format in which noise
is introduced accordingly as it is shown in Figure 3. It is certain that the noise affects the first and
second order derivative operation of Equations (1) and (2). Some smoothing filters are used to reduce
the noise but this significantly change the content of the object (straight or curved lines) in the images
which is not appreciated due to the problematic goal (i.e., to compare straight lines vs curved ones).
Instead of smoothing of the whole image, a template mask for each of the images is used to detect the
background which is irrelevant to the object content, as it is shown in Figure 4. Each mask is generated
automatically using morphological operations. When the images with different pixel structure and
form are compared they are contaminated with the same amount of noise, thus effect of the noise
on the results considered to be insignificant and the comparison results are obtained only from the
masked area on the respective images. The images are converted to grayscale images, and then the fill
factor of images is estimated by the method explained in [
20
]; fill factor value is estimated to be 36%.
The impact of curved versus straight edges on the enriched hexagonal (Hex_E), enriched estimated
square (SQ_E), enriched half pixel shifted (HS_E), and original (SQ) images is evaluated by computing
the first and second order gradient operations; see Sections 4.1 and 4.2, on the images. All images
have the same resolution to ensure that the resolution is not affecting the number of gradients. All the
processing is programmed and implemented by Matlab2017a on a stationary computer with an Intel
i7-6850k CPU (Intel Corporation, California, USA, https://ark.intel.com/products/94188/) and a
32 GB RAM memory to keep the process stable and fast.
Sensors 2018, 18, x 8 of 19
of images and used for the experiment. The images of the database are generated by computer
graphic tools (i.e., without natural noise). However, they are compressed as jpg format in which noise
is introduced accordingly as it is shown in Figure 3. It is certain that the noise affects the first and
second order derivative operation of Equations (1) and (2). Some smoothing filters are used to reduce
the noise but this significantly change the content of the object (straight or curved lines) in the images
which is not appreciated due to the problematic goal (i.e., to compare straight lines vs curved ones).
Instead of smoothing of the whole image, a template mask for each of the images is used to detect the
background which is irrelevant to the object content, as it is shown in Figure 4. Each mask is generated
automatically using morphological operations. When the images with different pixel structure and
form are compared they are contaminated with the same amount of noise, thus effect of the noise on
the results considered to be insignificant and the comparison results are obtained only from the
masked area on the respective images. The images are converted to grayscale images, and then the
fill factor of images is estimated by the method explained in [20]; fill factor value is estimated to be
36%. The impact of curved versus straight edges on the enriched hexagonal (Hex_E), enriched
estimated square (SQ_E), enriched half pixel shifted (HS_E), and original (SQ) images is evaluated
by computing the first and second order gradient operations; see Sections 4.1 and 4.2, on the images.
All images have the same resolution to ensure that the resolution is not affecting the number of
gradients. All the processing is programmed and implemented by Matlab2017a on a stationary
computer with an Intel i7-6850k CPU (Intel Corporation, California, USA,
https://ark.intel.com/products/94188/) and a 32 GB RAM memory to keep the process stable and fast.
Figure 3. Twenty-three pairs of images from the database, where the images in first and third rows
have sharp contours and the images in the second and fourth rows have the curved contours.
Figure 4. One of the database images and its template mask for detecting the background.
6. Results and Discussion
One of the original images and the set of enriched related generated images; hexagonal,
estimated square, and half-pixel shifted images, which were explained in Section 3 are shown in
Figure 5. The images from left to right in the first row are the original image, and the related generated
images. The images in the second row of Figure 5 are the zoomed region of the images (shown as red
square). The generated images show better dynamic range in comparison to the original images, as
it was shown in [21].
Figure 3.
Twenty-three pairs of images from the database, where the images in first and third rows
have sharp contours and the images in the second and fourth rows have the curved contours.
Sensors 2018,18, 429 9 of 19
Sensors 2018, 18, x 8 of 19
of images and used for the experiment. The images of the database are generated by computer
graphic tools (i.e., without natural noise). However, they are compressed as jpg format in which noise
is introduced accordingly as it is shown in Figure 3. It is certain that the noise affects the first and
second order derivative operation of Equations (1) and (2). Some smoothing filters are used to reduce
the noise but this significantly change the content of the object (straight or curved lines) in the images
which is not appreciated due to the problematic goal (i.e., to compare straight lines vs curved ones).
Instead of smoothing of the whole image, a template mask for each of the images is used to detect the
background which is irrelevant to the object content, as it is shown in Figure 4. Each mask is generated
automatically using morphological operations. When the images with different pixel structure and
form are compared they are contaminated with the same amount of noise, thus effect of the noise on
the results considered to be insignificant and the comparison results are obtained only from the
masked area on the respective images. The images are converted to grayscale images, and then the
fill factor of images is estimated by the method explained in [20]; fill factor value is estimated to be
36%. The impact of curved versus straight edges on the enriched hexagonal (Hex_E), enriched
estimated square (SQ_E), enriched half pixel shifted (HS_E), and original (SQ) images is evaluated
by computing the first and second order gradient operations; see Sections 4.1 and 4.2, on the images.
All images have the same resolution to ensure that the resolution is not affecting the number of
gradients. All the processing is programmed and implemented by Matlab2017a on a stationary
computer with an Intel i7-6850k CPU (Intel Corporation, California, USA,
https://ark.intel.com/products/94188/) and a 32 GB RAM memory to keep the process stable and fast.
Figure 3. Twenty-three pairs of images from the database, where the images in first and third rows
have sharp contours and the images in the second and fourth rows have the curved contours.
Figure 4. One of the database images and its template mask for detecting the background.
6. Results and Discussion
One of the original images and the set of enriched related generated images; hexagonal,
estimated square, and half-pixel shifted images, which were explained in Section 3 are shown in
Figure 5. The images from left to right in the first row are the original image, and the related generated
images. The images in the second row of Figure 5 are the zoomed region of the images (shown as red
square). The generated images show better dynamic range in comparison to the original images, as
it was shown in [21].
Figure 4. One of the database images and its template mask for detecting the background.
6. Results and Discussion
One of the original images and the set of enriched related generated images; hexagonal, estimated
square, and half-pixel shifted images, which were explained in Section 3are shown in Figure 5.
The images from left to right in the first row are the original image, and the related generated images.
The images in the second row of Figure 5are the zoomed region of the images (shown as red square).
The generated images show better dynamic range in comparison to the original images, as it was
shown in [21].
Sensors 2018, 18, x 9 of 19
Figure 5. One of original images and its set of generated images.
6.1. First Order Gradient Operation
The sharp transitions in the contour of each objects in the images—the curviness—is quantified
by implementing the first and second order gradient operations on the pair of original images and
their set of generated images; each operation process is explained in more details in Section 4. For the
images on the hexagonal grid; hexagonal and half-pixel shift images, the first order gradients are
computed at six directions, which are 0, 60, 120, 180, 240, and 300 degrees. Due to resolution similarity
of the generated images the on hexagonal grid, their number of pixels and the computed gradient
elements are the same. The top and middle row of Figure 6 shows the sorted first order gradient
values from the generated Hex_E image (i.e., the image shown in Figure 5) in comparison to the
generated HS and HS_E image at 0, 60 and 120 degrees from left to right respectively. The amount of
spreading of the gradient values reveals the correlation between the grid structure of the images. The
more similar the image grids are, the amount of spreading is less. The more densely the points are
distributed, the less variation from the gradient results are expected. Due to the grid similarity of
original images and the SQ_E images, the correlations of sorted gradient values at 0, 45 and 90 degrees
between them are linear which are shown in the bottom row of Figure 6. However, the correlations
of sorted gradient values at 0, 60 and 120 degrees on the pseudo hexagonal grid structure and
hexagonal grid structure are nonlinear and dissimilar; shown in top and middle rows of Figure 6.
Figure 6. Cont.
Figure 5. One of original images and its set of generated images.
6.1. First Order Gradient Operation
The sharp transitions in the contour of each objects in the images—the curviness—is quantified
by implementing the first and second order gradient operations on the pair of original images and
their set of generated images; each operation process is explained in more details in Section 4. For the
images on the hexagonal grid; hexagonal and half-pixel shift images, the first order gradients are
computed at six directions, which are 0, 60, 120, 180, 240, and 300 degrees. Due to resolution similarity
of the generated images the on hexagonal grid, their number of pixels and the computed gradient
elements are the same. The top and middle row of Figure 6shows the sorted first order gradient values
from the generated Hex_E image (i.e., the image shown in Figure 5) in comparison to the generated HS
and HS_E image at 0, 60 and 120 degrees from left to right respectively. The amount of spreading of
the gradient values reveals the correlation between the grid structure of the images. The more similar
the image grids are, the amount of spreading is less. The more densely the points are distributed, the
less variation from the gradient results are expected. Due to the grid similarity of original images and
the SQ_E images, the correlations of sorted gradient values at 0, 45 and 90 degrees between them are
linear which are shown in the bottom row of Figure 6. However, the correlations of sorted gradient
values at 0, 60 and 120 degrees on the pseudo hexagonal grid structure and hexagonal grid structure
are nonlinear and dissimilar; shown in top and middle rows of Figure 6.
Sensors 2018,18, 429 10 of 19
Sensors 2018, 18, x 9 of 19
Figure 5. One of original images and its set of generated images.
6.1. First Order Gradient Operation
The sharp transitions in the contour of each objects in the images—the curviness—is quantified
by implementing the first and second order gradient operations on the pair of original images and
their set of generated images; each operation process is explained in more details in Section 4. For the
images on the hexagonal grid; hexagonal and half-pixel shift images, the first order gradients are
computed at six directions, which are 0, 60, 120, 180, 240, and 300 degrees. Due to resolution similarity
of the generated images the on hexagonal grid, their number of pixels and the computed gradient
elements are the same. The top and middle row of Figure 6 shows the sorted first order gradient
values from the generated Hex_E image (i.e., the image shown in Figure 5) in comparison to the
generated HS and HS_E image at 0, 60 and 120 degrees from left to right respectively. The amount of
spreading of the gradient values reveals the correlation between the grid structure of the images. The
more similar the image grids are, the amount of spreading is less. The more densely the points are
distributed, the less variation from the gradient results are expected. Due to the grid similarity of
original images and the SQ_E images, the correlations of sorted gradient values at 0, 45 and 90 degrees
between them are linear which are shown in the bottom row of Figure 6. However, the correlations
of sorted gradient values at 0, 60 and 120 degrees on the pseudo hexagonal grid structure and
hexagonal grid structure are nonlinear and dissimilar; shown in top and middle rows of Figure 6.
Figure 6. Cont.
Sensors 2018, 18, x 10 of 19
Figure 6. The gradients correlation between the Hex_E image (second column) and HS_E image
(fourth column) shown in Figure 5 at directions of 0, 60 and 120 degrees (top row). And the gradients
correlation between the SQ image (first column) and SQ_E image (third column) shown in Figure 5
at directions of 0, 45 and 90 degrees (bottom row).
The same is the correlation of sorted gradient values at 0 degree between the SQ image grid
structure to both the pseudo hexagonal grid structure and hexagonal grid structure which are shown
in Figure 7 on the first and third columns from left respectively; i.e., the correlation is in each case
nonlinear and dissimilar.
Figure 7 shows that the gradient results from the four types of generated images in comparison
to the original SQ image is different from each other; especially the second plot from left. This is
because the grids in HS, HS_E and Hex_E images are more alike to each other and more different
from the square grid (i.e., the grid of SQ and SQ_E images). The similarity/dissimilarity of each two
grid structures are possible to visualize; as they are shown in Figures 6 and 7. However, to quantify
such a similarity/dissimilarity the first order gradient operation can be used as it is described in
Section 4.1. Accordingly, the covariance of the gradient values of each two images are computed
where each compared two images have the same contents but different grid structures. Then the
eigenvalues and eigenvectors of each covariance matrix is computed using singular value
decomposition (SVD) method.
Figure 7. The gradient cross comparison at 0 degree between one of the original images and its
generated images: Hex_E image (first), SQ_E image (second), HS_E image (third), and HS image
(fourth).
The first eigenvalues, which are also the largest ones, of the covariance matrix between each pair
of the original images and its set of generated images are shown in top of Figure 8, and the second
eigenvalues are shown in the bottom of Figure 8. The blue, red, green and black lines represent the
first or second eigenvalues of the covariance matrixes with respect to the four types of images
presented in Section 3, respectively. The continuous lines and dash lines represent the computed first
and second eigenvalues with respect to the original CC and SC images, respectively. Table 1 shows
Figure 6.
The gradients correlation between the Hex_E image (second column) and HS_E image
(fourth column) shown in Figure 5at directions of 0, 60 and 120 degrees (top row). And the gradients
correlation between the SQ image (first column) and SQ_E image (third column) shown in Figure 5at
directions of 0, 45 and 90 degrees (bottom row).
The same is the correlation of sorted gradient values at 0 degree between the SQ image grid
structure to both the pseudo hexagonal grid structure and hexagonal grid structure which are shown
in Figure 7on the first and third columns from left respectively; i.e., the correlation is in each case
nonlinear and dissimilar.
Figure 7shows that the gradient results from the four types of generated images in comparison
to the original SQ image is different from each other; especially the second plot from left. This is
because the grids in HS, HS_E and Hex_E images are more alike to each other and more different from
the square grid (i.e., the grid of SQ and SQ_E images). The similarity/dissimilarity of each two grid
structures are possible to visualize; as they are shown in Figures 6and 7. However, to quantify such a
similarity/dissimilarity the first order gradient operation can be used as it is described in Section 4.1.
Accordingly, the covariance of the gradient values of each two images are computed where each
Sensors 2018,18, 429 11 of 19
compared two images have the same contents but different grid structures. Then the eigenvalues and
eigenvectors of each covariance matrix is computed using singular value decomposition (SVD) method.
Sensors 2018, 18, x 10 of 19
Figure 6. The gradients correlation between the Hex_E image (second column) and HS_E image
(fourth column) shown in Figure 5 at directions of 0, 60 and 120 degrees (top row). And the gradients
correlation between the SQ image (first column) and SQ_E image (third column) shown in Figure 5
at directions of 0, 45 and 90 degrees (bottom row).
The same is the correlation of sorted gradient values at 0 degree between the SQ image grid
structure to both the pseudo hexagonal grid structure and hexagonal grid structure which are shown
in Figure 7 on the first and third columns from left respectively; i.e., the correlation is in each case
nonlinear and dissimilar.
Figure 7 shows that the gradient results from the four types of generated images in comparison
to the original SQ image is different from each other; especially the second plot from left. This is
because the grids in HS, HS_E and Hex_E images are more alike to each other and more different
from the square grid (i.e., the grid of SQ and SQ_E images). The similarity/dissimilarity of each two
grid structures are possible to visualize; as they are shown in Figures 6 and 7. However, to quantify
such a similarity/dissimilarity the first order gradient operation can be used as it is described in
Section 4.1. Accordingly, the covariance of the gradient values of each two images are computed
where each compared two images have the same contents but different grid structures. Then the
eigenvalues and eigenvectors of each covariance matrix is computed using singular value
decomposition (SVD) method.
Figure 7. The gradient cross comparison at 0 degree between one of the original images and its
generated images: Hex_E image (first), SQ_E image (second), HS_E image (third), and HS image
(fourth).
The first eigenvalues, which are also the largest ones, of the covariance matrix between each pair
of the original images and its set of generated images are shown in top of Figure 8, and the second
eigenvalues are shown in the bottom of Figure 8. The blue, red, green and black lines represent the
first or second eigenvalues of the covariance matrixes with respect to the four types of images
presented in Section 3, respectively. The continuous lines and dash lines represent the computed first
and second eigenvalues with respect to the original CC and SC images, respectively. Table 1 shows
Figure 7.
The gradient cross comparison at 0 degree between one of the original images and
its generated images: Hex_E image (first), SQ_E image (second), HS_E image (third), and HS
image (fourth).
The first eigenvalues, which are also the largest ones, of the covariance matrix between each pair
of the original images and its set of generated images are shown in top of Figure 8, and the second
eigenvalues are shown in the bottom of Figure 8. The blue, red, green and black lines represent the first
or second eigenvalues of the covariance matrixes with respect to the four types of images presented in
Section 3, respectively. The continuous lines and dash lines represent the computed first and second
eigenvalues with respect to the original CC and SC images, respectively. Table 1shows the summary
of the comparison results of the two figures in Figure 8. The different properties among the types of
images are caused by the diversity of their grid structure, pixel form or fill factor value. ‘Yes’ and
‘No’ in the table represent similarity and dissimilarity of such a property in relation between each
generated image to the SQ image. The values in the last four columns of Table 1are the sums of
the first and second eigenvalues of the respected image type shown in Figure 8. In the table, the
increase of the first or second eigenvalue indicates the increase of similarity or dissimilarity between
the generated image and SQ image respectively. The SQ_E and HS images in relation to the SQ image
show higher similarity than the other image types; see the first eigenvalue results in Table 1and top
figure in Figure 8. The comparison of these two types of images show that the grid structure is more
important than pixel form and fill factor value to cause differences between them. The Hex_E and
HS_E images in relation to the SQ image show higher dissimilarity, respectively. The comparison of
these two types of images show that when the grid structures are the same the pixel form is more
important than fill factor value to cause differences between them. The results show that the choice
of grid structure, pixel form, and fill factor value are respectively important in generation of a new
type of images. Here we should note that these three properties are not quite independent from each
other. In Table 1, the results related to SC and CC for all four types of images show that they are clearly
distinctive. However, the detail comparison of SC and CC in Figure 8show that it is not possible to
have a clear conclusion between SC and CC by first order operation; due to the results variation.
Sensors 2018,18, 429 12 of 19
Sensors 2018, 18, x 11 of 19
the summary of the comparison results of the two figures in Figure 8. The different properties among
the types of images are caused by the diversity of their grid structure, pixel form or fill factor value.
‘Yes’ and ‘No’ in the table represent similarity and dissimilarity of such a property in relation between
each generated image to the SQ image. The values in the last four columns of Table 1 are the sums of
the first and second eigenvalues of the respected image type shown in Figure 8. In the table, the
increase of the first or second eigenvalue indicates the increase of similarity or dissimilarity between
the generated image and SQ image respectively. The SQ_E and HS images in relation to the SQ image
show higher similarity than the other image types; see the first eigenvalue results in Table 1 and top
figure in Figure 8. The comparison of these two types of images show that the grid structure is more
important than pixel form and fill factor value to cause differences between them. The Hex_E and
HS_E images in relation to the SQ image show higher dissimilarity, respectively. The comparison of
these two types of images show that when the grid structures are the same the pixel form is more
important than fill factor value to cause differences between them. The results show that the choice
of grid structure, pixel form, and fill factor value are respectively important in generation of a new
type of images. Here we should note that these three properties are not quite independent from each
other. In Table 1, the results related to SC and CC for all four types of images show that they are
clearly distinctive. However, the detail comparison of SC and CC in Figure 8 show that it is not
possible to have a clear conclusion between SC and CC by first order operation; due to the results
variation.
Figure 8. The first (Top) and second (Bottom) eigenvalues of the gradient values in a cross comparison
between original image and four types of generated images. The blue lines represent the Hex_E
images; the red lines represent the SQ_E images, the green lines represent the HS_E and the black
lines represent the HS images. The continuous lines and dash lines represent the CC and SC images
respectively.
Figure 8.
The first (
Top
) and second (
Bottom
) eigenvalues of the gradient values in a cross comparison
between original image and four types of generated images. The blue lines represent the Hex_E images;
the red lines represent the SQ_E images, the green lines represent the HS_E and the black lines represent
the HS images. The continuous lines and dash lines represent the CC and SC images respectively.
Table 1. The summery of the comparison results of the two figures in Figure 8.
Grid Structure Pixel Form Fill Factor First Eigenvalue Second Eigenvalue
SC CC SC CC
SQ_E Yes Yes No 272.98 277.86 6.43 8.51
HS No Yes Yes 260.66 264.52 60.12 61.29
HS_E No Yes No 210.19 212.98 106.33 107.89
Hex_E No No No 187.28 190.26 112.32 114.14
6.2. Hessian Matrix on SQ, and SQ_E Images
Tables 2and 3show the measured
P1
,
P2
, and
P3
parameters (Equations (3)–(5)) between SC
and CC images having SQ or SQ_E image type using Hessian matrix; for more detail see Section 4.2.
The bold result values in the tables show the higher one in comparison of each pair of SC and
CC images. In Table 2, the results related to the CC images have higher values than SC results.
D(SC,CC)
Pj(j=1, 2, 3)
indicates that the contours in the CC images are distinctively different (have
more curviness) than the ones in SC images. Table 3shows the similar results, that the results values
are higher in majority of CC images in respect to SC results; 94% of measurement values. Table 4shows
the dissimilarity measurement of
D(SC,CC)
Pj(j=1, 2, 3)
in Equation (7). The comparison between SQ
and SQ_E images using
D(SC,CC)
P1
and
D(SC,CC)
P2
values is not conclusive due to undistinctive result
values. However, using the
D(SC,CC)
P3
values the comparison is possible due to clear distinctive results.
According to this comparison the SQ_E images have in average 62.5% better performance than SQ
images. As far as the dissimilarity of the
D(SC,CC)
P3
measures the second order structureness, we can
conclude that the SQ_E image type far better can perform in detection of curviness than SQ image type.
Sensors 2018,18, 429 13 of 19
Table 2. The measured P1,P2,P3parameters of SQ images using Hessian matrix.
Image Index
P1P2P3
SC CC SC CC SC CC
1 49.047 50.953 44.121 55.879 45.163 54.837
2 48.41 51.59 48.407 51.593 48.601 51.399
3 49.468 50.532 23.679 76.321 28.143 71.857
4 49.42 50.58 32.887 67.113 46.829 53.171
5 47.206 52.794 38.418 61.582 37.502 62.498
6 49.914 50.086 46.71 53.29 47.856 52.144
7 47.701 52.299 36.29 63.71 46.749 53.251
8 49.791 50.209 14.97 85.03 46.549 53.451
9 49.575 50.425 15.392 84.608 48.163 51.837
10 49.408 50.592 16.171 83.829 48.056 51.944
11 49.011 50.989 36.452 63.548 49.222 50.778
12 49.075 50.925 14.809 85.191 43.015 56.985
13 48.724 51.276 16.613 83.387 45.124 54.876
14 48.163 51.837 26.797 73.203 44.71 55.29
15 48.899 51.101 16.959 83.041 46.383 53.617
16 49.167 50.833 18.203 81.797 46.13 53.87
17 49.568 50.432 25.92 74.08 45.673 54.327
18 48.741 51.259 37.231 62.769 45.062 54.938
19 49.474 50.526 28.151 71.849 48.687 51.313
20 49.651 50.349 14.972 85.028 44.699 55.301
21 48.758 51.242 44.467 55.533 42.291 57.709
22 48.834 51.166 46.471 53.529 45.996 54.004
23 48.612 51.388 17.293 82.707 46.214 53.786
Table 3. The measured P1,P2,P3values of SQ_E images using Hessian matrix.
Image Index
P1P2P3
SC CC SC CC SC CC
1 49.353 50.647 44.349 55.651 44.976 55.024
2 49.337 50.663 48.76 51.24 48.343 51.657
351.625 48.375 66.592 33.408 30.413 69.587
450.057 49.943 30.517 69.483 43.874 56.126
5 47.565 52.435 37.025 62.975 35.033 64.967
650.474 49.526 48.374 51.626 47.124 52.876
7 48.212 51.788 36.412 63.588 44.62 55.38
8 49.627 50.373 15.485 84.515 41.143 58.857
9 49.341 50.659 15.857 84.143 47.783 52.217
10 49.185 50.815 16.628 83.372 47.98 52.02
11 49.821 50.179 34.757 65.243 41.404 58.596
12 48.961 51.039 15.126 84.874 43.718 56.282
13 48.858 51.142 19.546 80.454 45.803 54.197
14 48.583 51.417 32.959 67.041 45.682 54.318
15 49.016 50.984 17.626 82.374 46.96 53.04
16 49.262 50.738 23.977 76.023 46.319 53.681
17 49.564 50.436 25.45 74.55 44.955 55.045
18 48.858 51.142 35.824 64.176 44.511 55.489
19 49.473 50.527 33.351 66.649 47.252 52.748
20 49.399 50.601 19.883 80.117 44.592 55.408
21 48.437 51.563 43.321 56.679 41.176 58.824
22 49.108 50.892 46.233 53.767 44.769 55.231
23 48.605 51.395 17.534 82.466 46.788 53.212
Sensors 2018,18, 429 14 of 19
Table 4.
The dissimilarity measurement of
D(SC,CC)
Pj(j=1, 2, 3)
for SQ and SQ_E image types using
Hessian matrix.
Image Index SQ SQ_E
DSC,CC
P1
DSC,CC
P2
DSC,CC
P3
DSC,CC
P1
DSC,CC
P2
DSC,CC
P3
1 9.475 12.085 19.567 8.504 11.644 31.7
2 5.942 10.548 10.423 5.117 10.378 19.319
3 11.581 13.929 18.852 10.244 13.112 28.209
4 13.036 11.043 16.944 12.121 11.14 27.646
5 6.784 8.598 7.894 6.535 8.884 14.992
6 8.807 11.69 14.253 7.345 11.121 23.045
7 9.523 9.535 12.147 8.68 9.189 20.22
8 5.967 9.278 9.236 5.789 9.207 17.652
9 6.01 10.419 8.565 6.001 10.462 15.91
10 5.019 9.401 9.651 4.979 9.293 18.443
11 12.695 9.951 11.685 11.886 9.932 19.111
12 7.973 10.289 11.85 7.81 10.494 18.112
13 6.344 13.968 13.87 6.164 12.58 23.444
14 3.615 11.937 12.011 3.496 9.546 19.285
15 10.371 11.426 11.641 9.43 11.762 15.419
16 9.994 15.302 17.45 9.42 14.25 25.318
17 8.455 14.049 17.837 8.149 14.028 28.25
18 6.252 12.369 14.174 6.28 12.709 19.231
19 7.297 11.696 11.081 6.889 9.639 15.6
20 8.232 15.834 17.783 7.755 14.707 29.075
21 6.396 8.167 7.962 6.236 8.013 15.206
22 11.36 10.291 16.176 10.966 10.115 28.117
23 5.759 10.059 9.716 5.709 10.14 17.785
6.3. Saddle and Extremum Points
The detected saddle and extremum points from the second order gradient operation on the same
HS, HS_E and Hex_E images shown in Figure 5are shown in Figure 9, respectively. The HS and HS_E
images are generated from its corresponding SQ and SQ_E images by converting the square grid to a
hexagonal grid, implementing the half pixel shifting method proposed in Section 3.3. As it is discussed
in Section 4.3, in comparison to SQ and SQ_E images the HS and HS_E images are better at detecting
the critical points. The number of saddle and extremum points in each SC or CC image having Hex_E,
SQ_E and SQ image types are shown in Figure 10. This shows that in 74% of pairs of CC and SC
images of each image type, CC images have detected more critical points than SC images. However,
in the figure the comparison results among Hex_E, SQ and SQ_E image types are still undistinctive.
The top and middle figures in Figure 11 show that the number of common saddle and extremum
points respectively, and the bottom figure shows the total number of the critical points. The common
points are those points which are in the same position in each pair of SC and CC images. The results
indicate that the SQ image type detects more common saddle points; in 87% of SC and CC image
pairs, and Hex_E image type detects more extremum points; in 74% of image pairs. Due to that the
critical points in SQ, SQ_E and Hex_E images are detected in the hexagonal grid; see Section 4.3, the
difference of the results between image types is affected by having different pixel form and fill factor
value. The results values between SQ_E and Hex_E image type in Figure 11 are close to each other;
indicating that the pixel form has more effect on the second order gradient than the fill factor.
Sensors 2018,18, 429 15 of 19
Sensors 2018, 18, x 14 of 19
6.3. Saddle and Extremum Points
The detected saddle and extremum points from the second order gradient operation on the same
HS, HS_E and Hex_E images shown in Figure 5 are shown in Figure 9, respectively. The HS and
HS_E images are generated from its corresponding SQ and SQ_E images by converting the square
grid to a hexagonal grid, implementing the half pixel shifting method proposed in Section 3.3. As it
is discussed in Section 4.3, in comparison to SQ and SQ_E images the HS and HS_E images are better
at detecting the critical points. The number of saddle and extremum points in each SC or CC image
having Hex_E, SQ_E and SQ image types are shown in Figure 10. This shows that in 74% of pairs of
CC and SC images of each image type, CC images have detected more critical points than SC images.
However, in the figure the comparison results among Hex_E, SQ and SQ_E image types are still
undistinctive. The top and middle figures in Figure 11 show that the number of common saddle and
extremum points respectively, and the bottom figure shows the total number of the critical points.
The common points are those points which are in the same posit ion i n each pair of SC a nd CC image s.
The results indicate that the SQ image type detects more common saddle points; in 87% of SC and
CC image pairs, and Hex_E image type detects more extremum points; in 74% of image pairs. Due to
that the critical points in SQ, SQ_E and Hex_E images are detected in the hexagonal grid; see Section
4.3, the difference of the results between image types is affected by having different pixel form and
fill factor value. The results values between SQ_E and Hex_E image type in Figure 11 are close to
each other; indicating that the pixel form has more effect on the second order gradient than the fill
factor.
Figure 9. The detected saddle and extremum points on HS, HS_E and Hex_E images.
Figure 9. The detected saddle and extremum points on HS, HS_E and Hex_E images.
Sensors 2018, 18, x 15 of 19
Figure 10. The number of the saddle and extremum points in each SC and CC image having Hex_E,
SQ_E and SQ image types.
Figure 11. The number of the common saddle (Top), extremum points (Middle) and the critical points
(Bottom) between SC and CC image pairs having Hex_E, SQ_E and SQ image types.
The normalized nonlinear dissimilarity measurement values of  and  for SC
and CC from three pairs of comparisons between SQ_E, SQ and Hex_E images are computed by
Equations (7) and (8) and shown in Table 5, where the higher value represents the larger dissimilarity
of the contours between respected image types. For each pair of SC and CC, the larger value of 
or  is shown bolded. The correlations of the three pairs of comparisons in Table 5 are
shown in Figure 12, where the points of four colors of blue, red, green and black represent the
Figure 10.
The number of the saddle and extremum points in each SC and CC image having Hex_E,
SQ_E and SQ image types.
Sensors 2018, 18, x 15 of 19
Figure 10. The number of the saddle and extremum points in each SC and CC image having Hex_E,
SQ_E and SQ image types.
Figure 11. The number of the common saddle (Top), extremum points (Middle) and the critical points
(Bottom) between SC and CC image pairs having Hex_E, SQ_E and SQ image types.
The normalized nonlinear dissimilarity measurement values of  and  for SC
and CC from three pairs of comparisons between SQ_E, SQ and Hex_E images are computed by
Equations (7) and (8) and shown in Table 5, where the higher value represents the larger dissimilarity
of the contours between respected image types. For each pair of SC and CC, the larger value of 
or  is shown bolded. The correlations of the three pairs of comparisons in Table 5 are
shown in Figure 12, where the points of four colors of blue, red, green and black represent the
Figure 11.
The number of the common saddle (
Top
), extremum points (
Middle
) and the critical points
(Bottom) between SC and CC image pairs having Hex_E, SQ_E and SQ image types.
Sensors 2018,18, 429 16 of 19
The normalized nonlinear dissimilarity measurement values of
Rsaddle
and
Rextremum
for SC and
CC from three pairs of comparisons between SQ_E, SQ and Hex_E images are computed by Equations
(7) and (8) and shown in Table 5, where the higher value represents the larger dissimilarity of the
contours between respected image types. For each pair of SC and CC, the larger value of
Rsaddle
or
Rextremum is shown bolded. The correlations of the three pairs of comparisons in Table 5are shown in
Figure 12, where the points of four colors of blue, red, green and black represent the correspondent
values of
Rsaddle
and
Rextremum
for each pair of SC and CC images respectively. In the figure, the three
axes represent the three pairs of comparison between image types. The results of Table 5show that the
Rsaddle
values for SC and CC images are close to each other, which is also verified in Figure 12; i.e., the
blue points are mixed with red points. On the contrast, the Rextremum values for SC and CC images in
Table 5are distinctive which is shown in Figure 12 by the green and black points. According to these
results our conclusion is that only the extremum points can be used for quantifying the curviness;
i.e., to distinguish a CC image from a SC image. Comparing the two comparisons of SQ_E & SQ
and Hex_E & SQ_E and considering the three properties among the type of images; see Table 1, the
measured dissimilarity in Table 5for each of the comparisons is caused by fill factor and pixel form
respectively. This is due to that SQ_E and SQ images are converted to hexagonal grid by half pixel
shift for detection of critical points. Thus the grid structures of all three types of images in the two
comparisons are the same. According to Table 5, the dissimilarity comparison values of the Hex_E &
SQ_E are higher than the SQ_E & SQ, which shows that the pixel form is more important property
than the fill factor to cause dissimilarity between images. This is consistent with the results which are
presented in Table 1. Thus, from the results we concluded that the importance of the three properties
from high to low is the grid structure, the pixel form and the fill factor, respectively.
Table 5. The Rsaddl e and Rex tremum values for SC and CC between SQ_E, SQ and Hex_E image types.
Image Index
SQ_E & SQ Hex_E & SQ_E Hex_E & SQ
Rsaddle Rextremum Rsaddle Rextremum Rsaddle Rextremum
SC CC SC CC SC CC SC CC SC CC SC CC
10.865 0.754 0.299 0.812 0.940 0.934 0.769 0.814 0.946 0.916 0.596 0.922
20.700 0.690 0.394 0.499 0.912 0.922 0.710 0.75 0.909 0.918 0.688 0.781
30.866 0.836 0.343 0.868 0.941 0.928 0.756 0.866 0.957 0.954 0.610 0.954
40.857 0.820 0.329 0.828 0.948 0.940 0.739 0.811 0.953 0.938 0.620 0.926
50.690 0.538 0.328 0.385 0.896 0.890 0.701 0.778 0.923 0.910 0.668 0.801
60.786 0.756 0.277 0.915 0.929 0.919 0.767 0.841 0.892 0.893 0.538 0.967
70.836 0.731 0.340 0.792 0.931 0.930 0.735 0.802 0.933 0.899 0.638 0.909
8 0.528 0.542 0.342 0.510 0.876 0.867 0.680 0.747 0.871 0.896 0.649 0.797
90.886 0.769 0.515 0.792 0.934 0.933 0.763 0.821 0.943 0.902 0.698 0.896
10 0.631 0.524 0.340 0.865 0.894 0.919 0.662 0.786 0.908 0.770 0.613 0.926
11 0.807 0.824 0.292 0.729 0.922 0.932 0.707 0.804 0.948 0.939 0.623 0.878
12 0.877 0.687 0.273 0.898 0.926 0.940 0.734 0.786 0.94 0.805 0.577 0.942
13 0.782 0.757 0.260 0.749 0.928 0.939 0.739 0.798 0.921 0.907 0.608 0.886
14 0.855 0.828 0.324 0.871 0.940 0.949 0.741 0.831 0.943 0.917 0.602 0.928
15 0.893 0.877 0.321 0.830 0.930 0.947 0.725 0.809 0.950 0.944 0.583 0.929
16 0.790 0.744 0.114 0.940 0.926 0.938 0.751 0.798 0.866 0.833 0.478 0.964
17 0.757 0.698 0.149 0.900 0.933 0.942 0.737 0.783 0.865 0.818 0.530 0.944
18 0.877 0.654 0.280 0.921 0.934 0.938 0.729 0.774 0.947 0.784 0.583 0.954
19 0.765 0.671 0.193 0.890 0.925 0.940 0.731 0.772 0.861 0.810 0.522 0.937
20 0.833 0.803 0.187 0.906 0.941 0.948 0.761 0.807 0.908 0.889 0.537 0.952
21 0.683 0.709 0.308 0.704 0.879 0.910 0.692 0.868 0.920 0.932 0.620 0.915
22 0.813 0.784 0.349 0.828 0.941 0.951 0.775 0.822 0.937 0.899 0.673 0.905
23 0.656 0.759 0.319 0.671 0.898 0.932 0.685 0.796 0.913 0.931 0.620 0.859
Sensors 2018,18, 429 17 of 19
Sensors 2018, 18, x 17 of 19
Figure 12. The co-relation of  and  values for SC and CC images from Table 5.
Table 6. The  and  values of three comparisons computed by the Equation (9)
and Equation (10).
Image
Index
SQ_E vs. SQ Hex_E vs. SQ_E Hex_E vs. SQ
     
1 0.885 0.505 0.963 0.945 0.959 0.850
2 0.776 0.630 0.949 0.919 0.947 0.907
3 0.907 0.567 0.961 0.942 0.974 0.843
4 0.899 0.518 0.967 0.934 0.969 0.841
5 0.714 0.525 0.933 0.901 0.950 0.886
6 0.850 0.516 0.954 0.945 0.932 0.819
7 0.868 0.555 0.958 0.932 0.951 0.863
8 0.638 0.567 0.919 0.894 0.926 0.875
9 0.893 0.727 0.960 0.948 0.954 0.897
10 0.708 0.555 0.942 0.927 0.913 0.856
11 0.878 0.538 0.955 0.920 0.967 0.845
12 0.880 0.459 0.960 0.942 0.939 0.825
13 0.845 0.492 0.960 0.939 0.948 0.858
14 0.900 0.608 0.967 0.953 0.959 0.862
15 0.929 0.512 0.963 0.927 0.969 0.826
16 0.847 0.260 0.960 0.950 0.906 0.738
17 0.821 0.323 0.963 0.943 0.902 0.791
18 0.875 0.407 0.961 0.938 0.940 0.802
19 0.818 0.379 0.960 0.936 0.898 0.781
20 0.882 0.359 0.967 0.950 0.938 0.794
21 0.785 0.560 0.935 0.920 0.955 0.864
22 0.868 0.623 0.967 0.954 0.951 0.880
23 0.794 0.569 0.948 0.913 0.953 0.860
7. Conclusions
In this paper, we present a software-based method to generate images with hexagonal pixel form
on a hexagonal sensor grid. Each original rectangular pixel form is deformed to a hexagonal one
using modelling of the incident photons onto the sensor surface. Four different image sensor forms
and structures, including the proposed method, are evaluated by measuring their ability to detect
curviness. We introduce a method for curviness quantification by comparison of the sharp transitions
in contour of all correspondent objects in pair of images which have exact similar contents but two
different contours. The quantification measurements are achieved by implementing first and second
order gradient operations in form of several introduced and defined dissimilarity parameters. We
show how first and second gradient operations, Hessian matrix computation, and measurement of
the dissimilarity parameters can be implemented on both square and hexagonal grid structures. We
Figure 12. The co-relation of Rs addl e and Rextremum values for SC and CC images from Table 5.
Table 6shows the results of the
RPsaddl e
and
RPextremum
values of three comparisons computed by
the Equation (9) and Equation (10). The values in each column represent dissimilarity values based on
the common detected saddle or extremum points in each pair of SC and CC images which is used to
compare two different image types. The dissimilarity values in the table are not consistent for each
two image types in comparison to previous results, indicating that processing SC and CC together is
not an adequate way to quantify the difference between two image types. By combining the results
in Tables 5and 6, we conclude that according to all the measurement the Hex_E image type has the
largest dissimilarity in comparison to the other image types, which means the Hex_E is the best image
type among our tested image types for quantifying the curviness of a contour.
Table 6.
The
RPsaddl e
and
RPextremum
values of three comparisons computed by the Equation (9) and
Equation (10).
Image Index SQ_E vs. SQ Hex_E vs. SQ_E Hex_E vs. SQ
RPsaddle RPextremum RPsaddle RPextremum RPsaddle RPextremum
1 0.885 0.505 0.963 0.945 0.959 0.850
2 0.776 0.630 0.949 0.919 0.947 0.907
3 0.907 0.567 0.961 0.942 0.974 0.843
4 0.899 0.518 0.967 0.934 0.969 0.841
5 0.714 0.525 0.933 0.901 0.950 0.886
6 0.850 0.516 0.954 0.945 0.932 0.819
7 0.868 0.555 0.958 0.932 0.951 0.863
8 0.638 0.567 0.919 0.894 0.926 0.875
9 0.893 0.727 0.960 0.948 0.954 0.897
10 0.708 0.555 0.942 0.927 0.913 0.856
11 0.878 0.538 0.955 0.920 0.967 0.845
12 0.880 0.459 0.960 0.942 0.939 0.825
13 0.845 0.492 0.960 0.939 0.948 0.858
14 0.900 0.608 0.967 0.953 0.959 0.862
15 0.929 0.512 0.963 0.927 0.969 0.826
16 0.847 0.260 0.960 0.950 0.906 0.738
17 0.821 0.323 0.963 0.943 0.902 0.791
18 0.875 0.407 0.961 0.938 0.940 0.802
19 0.818 0.379 0.960 0.936 0.898 0.781
20 0.882 0.359 0.967 0.950 0.938 0.794
21 0.785 0.560 0.935 0.920 0.955 0.864
22 0.868 0.623 0.967 0.954 0.951 0.880
23 0.794 0.569 0.948 0.913 0.953 0.860
Sensors 2018,18, 429 18 of 19
7. Conclusions
In this paper, we present a software-based method to generate images with hexagonal pixel
form on a hexagonal sensor grid. Each original rectangular pixel form is deformed to a hexagonal
one using modelling of the incident photons onto the sensor surface. Four different image sensor
forms and structures, including the proposed method, are evaluated by measuring their ability to
detect curviness. We introduce a method for curviness quantification by comparison of the sharp
transitions in contour of all correspondent objects in pair of images which have exact similar contents
but two different contours. The quantification measurements are achieved by implementing first and
second order gradient operations in form of several introduced and defined dissimilarity parameters.
We show how first and second gradient operations, Hessian matrix computation, and measurement of
the dissimilarity parameters can be implemented on both square and hexagonal grid structures. We
pay special attention in detection of critical points (i.e., saddle and extremum points) using different
image types.
The grid structure, pixel form and fill factor are proposed for representing the three major
properties of the sensor characteristics and the results indicate that the grid structure is the most
important one that makes difference between the type of images, and the pixel form is the second
important one. The results of curviness quantification indicate that the detection of extremum points
can be used to highly distinct CC from SC images. We show that enriched hexagonal image (i.e., Hex_E)
is best in detection of curviness; according to its curviness measurement results, in comparison to the
other tested image types. In the future, we intend to study other grid structures and pixel forms.
Author Contributions:
Wei Wen and Siamak Khatibi equally contributed to the textual content of this article.
Wei Wen and Siamak Khatibi conceived and designed the methodology and experiments. Wei Wen performed
the experiments. Wei Wen and Siamak Khatibi analyzed the data. Wei Wen and Siamak Khatibi wrote the
paper together.
Conflicts of Interest: The authors declare no conflict of interest.
References
1.
Wen, W.; Khatibi, S. Novel Software-Based Method to Widen Dynamic Range of CCD Sensor Images.
In Proceedings of the International Conference on Image and Graphics, Tianjin, China, 13–16 August 2015;
pp. 572–583.
2.
Wen, W.; Khatibi, S. Back to basics: Towards novel computation and arrangement of spatial sensory in
images. Acta Polytech. 2016,56, 409–416. [CrossRef]
3.
He, X.; Jia, W. Hexagonal Structure for Intelligent Vision. In Proceedings of the 2005 International Conference
on Information and Communication Technologies, Karachi, Pakistan, 27–28 August 2005; pp. 52–64.
4. Horn, B. Robot Vision; MIT Press: Cambridge, MA, USA, 1986.
5.
Yabushita, A.; Ogawa, K. Image reconstruction with a hexagonal grid. In Proceedings of the 2002 IEEE
Nuclear Science Symposium Conference Record, Norfolk, VA, USA, 10–16 November 2002; Volume 3,
pp. 1500–1503.
6.
Staunton, R.C.; Storey, N. A comparison between square and hexagonal sampling methods for pipeline
image processing. In Proceedings of the 1989 Symposium on Visual Communications, Image Processing, and
Intelligent Robotics Systems, Philadelphia, PA, USA, 1–3 November 1989; International Society for Optics
and Photonics: Bellingham, WA, USA, 1990; pp. 142–151.
7.
Singh, I.; Oberoi, A.; Oberoi, M. Performance Evaluation of Edge Detection Techniques for Square, Hexagon
and Enhanced Hexagonal Pixel Images. Int. J. Comput. Appl. 2015,121. [CrossRef]
8.
Gardiner, B.; Coleman, S.A.; Scotney, B.W. Multiscale Edge Detection Using a Finite Element Framework for
Hexagonal Pixel-Based Images. IEEE Trans. Image Process. 2016,25, 1849–1861. [CrossRef] [PubMed]
9.
Burdescu, D.; Brezovan, M.; Ganea, E.; Stanescu, L. New Algorithm for Segmentation of Images Represented
as Hypergraph Hexagonal-Grid. In Pattern Recognition and Image Analysis; Springer: Berlin/Heidelberg,
Germany, 2011; pp. 395–402.
Sensors 2018,18, 429 19 of 19
10.
Argyriou, V. Sub-Hexagonal Phase Correlation for Motion Estimation. IEEE Trans. Image Process.
2011
,20,
110–120. [CrossRef] [PubMed]
11.
Senthilnayaki, M.; Veni, S.; Kutty, K.A.N. Hexagonal Pixel Grid Modeling for Edge Detection and Design
of Cellular Architecture for Binary Image Skeletonization. In Proceedings of the 2006 Annual IEEE India
Conference, New Delhi, India, 15–17 September 2006; pp. 1–6.
12.
Linnér, E.; Strand, R. Comparison of restoration quality on square and hexagonal grids using normalized
convolution. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012),
Tsukuba, Japan, 11–15 November 2012; pp. 3046–3049.
13.
Jeevan, K.M.; Krishnakumar, S. An Algorithm for the Simulation of Pseudo Hexagonal Image Structure
Using MATLAB. Int. J. Image Graph. Signal Process. 2016,8, 57–63.
14.
He, X. 2D-Object Recognition with Spiral Architecture. Ph.D. Thesis, University of Technology, Sydney,
Australia, 1999.
15.
Coleman, S.; Gardiner, B.; Scotney, B. Adaptive tri-direction edge detection operators based on the spiral
architecture. In Proceedings of the 2010 17th IEEE International Conference on Image Processing (ICIP),
Hong Kong, China, 26–29 September 2010; pp. 1961–1964.
16.
Wu, Q.; He, S.; Hintz, T. Virtual Spiral Architecture. In Proceedings of the International Conference on
Parallel and Distributed Processing Techniques and Applications, Las Vegas, NV, USA, 21–24 June 2004;
CSREA Press: Las Vegas, NV, USA, 2004.
17.
Her, I.; Yuan, C.-T. Resampling on a pseudohexagonal grid. CVGIP Graph. Models Image Process.
1994
,56,
336–347. [CrossRef]
18.
Van De Ville, D.; Philips, W.; Lemahieu, I. Least-squares spline resampling to a hexagonal lattice.
Signal Process. Image Commun. 2002,17, 393–408. [CrossRef]
19.
Li, X.; Gardiner, B.; Coleman, S.A. Square to Hexagonal lattice Conversion in the Frequency Domain.
In Proceedings of the 2017 IEEE International Conference on Image Processing, Beijing, China,
17–20 September 2017.
20.
Wen, W.; Khatibi, S. Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image. Sensors
2017
,
17, 620. [CrossRef] [PubMed]
21.
Wen, W.; Khatibi, S. A software method to extend tonal levels and widen tonal range of CCD sensor images.
In Proceedings of the 2015 9th International Conference on Signal Processing and Communication Systems
(ICSPCS), Cairns, Australia, 14–16 December 2015; pp. 1–6.
22.
Coleman, S.; Scotney, B.; Gardiner, B. Tri-directional gradient operators for hexagonal image processing.
J. Vis. Commun. Image Represent. 2016,38, 614–626. [CrossRef]
23.
Rubin, E. Visuell Wahrgenommene Figuren: Studien in Psychologischer Analyse; Gyldendalske Boghandel:
Copenhagen, Denmark, 1921; Volume 1.
24.
Pinna, B.; Deiana, K. Material properties from contours: New insights on object perception. Vis. Res.
2015
,
115, 280–301. [CrossRef] [PubMed]
25.
Tirunelveli, G.; Gordon, R.; Pistorius, S. Comparison of square-pixel and hexagonal-pixel resolution in
image processing. In Proceedings of the 2002 Canadian Conference on Electrical and Computer Engineering,
Winnipeg, MB, Canada, 12–15 May 2002; Volume 2, pp. 867–872.
26.
Frangi, A.F.; Niessen, W.J.; Vincken, K.L.; Viergever, M.A. Multiscale vessel enhancement filtering.
In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted
Intervention, Cambridge, MA, USA, 11–13 October 1998; Springer: Berlin, Germany, 1998; pp. 130–137.
27.
Vazquez, M.; Huyhn, N.; Chang, J.-M. Multi-Scale vessel Extraction Using Curvilinear Filter-Matching
Applied to Digital Photographs of Human Placentas. Ph.D. Thesis, California State University, Long Beach,
CA, USA, 2001.
28.
Kuijper, A. On detecting all saddle points in 2D images. Pattern Recognit. Lett.
2004
,25, 1665–1672. [CrossRef]
29.
Bar, M.; Neta, M. Humans Prefer Curved Visual Objects. Psychol. Sci.
2006
,17, 645–648. [CrossRef] [PubMed]
©
2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
... The virtual hexagonal enriched image has a hexagonal pixel form on a hexagonal arrangement. The generation process is similar to the resampling process in [17,18], which has three steps: projecting the original image pixel intensities onto a grid of sub-pixels; estimating the values of subpixels at the resampling positions; estimating each new hexagonal pixel intensity in a new hexagonal arrangement where the subpixels are projected back to a hexagonal grid, which are shown as red grids in Figure 2. In this arrangement the distance between each two hexagonal pixels is the same and the resolution of the generated Hex_E image is the same as the original image. ...
... Figure 14 shows the mean (a) and variance (b) of ratio values of ten corresponding pixel sets between each SQ and to Hex_E image. The mean (a) shows the nonlinear relation between SQ to Hex_E which was previously shown in [3,18]. The mean (a) also shows that the relation between to Hex_E is similar to the relation between SQ to Hex_E and behaves in a nonlinear manner. ...
... Figure 14 shows the mean (a) and variance (b) of ratio values of ten corresponding pixel sets between each SQ and SQ CEhex to Hex_E image. The mean (a) shows the nonlinear relation between SQ to Hex_E which was previously shown in [3,18]. The mean (a) also shows that the relation between SQ CEhex to Hex_E is similar to the relation between SQ to Hex_E and behaves in a nonlinear manner. ...
Article
Full-text available
The study of the evolution process of our visual system indicates the existence of variational spatial arrangement; from densely hexagonal in the fovea to a sparse circular structure in the peripheral retina. Today’s sensor spatial arrangement is inspired by our visual system. However, we have not come further than rigid rectangular and, on a minor scale, hexagonal sensor arrangements. Even in this situation, there is a need for directly assessing differences between the rectangular and hexagonal sensor arrangements, i.e., without the conversion of one arrangement to another. In this paper, we propose a method to create a common space for addressing any spatial arrangements and assessing the differences among them, e.g., between the rectangular and hexagonal. Such a space is created by implementing a continuous extension of discrete Weyl Group orbit function transform which extends a discrete arrangement to a continuous one. The implementation of the space is demonstrated by comparing two types of generated hexagonal images from each rectangular image with two different methods of the half-pixel shifting method and virtual hexagonal method. In the experiment, a group of ten texture images were generated with variational curviness content using ten different Perlin noise patterns, adding to an initial 2D Gaussian distribution pattern image. Then, the common space was obtained from each of the discrete images to assess the differences between the original rectangular image and its corresponding hexagonal image. The results show that the space facilitates a usage friendly tool to address an arrangement and assess the changes between different spatial arrangements by which, in the experiment, the hexagonal images show richer intensity variation, nonlinear behavior, and larger dynamic range in comparison to the rectangular images.
... (b) The grid and pixel are hexagonal and square respectively and there is no or fixed gap in [29], where the hexagonal grid is generated by a half-pixel shifting, its results show that the generated hexagonal images are superior in detection of curvature edges to the square images. (c) The grid and pixel are hexagonal and there is no gap [30]. In this work, the impact of the three sensor properties, the grid structure, pixel form and fill factor, is examined by curviness quantification using gradient computation. ...
... In [30], the gradient is proved to be an effective parameter for examining the impact of different sensor grids and pixel forms on curviness. In this paper, the histogram of gradient (HoG) is used for evaluating the characteristic of the sensors having different configurations. ...
Article
Full-text available
Our vision system has a combination of different sensor arrangements from hexagonal to elliptical ones. Inspired from this variation in type of arrangements we propose a general framework by which it becomes feasible to create virtual deformable sensor arrangements. In the framework for a certain sensor arrangement a configuration of three optional variables are used which includes the structure of arrangement, the pixel form and the gap factor. We show that the histogram of gradient orientations of a certain sensor arrangement has a specific distribution (called ANCHOR) which is obtained by using at least two generated images of the configuration. The results showed that ANCHORs change their patterns by the change of arrangement structure. In this relation pixel size changes have 10-fold more impact on ANCHORs than gap factor changes. A set of 23 images; randomly chosen from a database of 1805 images, are used in the evaluation where each image generates twenty-five different images based on the sensor configuration. The robustness of ANCHORs properties is verified by computing ANCHORs for totally 575 images with different sensor configurations. We believe by using the framework and ANCHOR it becomes feasible to plan a sensor arrangement in the relation to a specific application and its requirements where the sensor arrangement can be planed even as combination of different ANCHORs.
... The most popular geometric figures are: triangles, squares, and hexagons; these are the only regular shapes that tessellates the plane with no gaps [49]. In studies performed for the purposes of the article hexagonal grid was used-the higher usefulness of the hexagon over other figures has been repeatedly confirmed in the case of spatial analyses [50][51][52], as well as in other cases [53][54][55][56][57]. The aim of this research was to find the optimal size of the hexagonal grid, to the level of which the data can be generalized, without a significant decrease in the reliability of spatial modeling for various sustainable heritage management needs. ...
Article
Full-text available
Cultural heritage is a very important element affecting the sustainable development. To analyze the various forms of spatial management inscribed into sustainable development, information on the location of objects and their concentration at specific areas is necessary. The main goal of the article was to show the possibility of using various GIS tools in modeling the distribution of historical objects. For spatial analysis, it is optimal to use the point location of objects. Often, however, it is extremely difficult, laborious, expensive, and sometimes impossible to obtain. Thus, various map content generalizations were analyzed in the article; the main goal was to find the level for which the data with an acceptable loss of accuracy can be generalized. Such analyses can be extremely useful in sustainable heritage management. Article also shows how cultural heritage fits into the sustainable heritage management. The research included non-movable monuments in Poland. The obtained results showed the universality of this type of research both in the thematic sense (can be used for various types of objects) and spatial sense (can be performed locally, at the country level, or even at the continental level).
... These results further demonstrates the superiority of computational efficiency and performance. Since then, especially with the rise of biologically inspired image processing, hexagonal image processing has begun to attract researchers' attention, and in recent years the research has spread to broad applications like image restoration [6], image registration [7], [8], edge detection [9], [10], image labeling [11], morphological processing [12], [13], shape retrieval [14], hexagonal Gabor filtering [15], ultrasound image processing [16], computed tomography (CT) image reconstruction [17], [18], [19], [20], image sensor design [21], [22], [23], and convolutional neural network [24], [25], [26]. ...
Preprint
Full-text available
More efficient square to hexagonal lattice conversion
Article
Hexagonal image sampling and processing are theoretically superior to the most commonly used square lattice based sampling and processing, but due to the lack of commercial image sensors, current research mainly relies on virtually hexagonally sampled data through square to hexagonal lattice conversion, which is a typical 2-D interpolation problem. This paper presents a simplified and efficient square to hexagonal lattice conversion method. The method firstly utilizes the separable nature of the interpolation kernel to simplify the original 2-D interpolation into 1-D interpolation along the horizontal direction only, and then it applies the 1-D multirate technique to further simplify the shift-variant 1-D interpolation into shift-invariant 1-D convolutions. Compared with the original 2-D interpolation version, the proposed method becomes both simple and computationally efficient, and it is also suitable for implementation with parallel processing and hardware. Finally, experiments are performed and the results are consistent with the analysis.
Article
Full-text available
Achieving a high fill factor is a bottleneck problem for capturing high-quality images. There are hardware and software solutions to overcome this problem. In the solutions, the fill factor is known. However, this is an industrial secrecy by most image sensor manufacturers due to its direct effect on the assessment of the sensor quality. In this paper, we propose a method to estimate the fill factor of a camera sensor from an arbitrary single image. The virtual response function of the imaging process and sensor irradiance are estimated from the generation of virtual images. Then the global intensity values of the virtual images are obtained, which are the result of fusing the virtual images into a single, high dynamic range radiance map. A non-linear function is inferred from the original and global intensity values of the virtual images. The fill factor is estimated by the conditional minimum of the inferred function. The method is verified using images of two datasets. The results show that our method estimates the fill factor correctly with significant stability and accuracy from one single arbitrary image according to the low standard deviation of the estimated fill factors from each of images and for each camera.
Article
Full-text available
The current camera has made a huge progress in the sensor resolution and the lowluminance performance. However, we are still far from having an optimal camera as powerful as our eye is. The study of the evolution process of our visual system indicates attention to two major issues: the form and the density of the sensor. High contrast and optimal sampling properties of our visual spatial arrangement are related directly to the densely hexagonal form. In this paper, we propose a novel software-based method to create images on a compact dense hexagonal grid, derived from a simulated square sensor array by a virtual increase of the fill factor and a half a pixel shifting. After that, the orbit functions are proposed for a hexagonal image processing. The results show it is possible to achieve an image processing in the orbit domain and the generated hexagonal images are superior, in detection of curvature edges, to the square images. We believe that the orbit domain image processing has a great potential to be the standard processing for hexagonal images.
Conference Paper
Full-text available
As one of important outcomes of the past decades of researches on sensor arrays for digital cameras, the manufacturers of sensor array technology have responded to the necessity and importance of obtaining an optimal fill factor, which has great impact on collection of incident photons on the sensor, with hardware solution e.g. by introducing microlenses. However it is still impossible to make a fill factor of 100% due to the physical limitations in practical development and manufacturing of digital camera. This has been a bottle neck problem for improving dynamic range and tonal levels for digital cameras e.g. CCD cameras. In this paper we propose a software method to not only widen the recordable dynamic range of a captured image by a CCD camera but also extend its tonal levels. In the method we estimate the fill factor and by a resampling process a virtual fill factor of 100% is achieved where a CCD image is rearranged to a new grid of virtual subpixels. A statistical framework including local learning model and Bayesian inference is used for estimating new sub-pixel intensity values. The highest probability of sub-pixels intensity values in each resampled pixel area is used to estimate the pixel intensity values of the new image. The results show that in comparison to the methods of histogram equalization and image contrast enhancement, which are generally used for improving the displayable dynamic range on only one image, the tonal levels and dynamic range of the image is extended and widen significantly and respectively.
Conference Paper
Full-text available
In the past twenty years, CCD sensor has made huge progress in improving resolution and low-light performance by hardware. However due to physical limits of the sensor design and fabrication, fill factor has become the bottle neck for improving quantum efficiency of CCD sensor to widen dynamic range of images. In this paper we propose a novel software-based method to widen dynamic range, by virtual increase of fill factor achieved by a resampling process. The CCD images are rearranged to a new grid of virtual pixels com-posed by subpixels. A statistical framework consisting of local learning model and Bayesian inference is used to estimate new subpixel intensity. By knowing the different fill factors, CCD images were obtained. Then new resampled images were computed, and compared to the respective CCD and optical image. The results show that the proposed method is possible to widen significantly the recordable dynamic range of CCD images and increase fill factor to 100 % virtually.
Article
Image processing has traditionally involved the use of square operators on regular rectangular image lattices. For many years the concept of using hexagonal pixels for image capture has been investigated, and several advantages of such an approach have been highlighted. We present a design procedure for hexagonal gradient operators, developed within the finite element framework, for use on hexagonal pixel based images. In order to evaluate the approach, we generate pseudo hexagonal images via resizing and resampling of rectangular images. This approach also allows us to present results visually without the use of hexagonal lattice capture or display hardware. We provide comparative results with existing gradient operators, both rectangular and hexagonal.
Article
In recent years the processing of hexagonal pixelbased images has been investigated, and as a result, a number of edge detection algorithms for direct application to such image structures have been developed. We build on this research by presenting a novel and efficient approach to the design of hexagonal image processing operators using linear basis and test functions within the finite element framework. Development of these scalable first order and Laplacian operators using this approach presents a framework both for obtaining large-scale neighbourhood operators in an efficient manner and for obtaining edge maps at different scales by efficient reuse of the 7- point Linear operator. We evaluate the accuracy of these proposed operators and compare the algorithmic performance using the efficient linear approach with conventional operator convolution for generating edge maps at different scale levels.