Page 1

Abstract— the eleventh-century royal portrait miniature

painting of King Gagik-Abas of Kars, Queen Goranduxt, and

Princess Marem is an important image within the realm of

Armenian art history. However, conflicting statements about

art historical context have been drawn by human visual

analysis. In this paper, we investigate a pattern classification

algorithm to discover the historic context using the texture

information of the art image. Specifically, our goal is using

computer-aided techniques to provide the second opinion for

the determination of whether the object held by the queen in the

image is a silk cloth resembling the veil she wears. Experimental

results showed that image data mining techniques is a possible

solution to analyze the art image for interesting and useful

patterns.

I. INTRODUCTION

ITHIN the realm of Armenian art history, the

eleventh-century royal portrait miniature painting of

King Gagik-Abas of Kars, Queen Goranduxt, and Princess

w

Manuscript received October 15, 2007, revision accepted for publication

December 15, 2007

Steve Read is with the Department of Mathematic, University of

California at Santa Barbara,

sread@math.ucsb.edu).

Yu Cao is with the Department of Computer Science, California State

University at Fresno, Fresno, CA 93740 USA (phone: 559-278-4635; fax:

559-278-4197; e-mail: yucao@csufresno.edu).

Hazel Antaramian-Hofman is with California State University at Fresno,

Fresno, CA 93740 USA; email: adam13andre13@csufresno.edu.

CA 93106 USA. (e-mail:

Marem is an important image (Fig 1(a)). It is the only

surviving medieval royal portrait painting of the Bagratuni

dynastic family, and it provides valuable contextual

information regarding the family’s visual representation as

nobility in medieval Armenia. This particular family, along

with its branches, was under major political and religious

pressures and influences of its neighboring hegemonic

powers: the Byzantine Empire and the Islamic state of the

Sultanate and Muslim emirates, during the early part of the

eleventh-century.

There have been several scholarly papers written about the

image. While the image is in poor visual condition (as shown

in Fig 1(a)) and not readily accessible to the public, there

have been several studies that have identified, by human

visual analysis, the texture component of image, including

the costumes and interior setting. There was one object in

the painting about which conflicting information had been

stated in scholarly journals. While some scholars remark that

the object held by the queen, which is shown in Fig 1(b), is a

silk cloth resembling the veil she wears, others have been

either silent about it or have provided another interpretation.

Confirming the object to be cloth greatly advances other

contextual theories about the painting in general and the

history of the family.

Recent years have seen many applications of image

analysis techniques for art image. For example, in order to

show that some painters as early as 1420 used concave

mirrors (and, later, converging lenses) to project real

inverted images onto their supports which they then traced

and painted over, D G. Stork [1] perform analyses of the

reflections and shadows to infer the source(s) of

illumination. Compelling evidence was obtained to support

the conclusion that this source is the candle flame depicted

within the painting and held by Christ. In [2], S Lyu et al.

introduced an image processing technique for high-resolution

digital scans of the original works to authenticate works of

art, specifically paintings and drawings. In [3], the authors

presented a computer-aided image analysis technique using

multi-scale, multi-orientation decomposition analysis (e.g.,

wavelets) of high resolution digitized versions of drawings

and paintings for authentication.

Hinted by the increasing popularity of computer-aided

image analysis techniques in the study of art, we investigate

pattern classification methods to analysis the royal portrait

miniature for the art historical context. Our goal is to

determine whether the object held by the queen (as shown in

Fig 1.(b)) is a silk cloth resembling the veil she wears. We

use texture features as our feature sets and employ both

Mining the Royal Portrait Miniature for the Art Historical Context

Steve Read, Yu Cao, Member, IEEE, Hazel Antaramian-Hofman

(a) (b)

Fig 1. Image examples of the eleventh-century royal

portrait miniature painting of King Gagik-Abas of Kars,

Queen Goranduxt, and Princess Marem: (a) The image

shows the figure “Queen Goranduxt”; (b) A sub image

(cropped from (a)) shows the object held by the queen.

Page 2

Principal Component Analysis (PCA) [4] and Linear

Discrimnant Analysis (LDA) [5] for feature classification.

Our preliminary results have shown that image analysis and

computer vision techniques are promising methods for the

study of art historical context.

The rest of the paper is organized as follows. We briefly

introduce the source of the art image in Section II. Then we

present our proposed approach in Section III. We show the

detailed experimental results in Section IV and draw a

conclusion in Section V.

II. SOURCE OF THE ART IMAGE

The miniature painting shown in Fig 1(a) and Fig 1(b) are

from the early eleventh-century, suspected to have been

commissioned sometime between 1045 and 1055. It was not

until 1911 that the image resurfaced in an old book binding

shop in Jerusalem. The miniature was cut from its original

codex. It has endured much wear and mutilation from its

original cut. It is presently preserved within the Gospels of

King Gagik-Abas of Kars. A high resolution image is

available on the California State University at Fresno,

Armenian Studies Program’s Arts of Armenia website [6].

III. PROPOSED APPROACH

The overview of our method is shown in Fig 2. Given an

art image, we first perform preprocessing operation. The

preprocessing step includes selecting the appropriate image

fragmentations and generating the sub images for training

and testing. Texture feature extraction step extracts the

texture features from each training and testing sub images.

Features are sent to train and testing the two classifiers: PCA

based classifier and LDA classifier. We present each step as

below.

The primary basis for our method comes from the methods

used in [7]. Firstly, we take the image and convert it to

greyscale, and use the numerical intensity values of each

pixel in our calculations. Images are pre-selected to have

been (2n by 2n) pixels. We then create a pyramid from the

image which contains 3 levels, the first level is (2n by 2n), the

second level is [2n-1 by 2n-1], and the third level is [2n-2 by 2n-

2]. For each level, a one dimensional filter is applied in the

horizontal direction, and then vertical direction. The filter is

either a high pass filter, or a low pass filter, with the

combinations of filters giving us a horizontal wavelet

decomposition (high low), a vertical wavelet decomposition

(low high), and a diagonal wavelet decomposition (high

high). One of the benefits of wavelet decomposition is that it

is a multi-scale multi-orientation approach to image analysis,

which we believe will prove useful in our pattern recognition

application. With this pyramid we then compute the mean,

variance, skewness, and kurtosis of each decomposition at

each resolution level to create our feature vector. This gives

us a total of (3 resolutions) * (3 decompositions) * (4

computed values) = 36 elements of our feature vectors.

We then try to improve upon the methods used in [7],

which used a distance preserving projection, by using more

sophisticated and application specific techniques for

dimensionality reduction. Firstly, we used Principal

Component Analysis (PCA),

dimensionality reduction. PCA is method of dimensionality

reduction which tries to preserve the strongest correlation

between high dimensional data points. The first step of

principal component analysis is to normalize each the of the

feature vectors so that the mean of each one is zero. Then we

compute the covariance between each of the vectors being

considered. In this case the vectors we used were the 36-

dimensional feature vectors of each of the training images.

After the covariance matrix is computed, then it is

diagonalized to find the eigenvalues. We then reduce the

dimensionality by considering only the features with the

largest corresponding eigenvalues in our covariance matrix.

For example, we reduced our vectors to having only five

dimensions. So we took the eigenvectors corresponding to

the five largest eigenvalues, and projected each of the

training vectors into a new reduced dimensionality space.

Then, we projected each of our testing vectors, the feature

vectors we obtain from our testing images, into this new

reduced dimensionality space, and then found the minimum

and maximum distances between our test vector, and the

training vectors. We also applied PCA with no

dimensionality reduction, that is, we projected each of the

training vectors into a 36-dimensional space using the

covariance matrix, and repeated the procedure for the testing

vectors. PCA is very good at taking a large set of training

images from a positive class, for example, a large set of face

images, and then determining whether or not any new images

are in that class, if they are images of a face. In our case, the

main difference between our method and existing

implementation of PCA was that instead of just taking the

image itself to be the high dimensional vector, we used the

as our method of

Images

Preprocessing

Texture Feature Extraction

Train and Testing Classifier

Fig 2 Overview of the proposed approach

Page 3

feature vector from [2], and then used PCA on the feature

vectors.

Since the positive class we consider is only a small part of

the overall image, PCA was not as successful as we would

have liked. Thus, we also used a second method for

dimensionality reduction, Linear Discriminant Analysis

(LDA), which takes multiple classes of training images and

can then determine whether any new images are in one of the

classes. The first step in LDA is for each of the classes, we

compute the mean for each class, and the total mean of all of

the training images. The classes we used for our analysis

were two, the class of images belonging to the cloth pattern

we were trying to identify, and those which did not belong to

the cloth pattern. From this class information, we then

compute the within class scatter matrix, by taking the sum

over each of the j classes, of the probability for class j

multiplied by the covariance matrix of class j. The

covariance matrix is computed using vectors normalized with

the within class mean. We then compute the between class

scatter matrix, which is the sum of a new covariance matrix

for each of the j classes, finding the variance between each

of the mean vectors for the classes, normalized against the

total mean of all the training vectors. Then, we compute the

generalized eigenvectors for the between class, and within

class scatter matrices. The number of eigenvectors can be at

most the number of classes minus one, and since we only had

two classes, there was only one generalized eigenvector

found. We then projected all of our training vectors into a

new reduced dimensionality space, and much like with PCA,

any testing vectors are projected into the same space, and

their maximum and minimum distances from the training set

are computed. For the purpose of our project, we were most

concerned with determining whether the object held in the

queen's hand was of the same class as the cloth of her robe.

The difficulty we found using LDA, is that with only two

classes and thirty six features, the matrix from which we

compute the eigenvectors is singular.

IV. RESULTS

For the PCA algorithm we used two sizes of training image

sets, the first set of thirty six images was used so that we had

a square matrix for our covariance calculations. The second

set was just an extension of our original set to forty nine

training images. For each of these sets, we computed the

projected feature vectors, then calculated the smallest and

greatest distances from any new image's projected feature

vector to the training set. This was done for thirty images

within the class, and thirty images outside the class, and from

this information an ROC curve was computed. An ROC

curve is a graph of the sensitivity versus 1 minus the

specificity. The sensitivity and specificty are varied under

the condition of a specified parameter. For a good method,

we would like the sensitivity and specificity to be as close to

one as possible. Given that our parameter varies to identify

more images positively, thus increasing our sensitivity from

zero to one, we would like to have less truly positive images

identified as negative, which would mean having 1 minus

(a)

(b)

Fig 3. ROC curve of PCA analysis: (a) The ROC curve

of 36 training images; (b) The ROC curve of 49 training

images

(a)

(b)

Fig 4. ROC curve of LDA analysis using 2 classes: (a)

The ROC curve of 36 training images; (b) The ROC

curve of 49 training images

Page 4

specificty less than the sensitivity. Overall, we can then

measure the performance of our method by finding the area

under the ROC, and seeing if it is close to one. In our case

the parameter used for the ROC curve computation was a

threshold on the maximum distance from the projected

training data. In addition to using different sizes of training

data, we also computed ROC curves for the PCA algorithm

with thirty six principal components, and for five principal

components. For the LDA algorithm, we used the original

forty nine positive training images, and added to those forty

nine more negative training images. The same sets of thirty

positive and negative test images again had their maximum

and minimum distances computed, and the ROC curve was

computed using an upper threshold on the maximum

distance.

As indicated in the ROC curves (Fig 3, Fig 4, and Fig 5),

the basic application of PCA or LDA did not fare well with

our particular set of data. For a good ROC curve, we would

like the area underneath it to be less than one half. In our

curves, none of the areas are less than one half. Each method

on its own has some drawbacks with the data that we used.

For PCA, the fact that we had a small area from which to

choose positive class images limited the amount of training

data that we could use. For LDA, the small number of

classes available to us, coupled with the large number of

features, created a very singular matrix for our computations.

Thus a natural area to explore further would be the use of a

hybrid PCA / LDA algorithm for dimension reduction.

Hopefully the hybrid algorithm will be able to overcome the

difficulties we encountered with each method alone. Since

we were unsatisfied with the identification algorithm for the

first problem in the introduction, there was no exploration

into the use of our algorithm for the other two problems.

V. CONCLUSION

In this paper, we introduce a vision-based image analysis

technique to discover the historic context using the texture

information extracted from the royal portrait miniature. Two

classifiers are employed: one is based on Principal

Component Analysis and the other is based on Linear

Discriminant Analysis. Extensive experiments have been

performed and showed that the methods we proposed have

the potential to be used in mining art context. However, the

performance we have achieved is not as good as we expect.

In the future, we plan to explore other more sophisticated

classification methods, such as Supporting Vector Machine.

Using other features (such as shape descriptors) will also be

considered as one of the future directions.

REFERENCES

[1] D. G. STORK, "DID GEORGES DE LA TOUR USE OPTICAL

PROJECTIONS

CARPENTER'S STUDIO?," PRESENTED AT SPIE ELECTRONIC

IMAGING, SAN JOSE, CALIFORNIA USA, 2005.

WHILE PAINTING

CHRIST IN THE

[2] S. LYU, D. ROCKMORE, AND H. FARID, "A DIGITAL

TECHNIQUE FOR ART AUTHENTICATION," PROCEEDINGS OF

THE NATIONAL ACADEMY OF SCIENCES, VOL. 101, PP.

17006-17010, 2004.

[3] S. LYU, D. ROCKMORE, AND H. FARID, "WAVELET

ANALYSIS FOR AUTHENTICATION,"

ART+MATH=X, BOULDER, CO, U.S.A, 2005.

PRESENTED AT

[4] J. SHLENS, "TUTORIAL ON PRINCIPAL COMPONENT

ANALYSIS,"

HTTP://WWW.SNL.SALK.EDU/~SHLENS/PUB/NOTES/PCA.PDF

, 2005.

VOL.

2007:

[5] WIKIPEDIA, "LINEAR DISCRIMINANT ANALYSIS," VOL.

2007:

HTTP://EN.WIKIPEDIA.ORG/WIKI/FISHER'S_LINEAR_DISCRI

MINANT, 2007.

[6] "ELEVENTH-CENTURY

PAINTING OF KING GAGIK-ABAS OF KARS, QUEEN

GORANDUXT, AND PRINCESS MAREM," VOL. 2007:

HTTP://ARMENIANSTUDIES.CSUFRESNO.EDU/IAA_MINIATU

RES/IMAGE.ASPX?INDEX=0176, 2007.

ROYAL PORTRAIT MINIATURE

[7] J. PORTILLA AND E. P. SIMONCELLI, "A PARAMETRIC

TEXTURE MODEL BASED ON JOINT STATISTICS OF

COMPLEX WAVELET COEFFICIENTS," INTERNATIONAL

JOURNAL OF COMPUTER VISION, VOL. 40, PP. 49-71, 2000.

(a)

(b)

Fig 5. ROC curve of LDA analysis using 5 classes: (a)

The ROC curve of 36 training images; (b) The ROC

curve of 49 training images