Initial Evaluation of Human Supervised Automated Breast Cancer Screening Using
by K. Venkataramani*, H. Madhu
, S. Sharma
, H. V. Ramprakash**, A. Rajendra**,
, and G. Manjunath*
* Xerox Research Center India, Prestige Technology Park, Marathahalli, Bangalore, India,
Krithika.Venkataramani@xerox.com, Himanshu.Madhu2@xerox.com, Shubhi.Sharma@xerox.com,
**Central Diagnostic Research Foundation, Indiranagar, Bangalore, India, firstname.lastname@example.org
Breast cancer has the highest incidence among cancers in women, in India and world-wide. Screening and
early detection play a large role in reducing mortality as breast cancer can be cured if it is detected in the early stages.
Mammography is considered the gold standard in screening, but it is not useful for younger women due to low sensitivity
with denser breasts and its harmful X-rays can cause an increase in the risk of cancer if used frequently. Sono-
mammography is typically used in correlation. Incidence rates are rising in younger women as compared to previous
decades, due to changes from environmental pollutants and socio-economic reasons. This is causing a relook at
thermography for low cost and non-harmful screening. In this paper, an automated thermographic screening tool is used
to classify 108 subjects from the patient database of Central Diagnostic Research Foundation, a diagnostic clinic. In
addition to classification, the location of the suspected tumor is also highlighted on the thermography images. The results
are promising with 100% sensitivity and 73% specificity. The algorithm used is novel, which combines features obtained
from the temperature distribution of the subject, in a personalized manner, to classify as well as localize the tumor.
Breast cancer is the leading cause of cancer deaths for women worldwide as well as in India, with around
5,00,000 and 70,000 in the world and India, respectively, in 2012. It also has the highest incidence among cancers in
women, with 1.7 million and 145,000 diagnosed worldwide and in India, respectively, during 2012 . Breast cancer is
curable, with a high survival rate of 97%, if diagnosed in the early stage . This can be achieved by regular screening.
Mammography is the current gold standard for breast cancer screening. However, mammograms not only are less
sensitive at detecting tumors in young women due to denser breast tissue, but also are harmful enough to cause cancer
in young women due to the radiation exposure . The sensitivity of mammography falls from 83% in less dense tissue
to 55% in the highest density tissue . While no empirical studies have directly measured the risk of developing breast
cancer due to regular mammographic screening, many simulation models have been used to estimate risk depending on
the dosage of radiation, frequency of screening and age when screening started[3-5]. The most recent estimate was of
125 breast cancers and 16 deaths per 100,000 women in US screened annually from 40 years to 74 years . For
women with BRCA mutations who have a high risk of developing breast cancer at a young age (<40 years) and are
recommended to undergo screening from as early as 25 years, mammography’s risk of inducing cancer negates its
benefits of detecting cancer if used for annual screening below 35 years. Even for a single screening mammogram at
35 years, the lifetime risk of radiation-induced breast cancer is 11 per 100,000 women . Sono-mammography is
another commonly used modality, which is typically used in correlation with mammography as its standalone approach
has too many false positives and false negatives for screening . In general, there have been insufficient large scale
studies on other modalities of screening . Thermography is evaluated here as an alternative imaging modality for
breast cancer screening.
Breast cancer incidence is increasing in younger women presently as compared to previous decades. Due to
excessive use of chemicals in our modern society, that causes adverse effects on our bodies, we are seeing problems
that were not heard of a hundred years ago. One such risk factor is xeno-hormones, a group of man-made laboratory
synthesized chemicals that are hormonally active agents [9,10]. Many of these xeno-hormones are proven carcinogens
[9,10]. They are also well known for their ability to damage the immune system and interrupt hormonal balance. Our cells
can’t always distinguish fully between our own hormones and xeno- hormones. The xeno-hormones that mimic the
female hormones, estrogen and progesterone, increase the risk of breast cancer. Synthetic estrogens and progestins are
found in oral contraceptives and conventional synthetic hormone replacement therapies . Estrogen dominance is
probably the leading cause of breast cancer risk from hormones . All American-grown, non-organic livestock are fed
estrogenic drugs to fatten them. Also, the grains they are fed are laden with chemical sprays that accumulate in animal
tissue and promote hormone disruption in the person consuming them. Petro chemically-derived pesticides, herbicides
and fungicides are also sources of xeno-hormones . The chemical Bisphenol A (BPA) used in plastic bottles,
containers and almost all food-can liners is also a xeno-hormone . Exposure to Bisphenol A has shown an increased
incidence of breast and prostate cancers. Solvents found in fingernail polish and polish remover, glue, have been found
to have the same cell proliferative properties and endocrine disruption. Emulsifiers found in soaps and cosmetics of the
past and present are also risk factors. Skin being the largest organ is very capable of transferring chemical through it at a
highly efficient rate.
Thermography is useful in screening for breast cancers that are affected by hormonal activity. Thermography
measures the infra-red radiation emitted by the body . The increased metabolic rate of malignant cells and the neo-
vascularity and angio-genesis caused by cancer, increases the temperature compared to the surrounding tissue, which is
visible in thermographic images. There was research in the 1970’s and 1980’s on thermography for breast cancer
screening [13,14] and thermography got approval by FDA since 1982 as a risk predictor for cancer . However, the
lower sensitivity and specificity compared to mammography reported in a study in 1977  resulted in a decline in its
usage. W ith the advent of high resolution thermal cameras, there is a relook at thermography . In a study of 100
subjects with carcinoma , the sensitivity of thermography was 83%, and its value is in signaling abnormality in
younger subjects with carcinoma where mammograms or clinical examination did not detect malignancy. Thermography
may be able to detect malignant tumors 5 years before mammography . Breast cancers that grow due to hormones
have abnormal thermograms. Their progression is also faster. Thermography can help in detecting women at high
risk for cancer. The thermal cameras are also of lower cost, small and mobile, enabling non-contact and non-invasive
screening for large populations in non-hospital settings. These advantages could be useful especially in less developed
countries like India where cost and availability of hospitals play a vital role in screening.
Automatic screening algorithms can help in outreach to large populations, as doctors can focus on a fewer
number of suspicious cases for further analysis. There have been several semi-automated algorithms for breast cancer
screening with thermography [18,19] whose specificity and sensitivity is comparable with that of mammography, but have
been tested on only a small number of subjects. Ng et al  report a sensitivity of 86% and specificity of 91% on 25
normal subjects and 25 subjects with malignant tumors in stage 2 or stage 3 cancer. Most of these approaches use
textural features and temperature moments for classification with standard classifiers. In this paper, we use a feature
fusion based segmentation algorithm designed in our previous work  for high sensitivity to determine the specificity in
a dataset consisting mostly of normal subjects and benign tumor subjects.
2. Data description
Anonymized subject data has been obtained from the diagnostic clinic of Central Diagnostic Research
Foundation through a collaboration. This data has been obtained from a subset of the subjects coming to the clinic over
the past four years for breast examination. These subjects had come either for regular breast cancer screening or for
diagnosing a clinical condition of the breast. The data included thermal images, and the radiologist/thermographer’s
report having thermography and sono-mammography (ultrasound) findings and conclusions obtained from combining the
findings from both modalities. It also included subject demographic data, including age, gender, medical history of the
subject such as pregnant or lactating, clinical complaints, as well as personal history and family history of cancer.
The thermographic data has been obtained from the Meditherm camera, with a resolution of 690478 pixels.
There is a specific protocol for capturing the thermography images of the subject. The subject is asked to wear a loose
fitting gown and wait in an AC room for 15 minutes so that there is normalization of body temperature and external heat
conditions are minimized. The subject is then seated on a swivel chair at a fixed distance from the thermography camera.
The camera focus is zoomed in so that only the relevant region of the subject’s body is captured; from below the neck to
just below the infra-mammary fold. The subject’s chair is also swiveled so that the angle of capture is exactly frontal, at
45 oblique, i.e. right and left oblique, and right/left lateral. The thermography camera temperature range is also
calibrated within 8C range for each subject, with the maximum temperature of this range corresponding to the maximum
body temperature of the person. This would allow the maximum body temperature of the person to be observed at the
color corresponding to the image’s maximum temperature (in this case white). This is to assist in visual interpretation of
the image by the radiologist/ thermographer. Figure 1 shows sample images of a normal subject. In this paper, the
default view of the Meditherm camera is used, which shows the isotherm view, where pixels within every 0.5C range are
shown in a different color. The default views of the same subject are shown in figure 2. The thermographers/radiologists
find this isotherm view helpful and typically make most of their observations based on this view.
The data used in the paper consists of 108 subjects, with statistics as given in table 1. Mostly normal subjects
are presented in this data, with a significant percentage of subjects with benign tumors, such as simple cysts,
fibroadenoma, and fibrocystic disease. There are five cases of cancer. Among normal subjects, there are a few cases of
subjects who have a high hormonal response that places them at a higher risk of breast cancer, as well as faster growing
cancerous cells, if breast cancer develops. There is one case of infection in the ribs. In screening the general population
of women, it is expected that mostly normal subjects would be observed, with a significant percentage of women having
benign tumors and/or inflammatory/infectious conditions, and a few subjects with malignancy that depends on the breast
cancer incidence rates of that country/population.
3. Automated screening software features
The breast cancer screening tool (figure 3 to 7) allows the user to analyse thermal images for semi-automatic
detection and classification of breast tumours. The tool consists of three sections: 1) patient data acquisition, 2) thermal
data analysis, and 3) conclusion, data storage, and report generation.
3.1. Patient data acquisition
This section of the software tool captures relevant information needed as a pre-requisite for patient classification
and conclusion. The section allows the user to enter patient demographic information, patient cancer history, family
cancer history, patient medical history, patient complaints, and clinical examination. The information captured in this
section helps to evaluate the probability of a patient developing breast cancer. For example, history of cancer present in
patient and/or her family increases the chances of developing cancer in the future. Patient medical history plays an
important role in final diagnosis of the patient; for e.g. medical history like pregnant or lactating mothers suggests that it
might lead to temperature increase seen in the thermal images. Moreover, medical history of patient gives insight into
any benign conditions that may lead to cancer. Patient complaints and clinical/physical examination give relevant
information on the present condition of the patient, which include important data on hormonal levels, breast nodules, and
other factors. The entire acquisition section captures data to help evaluate the patient condition in a holistic manner.
3.2. Thermal Analysis
This section of the tool focuses on performing thermal analysis on the captured thermal images of the patient.
The following operations summarizes the thermal analysis section:
Load: The thermal images of the patient are captured in different views like frontal view, lateral view, oblique
view etc. The tool allows the user to load all those thermal images, browse through them, view them in infrared view or
2D contour view (contour view is created by temperature separation of 0.5 degree Celsius).
Crop: Once all the thermal images are loaded, the tool allows the user to crop the regions of interest from all the
images. These cropped regions will be input for the auto-detect tumor algorithm as well as the manual-select process.
Auto-detect: After the cropping operation is over, the tool enables the auto-detect button which when clicked by
the user will initiate the tumor auto-detection algorithm. If any suspected tumor detected, the algorithm highlights the
tumor region, displays it in the best possible view and displays a message saying “suspected malignancy detected”. If no
suspected tumor is detected, then it just displays the frontal view thermal image with a message “no tumor present”. The
tool provides the user the facility to confirm whether the output of auto-detect algorithm is correct or not. This is important
to improve the accuracy of the auto-detect tumor algorithm.
Manual-select: After the auto-detect tumor process is done, the tool enables the manual-select process. The
user chooses the desired cropped region and clicks on the manual-select button. The cropped region is displayed in 2D
contour view and allows the user to mark the suspected tumor regions. The manual-select algorithm is then processed
and the best possible thermal image view along with the highlighted suspected tumor region is displayed. The user
perform the manual-select process only when the user suspects any tumor is present.
Comments: The tool provides the facility for the user to enter the thermobiological score of the patient and
quadrant-wise comments for right breast and left breast with respect to the captured thermal images and above
3.3. Patient evaluation, data storage and report generation
This section of the tool concludes the patient evaluation, persists all data collected from other sections of the
tool, and generates report for documentation purposes. It evaluates the acquired patient and thermal information to
classify the patient as normal, benign, malignant, or bilateral. All this data is then stored in a database that contains
records of all breast cancer screening patients; the information can be captured from the database and edited and
analysed further using the tool. Also, report containing all the relevant data is generated for doctor’s reference.
4. Automated screening algorithm
In this paper, we are using human supervision through the screening software for locating the breast regions,
i.e. our Regions of Interest (ROI). Fully automated ROI location is difficult, as the region is amorphous and needs to be
located at different angles in the different images. Current approaches to automate ROI segmentation have been
generally done on just frontal images  through heuristic approaches and hence are very noisy, as the body shape of
subjects vary. Our approaches to ROI selection through human supervision and automated algorithms for classification &
localizing the tumor are described in the following sub-sections.
4.1. Regions of Interest selection through human supervision
The right and left breast regions from the thermograms are the Regions of Interest (ROI) to us here to determine
the presence/absence of cancer or any abnormality. The infra-mammary folds are hot normally due to friction, and hence
were not considered in the ROI. The lymph nodes in the axilla regions are also possible regions where there may be
metastasis of breast cancer and are typically examined in sono-mammograms. In thermograms, the axilla regions are
generally hot due to friction and the presence of lymph nodes. Detection of abnormalities in the axilla regions from
thermography are out of scope in this study, although it is an interesting topic for future research.
Six thermal images are captured for each cancer subject. The right/left breast region is then manually
segmented using a free-form selection in the software described in Section 3. Figure 8 shows the segmented ROIs from
a sample subject.
4.2. Automated localization and classification algorithm on ROIs
The temperature map of the ROIs is obtained from the camera’s temperature colorbar. From the ROIs, the
tumor detection and location is done automatically using a multi-feature fusion algorithm . The split-and-merge
segmentation approach is used to detect/locate the tumors, although other segmentation approaches could be used. The
novelty lies in the usage of multiple features, their decision-based fusion and in using subject specific thresholds based
on their temperature distribution.
Cancerous tumors are typically “hot”, and are significantly hotter than the surrounding tissue. Small tumors in
the earlier stages of cancer growth are “warm.” As the surface body temperature is observed to vary by a few degree
centigrade across subjects, the temperature thresholds to determine “hot” areas need to be subject-dependent, and
based on the temperature distribution over the ROIs. Weak edges in the temperature map can also be obtained around
the tumors, but may not have closed boundaries. Temperature thresholds based on the temperature distribution and
edges around the tumor may be used to detect the tumor region. The following set of features are used to decide if a
particular block in the ROI belongs to a tumor or not.
1) The temperature threshold
based on the temperature distribution in the ROIs: If k
% of the block has
temperature above threshold
, decide as tumor.
2) Another temperature threshold
based on the maximum temperature in the ROIs: If k
% of the block has
temperature above threshold
, decide as tumor.
3) A temperature threshold
based on normal body surface temperature: If k
% of block has temperature
decide as normal.
4) Edges around the tumor: If the edge length is k
% of the block perimeter, decide as tumor.
These block decisions are combined using a decision fusion rule to maximize cancerous tumor detection and
minimize false positives. The optimal decision fusion is based on the inter-dependence of these features in tumor
detection. More details on selecting the optimal fusion rule is found in . Figure 3 and 4 shows the tumor location on
sample subjects with cancer.
5. Results of automated thermal screening with human supervised ROI selection
The algorithm described in Section 4.2 is tuned to maximize tumor detection in cancerous subjects. In this
=max-2C, with max denoting the maximum temperature, k
=100%, and with
the global decision as tumor block if decisions 1) and 2) and 3) decide as tumor block. The results of automatic tumor
localization and classification is shown in table 2 and figure 9. There is 100% sensitivity as all cancerous cases and the
suspicious case were declared as suspicious by the automated screening, and the tumor regions were correctly
localized. There is 73% specificity. Among the errors, i.e. normal/benign tumor cases declared suspicious by the
automated algorithm are in a)12 of the 14 cases where the radiologist required a repeat study after 3 months, 6 months,
or over a year, as he felt they needed regular monitoring to rule out malignancy, b) 9 cases where external heat
conditions caused a rise in temperature in parts of the body, c) 3 high risk subjects who had a high hormonal response. It
is better to have additional stages of automated classification after this automated algorithm to re-classify the suspected
subjects into those that are at high risk for cancer and/or require regular monitoring. The imaging protocols can be
modified or additional questions to the subject can be asked to rule out external heat conditions. The imaging protocols
can include an additional cooling step, where cool air is blown on the subjects to normalize the external heat conditions.
This can be done even after an initial evaluation. If the heat persists after this additional cooling protocol, then it would be
due to malignancy, inflammation or infection. Questions could include whether the subject was exposed to some external
conditions causing heat generation.
5. Conclusions and Future Work
A subset of 108 subjects from the patient data of Central Diagnostic Research Foundation were categorized by
the radiologist using age, gender, medical history, history of cancer in the patient and her family, thermography and
sono-mammography data findings. We have demonstrated an automatic breast cancer screening tool on thermography
data from this clinic. This tool also has visualization features to assist the doctors or thermographers in their diagnosis.
Using automated thermographic breast cancer screening on 108 subjects consisting of normal subjects, benign tumor
and malignant tumor subjects, promising results of 100% sensitivity and 73% specificity was obtained. The specificity of
automated screening could be improved in future with better imaging protocols, such as cooling the subject to remove
heat generated by external causes. The specificity can also be improved with additional algorithms that determine high
risk subjects with excessive hormonal responses and that separate borderline suspicious cases into different categories.
The screening tool will also be modified in future to take into account the age and medical history along with
thermography data to improve sensitivity and specificity. Further tests on larger number of subjects needs to be done to
determine the possibility of using thermography as a first line breast cancer screening modality, followed by additional
tests to diagnose breast cancer.
 International agency for research on cancer, WHO, 2012 cancer statistics, http://globocan.iarc.fr
 Ng E. Y. K., and Sudharsan N. M., "Numerical computation as a tool to aid thermographic interpretation". Journal
of medical engineering and technology, vol. 25(2), p.p. 53-60, 2001.
 Miglioretti D. L., Lange J., van Ravesteyn N., van den Broek J. J., Lee C. I., Melnikow J., et al. “Radiation-
Induced Breast Cancer and Breast Cancer Death From Mammography Screening [Abstract]”. Rockville, MD:
Agency for Healthcare Research and Quality, 2015.
 de Gonzalez A. B., Berg C. D., Visvanathan K., Robson M., “Estimated Risk of Radiation-Induced Breast Cancer
From Mammographic Screening for Young BRCA Mutation Carriers”. Journal of the National Cancer Institute.
Vol. 101(3), pp. 205-209, Feb 2009.
 Yaffe M. J., Mainprize J. G., “Risk of radiation-induced breast cancer from Mammographic screening”. Radiology,
vol. 258(1), Jan 2011.
 Kennedy D., et al, “A comparative review of thermography as a breast screening technique”. Integrative Cancer
Therapies, vol. 8(1), 2009.
 Sree S.V., Ng E. Y., Acharya R.U., Faust O., “Breast imaging: a survey”. World Journal of Clinical Oncology, vol.
2(4), April 10, 2011: 171-178.
 U.S. Preventive Services Task Force, Draft Recommendation Statement, Breast Cancer: Screening, 2015.
 Vinas R., et al, “Non-Genomic Effects of Xenoestrogen Mixtures”. International Journal of Environment Research
and Public Health. Vol. 9(8), 2012.
 Bidgoli S. A., et al, “Role of Xenoestrogens and endogenous sources of estrogens on the occurrence of
premenopausal breast cancer in Iran”. Asia Pacific Journal of Cancer Prevention, vol. 12, 2011.
 Pike M. C., et al, “Breast cancer in young women and the use of oral contraceptives: possible modifying effect of
formulation and age at use”. The Lancet, vol. 322(8356), Oct 1983.
 Wagner M., et al, “Endocrine disrupters in bottled mineral water: total estrogenic burden and migration from
plastic bottles”. Environmental Science and Pollution Research, vol. 16, 2009.
 Lapayowker MS, Barash I, Byrne R, et al., “Criteria for obtaining and interpreting breast thermograms”. Cancer.
Vol. 38, pp. 1931-1935, 1976.
 Gautherie M., Gros C.M.. “Breast thermography and cancer risk prediction”. Cancer. 1980; 45:51-56.
 Feig SA, Shaber GS, Schwartz GF, et al. “Thermography, mammography, and clinical examination in breast
cancer screening”. Review of 16,000 studies. Radiology. 1977;122:123-127.
 Keyserlingk JR, Ahlgren PD, Yu E, Belliveau N, Yassa M. “Functional infrared imaging of the breast”. IEEE Eng
Med Biol Mag, 19. 2000:30-41.
 Head et al., “Thermography as a predictor of prognosis of cancer in the breast”. Cancer, vol. 67, March 1991.
 Nair et al, “A Prospective Study of Computerized Digital Infrared Image Analysis (No Touch BreastScanTM) in
Biopsy Proven Breast Cancers”. Cancer Research, 69 (24), Supl. 3, Dec 15, 2009.
 Borchartt et al, “Breast thermography from an image processing viewpoint: A survey”. Signal Processing, 93,
 U.R. Acharya, E.Y.K. Ng, J.H. Tan, S.V. Sree, “Thermography based breast cancer detection using texture
features and support vector machine”, Journal of Medical Systems, 36, 2012: 1503–1510.
 Venkataramani K. Mestha L. K., et al, “Semi-automated breast cancer tumor detection with thermographic video
imaging”, IEEE Int. Conf. of the Engineering, Medicine and Biology Society, 2015.
 K. Venkataramani and B. V. K. Vijaya Kumar, “Designing classifiers for fusion-based biometric verification,
chapter in Biometrics: theory, methods and applications”, Eds. Boulgouris, Plataniotis, and Micheli-Tzankou,
Table 1. Statistics of the subject data used in this paper
Subject category Number of subjects
Cancerous tumor 5
Benign tumor 38
Normal subjects 60
Repeat study requested among
normal/benign tumor cases
14 out of 98
High risk due to hormonal responses among
normal/benign tumor subjects
3 out of 98
Table 2. Results of automated screening on human supervised ROI selection.
Subject category Number of subjects Classified as suspicious through
Cancerous tumor 5 5 out of 5
Benign tumor 38 13 out of 38
Normal subjects 60 12 out of 38
Suspicious 1 1 out of 1
Infection 1 0 out of 1
Lactating 2 1 out of 2
Repeat study requested among
normal/benign tumor cases
14 out of 98 12 out of 14
High risk due to hormonal responses among
normal/benign tumor subjects
3 out of 98 3 out of 3
Fig. 1. Images of a normal subject from the Meditherm camera at a resolution of 690
478 pixels. There is high
temperature in the root of the neck and upper part of the chest due to external heat exposure.
Fig. 2. Images of a normal subject in figure 1 in the isotherm view, with each color representing a 0.5
Fig. 3: Suspected Tumor Detected for the subject with breast cancer using auto-detect algorithm
Fig. 4. No Tumor Detected for the normal subject using auto-detect algorithm
Fig. 5. Manual-select process to detect the suspected tumor
Fig. 6. Suspected tumor for the subject with cancer displayed using Manual-select process
Fig. 7. Patient data acquisition
Fig. 8. Segmented ROI outlined in purple for a subject with cancer. The automated screening located the tumor outlined
Fig. 9. Sensitivity and Specificity percentage of the automated screening results.