No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Classification of tongue color based on CNN
... With the assistance of artificial intelligence (AI), tongue diagnosis will be objective and people without medical knowledge can give themselves a preliminary diagnosis of a health condition. In recent years, much effort has been spent on AI-based tongue diagnosis, especially in the field of tongue color recognition [5,6], tongue shape analysis [7], cracks segmentation [8], thickness, and moisture of tongue coating classification [9,10]. ...
... Swin-Transformer Encoder Z 10,11 Swin-Transformer Encoder Z [4][5][6][7][8][9] Swin-Transformer Encoder Z 2,3 Pre-trained Position Embedding Figure 4: Architecture of tongue segmentor. e tongue image is divided into several patches and then added with position embeddings to retain spatial information. ...
... Tongue partition. e tongue is divided into four areas automatically based on the midline and the detection parameters vary with areas adaptively.6 ...
Tongue diagnosis is a convenient and noninvasive clinical practice of traditional Chinese medicine (TCM), having existed for thousands of years. Prickle, as an essential indicator in TCM, appears as a large number of red thorns protruding from the tongue. The term “prickly tongue” has been used to describe the flow of qi and blood in TCM and assess the conditions of disease as well as the health status of subhealthy people. Different location and density of prickles indicate different symptoms. As proved by modern medical research, the prickles originate in the fungiform papillae, which are enlarged and protrude to form spikes like awn. Prickle recognition, however, is subjective, burdensome, and susceptible to external factors. To solve this issue, an end-to-end prickle detection workflow based on deep learning is proposed. First, raw tongue images are fed into the Swin Transformer to remove interference information. Then, segmented tongues are partitioned into four areas: root, center, tip, and margin. We manually labeled the prickles on 224 tongue images with the assistance of an OpenCV spot detector. After training on the labeled dataset, the super-resolutionfaster-RCNN extracts advanced tongue features and predicts the bounding box of each single prickle. We show the synergy of deep learning and TCM by achieving a 92.42% recall, which is 2.52% higher than the previous work. This work provides a quantitative perspective for symptoms and disease diagnosis according to tongue characteristics. Furthermore, it is convenient to transfer this portable model to detect petechiae or tooth-marks on tongue images.
... However, the diagnosis of human diseases based on tongue condition is a standard technology of TCM, but its practicability is limited by the following factors: (1) Tongue diagnosis means that its approach is subjective, qualitative, and difficult to diagnose automatically [19]; (2) Tongue diagnosis is usually based on the ability of eyes to make detailed discrimination, and requires many years of experience and training of practitioners to obtain competence; (3) The correctness of tongue diagnosis relies on the doctor's experience [20]; (4) traditional tongue diagnosis is always dedicated to identify the syndromes other than diseases; (5) TCM tongue diagnosis was affected by the external environment (such as light and temperature); (6) The results of traditional tongue examinations cannot be described scientifically and quantitatively. Therefore, there is an urgent need to establish a modern medical system for TCM tongue diagnosis , which ought to be in the direction of the leading modern standardization of science and technology, objectivity, quantification, automation and exhibition [21]. ...
... Many studies have been conducted on tongue image using CNN. Hou et al. used CNN to study the details and characteristics of tongues [21]. In the twoclass experiment, the CNN model has achieved better results in the classification of tongue image syndrome compared with traditional machine learning methods such as SVM, multilayer perceptron network (MLP) and random forest (RF) [15]. ...
... Tongue color could also provide beneficial information on blood congestion, water imbalance, and psychological problems. The tongue color recognition with high accuracy will contribute to the efficiency of TCM diagnosis [21]. ...
Tongue diagnosis is an important process to non-invasively assess the condition of a patient’s internal organs
in traditional Chinese medicine (TCM) and each part of the tongue is related to corresponding internal organs.
Due to continuing computer technological advances, especially the artificial intelligence (AI) methods have
achieved significant success in tackling tongue image acquisition, processing, and classification, novel AI
methods are being introduced in traditional Chinese medicine tongue diagnosis medical practices. Traditional
tongue diagnose depends on observations of tongue characteristics, such as color, shape, texture, moisture, etc.
by traditional Chinese medicine physicians. The appearance of the tongue color, texture and coating reflects
the improvement or deterioration of patient’s conditions. Moreover, AI can now distinguish patient’s condition
through tongue images, texture or coating, which is all possible increasingly with help from traditional Chinese
medicine physicians under the traditional Chinese medicine tongue theory. AI has enabled humans to do what
was previously unimagined: traditional Chinese medicine tongue diagnosis with feeding a large amount of
tongue image and tongue texture/coating data to train the AI modes. This review focuses on the research
advances of AI in TCM tongue diagnosis thus far to identify the major scientific methods and prospects. In this
article, we tried to review the AI application in resolving the tongue diagnosis of traditional Chinese medicine
on color correction, tongue image extraction, tongue texture/coating segmentation.
Keywords: Artificial intelligence, Traditional Chinese medicine, Tongue diagnosis, Machine learning, Deep
learning, Color model, Tongue segmentation, Tongue image extraction
... AlexNet [7], VGG [8] and ResNet [9], have been successfully applied for tongue image analysis. However, these previous tongue classificationrelated researches mainly focus on tongue color [10,11] and teeth masks [12,13]. To the best of our knowledge, there have been a little research about tongue size and shape classification, which is actually as significant as other tongue features in TCM. ...
... Tongue body classification: Inspection of the tongue body mainly includes the observation of the spirit, color, feature (shape, texture, sports, fissures and teeth masks), the movement of the tongue and its sublingual collateral vessels [1]. Hou et al. [10] modified the network structure of CaffeNet to classify tongue images according to tongue color. Shang et al. [11] designed a simple model with two convolutional layers and two fully connected layers for tongue color classification. ...
The size and shape of the tongue can reflect different pathological changes of the human body in Traditional Chinese Medicine (TCM). Recently, convolutional neural networks (CNNs) have been widely used for the classification of the color, thickness and teeth marks of the tongue. However, only a few works have been devoted to tongue size and shape classification, which is also key evidence for tongue diagnosis. In this work, we proposed an efficient deep network, TSC-WNet, for tongue size and shape classification. The proposed TSC-WNet consists of two subnetworks, i.e. TSC-Net and TSC-UNet. While TSC-Net is a straightforward and effective classification backbone, TSC-UNet is built for tongue segmentation and offers complementary beneficial features to enhance the classification performance of the networks. Our classification backbone requires fewer parameters than classic CNNs like AlexNet, VGG16 and ResNet18, and achieves better classification performance. Employing TSC-Net as the encoder, the TSC-UNet was used to provide the segmentation information for helping better tongue size and shape classification. Two different datasets, i.e. FJTCM/SZU and BioHit, were employed for performance evaluation. The experimental results show that TSC-Net achieves at least 2% higher accuracy and F1 score than the baseline networks. Ablation studies show that the fusion of TSC-Net and TSC-UNet at both input and feature levels can further improve the accuracy and F1 score by about 2%. The code is available at: https://github.com/Yating-Huang/TSC-WNet.
... In addition, some tongue processing algorithm studies have only focused on detection [12], segmentation [13,14], or classification [15,16]. The authors of [12] used a one-stage detector SSD with MobileNetV2 to detect tongue regions. ...
... The authors of [13,14] proposed a new end-to-end tongue localization and segmentation method and a fast tongue segmentation system based on U-Net. The authors of [15] explored the convolutional neural network method in order to classify tongue color from tongue images, and in [16], a multiple-instance method was presented for the recognition of tooth-marked tongues. Though these studies have made some progress, they are all independent, and there is no tongue system on the Android platform that integrates all three modules simultaneously. ...
To quickly and accurately identify the pathological features of the tongue, we developed an intelligent tongue diagnosis system that uses deep learning on a mobile terminal. We also propose an efficient and accurate tongue image processing algorithm framework to infer the category of the tongue. First, a software system integrating registration, login, account management, tongue image recognition, and doctor–patient dialogue was developed based on the Android platform. Then, the deep learning models, based on the official benchmark models, were trained by using the tongue image datasets. The tongue diagnosis algorithm framework includes the YOLOv5s6, U-Net, and MobileNetV3 networks, which are employed for tongue recognition, tongue region segmentation, and tongue feature classification (tooth marks, spots, and fissures), respectively. The experimental results demonstrate that the performance of the tongue diagnosis model was satisfying, and the accuracy of the final classification of tooth marks, spots, and fissures was 93.33%, 89.60%, and 97.67%, respectively. The construction of this system has a certain reference value for the objectification and intelligence of tongue diagnosis.
... Both feature extraction and classification can be done by using CNN. While processing time is not taken accounted on in many works [7,3,8], it is found that CNN method takes longer computation time compared with other methods. This technique had been applied in other image processing applications such as facial expression recognition system [9][10] and face recognition [11][12] as well. ...
... The work shows that increasing complexity of CNN model not necessary increases the accuracy. Hou et al. proposed a CNN based method for TDS [8]. CNN is used for both feature extraction and classification with one single step or model. ...
Tongue diagnosis is known as one of the effective and yet noninvasive techniques to evaluate patient’s health condition in traditional oriental medicine such as traditional Chinese medicine and traditional Korean medicine. However, due to ambiguity, practitioners may have different interpretation on the tongue colour, body shape and texture. Thus, research of automatic tongue diagnosis system is needed for aiding practitioners in recognizing the features for tongue diagnosis. In this paper, a tongue diagnosis system based on Convolution Neural Network (CNN) for classifying tongue colours is proposed. The system extracts all relevant information (i.e., features) from three-dimensional digital tongue image and classifies the image into one of the colours (i.e. red or pink). Several pre-processing and data augmentation methods have been carried out and evaluated, which include salt-and-pepper noises, rotations and flips. The proposed system achieves accuracy of up to 88.98% from 5-fold cross validation. Compared to the reported baseline Support Vector Machine (SVM) method, the proposed method using CNN results in roughly 30% improvement in recognition accuracy.
... Convolutional neural networks (CNN) have been used to classify tongue colors [3 identify cracked tongues, and classify pulse signals [32] in TCM. We use the CNN mo to classify texts of medical records in TCM, and thus call it TextCNN. ...
... Convolutional neural networks (CNN) have been used to classify tongue colors [31], identify cracked tongues, and classify pulse signals [32] in TCM. We use the CNN model to classify texts of medical records in TCM, and thus call it TextCNN. ...
Evidence-based treatment is the basis of traditional Chinese medicine (TCM), and the accurate differentiation of syndromes is important for treatment in this context. The automatic differentiation of syndromes of unstructured medical records requires two important steps: Chinese word segmentation and text classification. Due to the ambiguity of the Chinese language and the peculiarities of syndrome differentiation, these tasks pose a daunting challenge. We use text classification to model syndrome differentiation for TCM, and use multi-task learning (MTL) and deep learning to accomplish the two challenging tasks of Chinese word segmentation and syndrome differentiation. Two classic deep neural networks—bidirectional long short-term memory (Bi-LSTM) and text-based convolutional neural networks (TextCNN)—are fused into MTL to simultaneously carry out these two tasks. We used our proposed method to conduct a large number of comparative experiments. The experimental comparisons showed that it was superior to other methods on both tasks. Our model yielded values of accuracy, specificity, and sensitivity of 0.93, 0.94, and 0.90, and 0.80, 0.82, and 0.78 on the Chinese word segmentation task and the syndrome differentiation task, respectively. Moreover, statistical analyses showed that the accuracies of the non-joint and joint models were both within the 95% confidence interval, with pvalue < 0.05. The experimental comparison showed that our method is superior to prevalent methods on both tasks. The work here can help modernize TCM through intelligent differentiation.
... According to Table 1, we can find that the illnesses can change the size and 10 shape of the tongue by affecting these tongue tissues. For example, muscle, mucus, blood and gland of the tongue may become weak due to the blood deficiency and lead to thin tongue. ...
... However, these previous tongue classification related researches mainly focus on tongue color [10] [11] and teeth mask [12] [13]. To the best of our knowledge, 20 there has been a little research about tongue size and shape classification, which is actually as significant as other tongue features in TCM. ...
Tongue diagnosis is one of the most important diagnostic methods in Traditional
Chinese Medicine (TCM). The size and shape of the tongue can reflect different
pathological changes of the human body. With the advance in deep learning (DL),
Convolutional Neural Networks (CNNs) have been widely used for the classification of
the color, thickness and teeth marks of the tongue. However, only few works have
been devoted to tongue size and shape classification, which is also a key evidence for
tongue diagnosis. In this work, we proposed an efficient framework, TSC-WNet, for
tongue size and shape classification. The proposed TSC-WNet consists of two
subnetworks, i.e. TSC-Net and TSC-UNet. While TSC-Net is a simple and efficient
classification backbone, TSC-UNet is designed for tongue segmentation and provides
complimentary useful features to improve the classification performance of whole
network. Our classification backbone requires fewer parameters than classic CNNs like
AlexNet, VGG16 and ResNet18, and achieves better classification performance.
Employing TSC-Net as the encoder, the TSC-UNet was used to provide the
segmentation information for helping better tongue size and shape classification. Two
different datasets, i.e. FJTCM/SZ and BioHit, were employed for performance
evaluation. The experimental results show that, TSC-Net achieves at least 2% higher
accuracy and F 1 score than the baseline networks. Ablation studies show that the
fusion of TSC-Net and TSC-UNet at both input and feature level can further improve
the accuracy and F 1 score by about 2%.
... The dataset in our study was highly specific, consisting solely of images captured from uniform circular CPS with varying colors. Given that color recognition is a well-studied area, especially in CNN models designed for three-dimensional color images [27][28][29], we utilized pre-trained deep learning models specialized in image processing. By leveraging transfer learning, we were able to achieve high performance even with the limited training data. ...
A deep learning algorithm is introduced to accurately predict glucose concentrations using colorimetric paper sensor (CPS) images. We used an image dataset from CPS treated with five different glucose concentrations as input for deep learning models. Transfer learning was performed by modifying four established deep learning models—ResNet50, ResNet101, GoogLeNet, and VGG-19—to predict glucose concentrations. Additionally, we attempted to alleviate the challenge of requiring the large amount of training data by applying data augmentation techniques. Prediction performance was evaluated using coefficients of determination (R²), root mean squared error (RMSE), and relative-RMSE (rRMSE). GoogLeNet showed the highest coefficient of determination (R² = 0.994) and significantly lower prediction errors across all concentration levels compared with a traditional machine learning approach (two-sample t-test, p < 0.001). When data augmentation was performed using 20% of the entire training dataset, the mean prediction error was comparable to that of the original entire training dataset. We presented a novel approach for glucose concentration prediction using deep learning techniques based on transfer learning and data augmentation with image data. Our method uses images from CPS as input and eliminates the need for separate feature extraction, simplifying the prediction process.
Graphical Abstract
... Tongue feature classification via AI can assist doctors in objectively recognizing complex tongue features. For tongue color classification, Hou et al. 16 used a convolutional neural network to classify tongue color, and Ni et al. 17 proposed TongueCaps by combining CapsNet and a residual block structure to achieve end-to-end tongue color classification. Tooth marks have received increasing attention as more common and relatively obvious tongue features. ...
The classification of tongue shapes is essential for objective tongue diagnoses. However, the accuracy of classification is influenced by numerous factors. First, considerable differences exist between individuals with the same tongue shape. Second, the lips interfere with tongue shape classification. Additionally, small datasets make it difficult to conduct network training. To address these issues, this study builds a two-level nested tongue segmentation and tongue image classification network named IF-RCNet based on feature fusion and mixed input methods. In IF-RCNet, RCA-UNet is used to segment the tongue body, and RCA-Net is used to classify the tongue shape. The feature fusion strategy can enhance the network’s ability to extract tongue features, and the mixed input can expand the data input of RCA-Net. The experimental results show that tongue shape classification based on IF-RCNet outperforms many other classification networks (VGG 16, ResNet 18, AlexNet, ViT and MobileNetv4). The method can accurately classify tongues despite the negative effects of differences between homogeneous tongue shapes and the misclassification of normal versus bulgy tongues due to lip interference. The method exhibited better performance on a small dataset of tongues, thereby enhancing the accuracy of tongue shape classification and providing a new approach for tongue shape classification.
... Given the scarcity of existing literature on the application of machine learning techniques to forecast analogous datasets, initial investigative endeavors necessitate a foundation in model selection and validation. In the quest to construct a robust predictive model, the study has harnessed an array of machine learning methodologies, encompassing the Backpropagation Neural Network [48], Support Vector Machine (SVM) [49], Random Forest (RF) [50], and Convolutional Neural Network (CNN) [51]. To rigorously assess the impact of varying foundational models on predictive accuracy, a uniform dataset has been employed, with an 80% allocation for training and a 20% reservation for testing. ...
The aging population in China is increasing the prevalence of degenerative diseases such as knee osteoarthritis (KOA), which significantly impacts the elderly’s quality of life. Traditional Chinese Medicine (TCM) massage has been effective in alleviating KOA symptoms; however, the physician-patient ratio and the physical demands on practitioners pose challenges. This study introduces the KOA massage robot, designed to replicate the seated knee adjustment manipulation, a specific TCM technique. The robot’s structural design and a Particle Swarm Optimization-Back Propagation (PSO-BP) algorithm are integrated to reduce the manipulation required by physicians and to assist in KOA massage treatment. The robot’s force prediction accuracy was determined to be 5.34N on average, and its therapeutic efficacy was supported by Surface Electromyography (sEMG) and Visual Analog Scale (VAS) assessments, demonstrating pain relief and improved quadriceps muscle activation in KOA patients. The experimental validation involved a comparison between traditional manual massage and the robotic intervention. The results showed that the robot could achieve an average increase in integrated EMG (iEMG) of 47%, closely mirroring the 45.37% increase observed in the manual treatment group. Similarly on VAS scores, the robotic intervention group obtained a decrease of 28.12%. This study’s findings not only highlight the potential of integrating TCM principles with modern robotics but also pave the way for a new paradigm in elderly care, where personalized and efficient KOA management is within reach.
... In addition, the classification results of this model show either "gastritis" or "no gastritis", which does not relate to either a disease term or diagnosis in TCM language. In another deep learning model, the analysis of tongue color outperformed conventional imaging processing methods that lacked deep learning techniques [52]. Thus, these deep learning models do not relate to the positioning of tongue features and are therefore unsuitable for application in TCM clinical and teaching environments. ...
Background
Tongue diagnosis in traditional Chinese medicine (TCM) provides clinically important, objective evidence from direct observation of specific features that assist with diagnosis. However, the current interpretation of tongue features requires a significant amount of manpower and time. TCM physicians may have different interpretations of features displayed by the same tongue. An automated interpretation system that interprets tongue features would expedite the interpretation process and yield more consistent results.
Materials and methods
This study applied deep learning visualization to tongue diagnosis. After collecting tongue images and corresponding interpretation reports by TCM physicians in a single teaching hospital, various tongue features such as fissures, tooth marks, and different types of coatings were annotated manually with rectangles. These annotated data and images were used to train a deep learning object detection model. Upon completion of training, the position of each tongue feature was dynamically marked.
Results
A large high-quality manually annotated tongue feature dataset was constructed and analyzed. A detection model was trained with average precision (AP) 47.67%, 58.94%, 71.25% and 59.78% for fissures, tooth marks, thick and yellow coatings, respectively. At over 40 frames per second on a NVIDIA GeForce GTX 1060, the model was capable of detecting tongue features from any viewpoint in real time.
Conclusions/Significance
This study constructed a tongue feature dataset and trained a deep learning object detection model to locate tongue features in real time. The model provided interpretability and intuitiveness that are often lacking in general neural network models and implies good feasibility for clinical application.
... How to digitize, quantify, extract, and analyze tongue features in traditional Chinese medicine theory is the key to achieving digitalization and objectification of tongue diagnosis. In recent years, many scholars have started to study traditional Chinese medicine tongue diagnosis and proposed research methods to achieve objectification and intelligence of traditional Chinese medicine tongue diagnosis, achieving some results mainly including research on tongue image segmentation, texture features [16], [17] , color features [18], [19], [20], [21], and shape features [22], [23]. ...
Tongue diagnosis is a non-invasive, painless diagnostic method by observing the tongue image of patients to diagnose and analyze their pathological conditions, which provides an opportunity for the future development of tongue diagnosis. However, the traditional tongue diagnosis method mainly relies on the experience and judgment of doctors, and is also easily affected by external factors. These factors hinder the development and application of tongue diagnosis. Currently, most studies use machine learning, which is time consuming and labor intensive. Other studies use deep learning based on convolutional neural network (CNN), but the affine transformation of CNN is less robust and easily loses the spatial relationship between features. In this work, we propose a traditional Chinese medicine (TCM) syndrome classification model of skin diseases based on tongue image hierarchical feature fusion. By adding a multi-scale residual module to the feature extraction part of the capsule network, we can extracted richer feature of tongue image. At the same time, the attention mechanism module is embedded in the multi-scale residual module, with the help of the attention mechanism module, the interference of tongue impurity information is suppressed, and accurate features are extracted for classification. Through experiments, it has been proven that our proposed method has achieved accuracy of 89.6\% in the classification of tongue for acne syndrome, and accuracy of 91.6\% in the dermatitis syndrome.
... With the rapid development of deep learning, researchers have tried to use its powerful feature extraction and semantic representative capabilities to im-prove the performance of tongue color classification in TCM [2], [3]. Hou et al. [4] constructed a tongue image dataset and used a modified CaffeNet network to classify the tongue color. ...
Tongue color is an important tongue diagnostic index for traditional Chinese medicine (TCM). Due to the individual experience of TCM experts as well as ambiguous boundaries among the tongue color categories, there often exist noisy labels in annotated samples. Deep neural networks trained with the noisy labeled samples often have poor generalization capability because they easily overfit on noisy labels. A novel framework named confident-learning-assisted knowledge distillation (CLA-KD) is proposed for tongue color classification with noisy labels. In this framework, the teacher network plays two important roles. On the one hand, it performs confident learning to identify, cleanse and correct noisy labels. On the other hand, it learns the knowledge from the clean labels, which will then be transferred to the student network to guide its training. Moreover, we elaborately design a teacher network in an ensemble manner, named E-CA
2
-ResNet18, to solve the unreliability and instability problem resulted from the insufficient data samples. E-CA
2
-ResNet18 adopts ResNet18 as the backbone, and integrates channel attention (CA) mechanism and activate or not activation function together, which facilitates to yield a better performance. The experimental results on three self-established TCM tongue datasets demonstrate that, our proposed CLA-KD can obtain a superior classification accuracy and good robustness with a lower network model complexity, reaching 94.49%, 92.21%, 93.43% on the three tongue image datasets, respectively.
... Tongue segmentation algorithm is devised based on the proposed method by Sawabe et al. [18] in 2006. Similar approach by Hou et al. [19] is proposed using grayscale images to discriminate between foreground and background image. This segmentation algorithm employs Hue, Saturation and Value (HSV) color space as threshold variable. ...
Tongue inspection is a complementary diagnosis method that widely used in Traditional Chinese Medicine (TCM) by inspecting the tongue body constitution to decide the physiological and pathological functions of the human body. Since tongue manifestation is done by practitioner’s observation using naked eye, many limitations can affect the diagnosis result including environment condition and experiences of the practitioner. Lately, tongue diagnosis has been widely studied in order to solve these limitations via digital system. However, most of recent digital system are bulky and not equipped with intelligent diagnosis system that can finally predict the health status of the patient. In this research, digital tongue diagnosis system that uses intelligent diagnosis consisted of image segmentation analysis, tongue coating recognition analysis, and tongue color classification has been embedded on Raspberry Pi. Tongue segmentation implements Hue, Saturation and Value (HSV) color space with Brightness Conformable Multiplier (BCM) for adaptive brightness filtering to recognized tongue body accurately while eliminating perioral area. Tongue Coating Recognition uses threshold method to detect tongue coating and eliminate the unwanted features including shadow. Tongue color classification uses hybrid method consisted of k-means clustering and Support Vector Machine (SVM) to classify between red, light red and deep red tongue and further gave diagnosis based on color. This experiment concluded that it is feasible to embed the algorithm on Raspberry Pi to promote system portability while attaining similar accuracy for future telemedicine.
... Furthermore, AI can be used to diagnose breast cancer, and it can surpass human experts in breast cancer prediction by reducing false negatives and false positives [21]. However, the method for the determination of the sublingual varices severity has not been effectively developed; most existing research works have focused on tongue surface-related topics [22][23][24][25][26]. This study aims at developing a computerassisted system based on machine learning (ML) techniques to diagnose the severity of sublingual varicose veins. ...
Tongue diagnosis, a noninvasive examination, is an essential step for syndrome differentiation and treatment in traditional Chinese medicine (TCM). Sublingual vein (SV) is examined to determine the presence of blood stasis and blood stasis syndrome. Many studies have shown that the degree of SV stasis positively correlates with disease severity. However, the diagnoses of SV examination are often subjective because they are influenced by factors such as physicians’ experience and color perception, resulting in different interpretations. Therefore, objective and scientific diagnostic approaches are required to determine the severity of sublingual varices. This study aims at developing a computer-assisted system based on machine learning (ML) techniques for diagnosing the severity of sublingual varicose veins. We conducted a comparative study of the performance of several supervised ML models, including the support vendor machine, K-neighbor, decision tree, linear regression, and Ridge classifier and their variants. The main task was to differentiate sublingual varices into mild and severe by using images of patients’ SVs. To improve diagnostic accuracy and to accelerate the training process, we proposed using two model reduction techniques, namely, the principal component analysis in conjunction with the slice inverse regression and the convolution neural network (CNN), to extract valuable features during the preprocessing of data. Our results showed that these two extraction methods can reduce the training time for the ML methods, and the Ridge-CNN method can achieve an accuracy rate as high as 87.5%, which is similar to that of experienced TCM physicians. This computer-aided tool can be used for reference clinical diagnosis. Furthermore, it can be employed by junior physicians to learn and to use in clinical settings.
... Combing the characteristics of basic image processing and deep learning, Fu et al. [8] presented a computerized tongue coating nature diagnosis method using deep neural networks. Hou et al. [9] proposed a neural network for tongue color classification, which is more practical and accurate than the traditional one. Although the previous studies have attained a high level of accuracy, they only considered single-modal data and only a portion of patients' information. ...
Cerebral palsy is one of the most prevalent neurological disorders and the most frequent cause of disability. Identifying the syndrome by patients’ symptoms is the key to traditional Chinese medicine (TCM) cerebral palsy treatment. Artificial intelligence (AI) is advancing quickly in several sectors, including TCM. AI will considerably enhance the dependability and precision of diagnoses, expanding effective treatment methods’ usage. Thus, for cerebral palsy, it is necessary to build a decision-making model to aid in the syndrome diagnosis process. While the recurrent neural network (RNN) model has the potential to capture the correlation between symptoms and syndromes from electronic medical records (EMRs), it lacks TCM knowledge. To make the model benefit from both TCM knowledge and EMRs, unlike the ordinary training routine, we begin by constructing a knowledge-based RNN (KBRNN) based on the cerebral palsy knowledge graph for domain knowledge. More specifically, we design an evolution algorithm for extracting knowledge in the cerebral palsy knowledge graph. Then, we embed the knowledge into tensors and inject them into the RNN. In addition, the KBRNN can benefit from the labeled EMRs. We use EMRs to fine-tune the KBRNN, which improves prediction accuracy. Our study shows that knowledge injection can effectively improve the model effect. The KBRNN can achieve 79.31% diagnostic accuracy with only knowledge injection. Moreover, the KBRNN can be further trained by the EMRs. The results show that the accuracy of fully trained KBRNN is 83.12%.
... Machine learning has been widely used in the field of tongue diagnosis, but the current research focuses on the field of supervised learning, which requires manual calibration of the tongue image [6][7][8]. It is not difficult to label data with clear diagnostic criteria. ...
Background. The prevalence of diabetes increases year by year, posing a severe threat to human health. Current treatments are difficult to prevent the progression of diabetes and its complications. It is imperative to carry out individualized treatment of diabetes, but current diagnostic methods are difficult to specify an individualized treatment plan. Objective. Clarify the distribution law of tongue features of the diabetic population, and provide the diagnostic basis for individualized treatment of traditional Chinese medicine (TCM) in the treatment of diabetes. Methods. We use the TFDA-1 tongue diagnosis instrument to collect tongue images of people with diabetes and accurately calculate the color features, texture features, and tongue coating ratio features through the Tongue Diagnosis Analysis System (TDAS). Then, we used K-means and Self-organizing Maps (SOM) networks to analyze the distribution of tongue features in diabetic people. Statistical analysis of TDAS features was used to identify differences between clusters. Results. The silhouette coefficient of the K-means clustering result is 0.194, and the silhouette coefficient of the SOM clustering result is 0.127. SOM Cluster 3 and Cluster 4 are derived from K-means Cluster 1, and the intersections account for (76.7% 97.5%) and (22.3% and 70.4%), respectively. K-means Cluster 2 and SOM Cluster 1 are highly overlapping, and the intersection accounts for the ratios of 66.9% and 95.0%. K-means Cluster 3 and SOM Cluster 2 are highly overlaid, and the intersection ratio is 94.1% and 82.1%. For the clustering results of K-means, TB-a and TC-a of Cluster 3 are the highest (P
... The preprocessing of images is a technique that reshapes and processes captured raw images to convert them to images suitable for judging by machine learning. Conversion methods include processing that removes patterns such as shading and noise deemed unnecessary for making judgments [9][10][11] and processing that extracts feature patterns [12][13][14]. In our proposed method, we consider the extraction of carbide regions as material for judging quality to be an effective approach. ...
Quality control of special steel is accomplished through visual inspection of its microstructure based on microscopic images. This study proposes an “automatic-quality-level-estimation system” based on machine learning. Visual inspection of this type is sensory-based, so training data may include variations in judgments and training errors due to individual differences between inspectors, which makes it easy for a drop in generalization performance to occur due to overfitting. To deal with this issue, we here propose the preprocessing of inspection images and a data augmentation technique. Preprocessing reduces variation in images by extracting features that are highly related to the level of quality from inspection images. Data augmentation, meanwhile, suppresses the problem of overfitting when training with a small number of images by taking into account information on variation in judgment values obtained from on-site experience. While the correct-answer rate for judging quality level by an inspector was about 90%, the proposed method achieved a correct-answer rate of 92.5%, which indicates that the method shows promise for practical application.
... Based on the collaborative machine learning approach in the field of IoT eHealth architecture, Ref. [123] reviewed arrhythmia detection by the use of CNN. The classification of tongue color based on CNN was studied in [124]; for training and testing images, they used CNN. Their experimental results showed that as the dataset increases, the accuracy becomes higher. ...
Advances in technology have been able to affect all aspects of human life. For example, the use of technology in medicine has made significant contributions to human society. In this article, we focus on technology assistance for one of the most common and deadly diseases to exist, which is brain tumors. Every year, many people die due to brain tumors; based on “braintumor” website estimation in the U.S., about 700,000 people have primary brain tumors, and about 85,000 people are added to this estimation every year. To solve this problem, artificial intelligence has come to the aid of medicine and humans. Magnetic resonance imaging (MRI) is the most common method to diagnose brain tumors. Additionally, MRI is commonly used in medical imaging and image processing to diagnose dissimilarity in different parts of the body. In this study, we conducted a comprehensive review on the existing efforts for applying different types of deep learning methods on the MRI data and determined the existing challenges in the domain followed by potential future directions. One of the branches of deep learning that has been very successful in processing medical images is CNN. Therefore, in this survey, various architectures of CNN were reviewed with a focus on the processing of medical images, especially brain MRI images.
... The tongue can be classified by examining its color, shape, texture and other features. For exmaple, Hou et al. [1] performed tongue color classification by modified CaffeNet [2]. Tooth-marked tongue is identified according to whether there are tooth marks on the edge of the tongue. ...
Tongue coating can provide valuable diagnostic information to reveal the disorder of the internal body. However, tongue coating classification has long been a challenging task in Traditional Chinese Medicine (TCM) due to the fact that tongue coatings are polymorphous, different tongue coatings have different colors, shapes, textures and locations. Most existing analyses utilize handcrafted features extracted from a fixed location, which may lead to inconsistant performance when the size or location of the tongue coating region varies. To solve this problem, this paper proposes a novel paradigm by employing artificial intelligence to feature extraction and classification of tongue coating. It begins with exploiting prior knowledge of rotten-greasy tongue coating to obtain suspected tongue coating patches. Based on the resulting patches, tongue coating features extracted by Convolutional Neural Network (CNN) are used instead of handcrafted features. Moreover, a multiple-instance Support Vector Machine (MI-SVM) which can circumvent the uncertain location problem is applied to tongue coating classification. Experimental results demonstrate that the proposed method outperforms state-of-the-art tongue coating classification methods.
... CNN is widely used for problems based on computer vision like fast fashion guided clothing [24], safety harness detection [25], image reconstruction problems like HDR [26], and CT image reconstruction [27], color recognition problems like vehicle [28], and tongue [29] color recognition and image classification problems like indoor-outdoor images [30], and city-landscape images classification [31]. ...
Malware is a threat to people in the cyber world. It steals personal information and harms computer systems. Various developers and information security specialists around the globe continuously work on strategies for detecting malware. From the last few years, machine learning has been investigated by many researchers for malware classification. The existing solutions require more computing resources and are not efficient for datasets with large numbers of samples. Using existing feature extractors for extracting features of images consumes more resources. This paper presents a Convolutional Neural Network model with pre-processing and augmentation techniques for the classification of malware gray-scale images. An investigation is conducted on the Malimg dataset, which contains 9339 gray-scale images. The dataset created from binaries of malware belongs to 25 different families. To create a precise approach and considering the success of deep learning techniques for the classification of raising the volume of newly created malware, we proposed CNN and Hybrid CNN+SVM model. The CNN is used as an automatic feature extractor that uses less resource and time as compared to the existing methods. Proposed CNN model shows (98.03%) accuracy which is better than other existing CNN models namely VGG16 (96.96%), ResNet50 (97.11%) InceptionV3 (97.22%), Xception (97.56%). The execution time of the proposed CNN model is significantly reduced than other existing CNN models. The proposed CNN model is hybridized with a support vector machine. Instead of using Softmax as activation function, SVM performs the task of classifying the malware based on features extracted by the CNN model. The proposed fine-tuned model of CNN produces a well-selected features vector of 256 Neurons with the FC layer, which is input to SVM. Linear SVC kernel transforms the binary SVM classifier into multi-class SVM, which classifies the malware samples using the one-against-one method and delivers the accuracy of 99.59%.
... Some studies have applied deep learning to the analysis of tongue images, but deep learning is yet to be applied to the clinical interpretation of tongue diagnoses in TCM. For example, Meng et al. designed the CHDNet model, which combined deep learning and support vector machine classifiers to extract and classify tongue features [13]. However, the digital features extracted by this model did not visualize the tongue features mentioned in TCM. ...
Background Traditional Chinese medicine (TCM) describes physiological and pathological changes inside and outside the human body by the application of four methods of diagnosis. One of the four methods, tongue diagnosis, is widely used by TCM physicians, since it allows direct observations that prevent discrepancies in the patient’s history and, as such, provides clinically important, objective evidence. The clinical significance of tongue features has been explored in both TCM and modern medicine. However, TCM physicians may have different interpretations of the features displayed by the same tongue, and therefore intra- and inter-observer agreements are relatively low. If an automated interpretation system could be developed, more consistent results could be obtained, and learning could also be more efficient. This study will apply a recently developed deep learning method to the classification of tongue features, and indicate the regions where the features are located. Methods A large number of tongue photographs with labeled fissures were used. Transfer learning was conducted using the ImageNet-pretrained ResNet50 model to determine whether tongue fissures were identified on a tongue photograph. Often, the neural network model lacks interpretability, and users cannot understand how the model determines the presence of tongue fissures. Therefore, Gradient-weighted Class Activation Mapping (Grad-CAM) was also applied to directly mark the tongue features on the tongue image. Results Only 6 epochs were trained in this study and no graphics processing units (GPUs) were used. It took less than 4 minutes for each epoch to be trained. The correct rate for the test set was approximately 70%. After the model training was completed, Grad-CAM was applied to localize tongue fissures in each image. The neural network model not only determined whether tongue fissures existed, but also allowed users to learn about the tongue fissure regions. Conclusions This study demonstrated how to apply transfer learning using the ImageNet-pretrained ResNet50 model for the identification and localization of tongue fissures and regions. The neural network model built in this study provided interpretability and intuitiveness, (often lacking in general neural network models), and improved the feasibility for clinical application.
... Some studies have applied deep learning to the analysis of tongue images, but deep learning is yet to be applied to the clinical interpretation of tongue diagnoses in TCM. For example, Meng et al. designed the CHDNet model, which combined deep learning and support vector machine classifiers to extract and classify tongue features [13]. However, the digital features extracted by this model did not visualize the tongue features mentioned in TCM. ...
Background: Traditional Chinese medicine (TCM) describes physiological and pathological changes inside and outside the human body by the application of four methods of diagnosis. One of the four methods, tongue diagnosis, is widely used by TCM physicians, since it allows direct observations that prevent discrepancies in the patient’s history and, as such, provides clinically important, objective evidence. The clinical significance of tongue features has been explored in both TCM and modern medicine. However, TCM physicians may have different interpretations of the features displayed by the same tongue, and therefore intra- and inter-observer agreements are relatively low. If an automated interpretation system could be developed, more consistent results could be obtained, and learning could also be more efficient. This study will apply a recently developed deep learning method to the classification of tongue features, and indicate the regions where the features are located.
Methods: A large number of tongue photographs with labeled fissures were used. Transfer learning was conducted using the ImageNet-pretrained ResNet50 model to determine whether tongue fissures were identified on a tongue photograph. Often, the neural network model lacks interpretability, and users cannot understand how the model determines the presence of tongue fissures. Therefore, Gradient-weighted Class Activation Mapping (Grad-CAM) was also applied to directly mark the tongue features on the tongue image.
Results: Only 6 epochs were trained in this study and no graphics processing units (GPUs) were used. It took less than 4 minutes for each epoch to be trained. The correct rate for the test set was approximately 70%. After the model training was completed, Grad-CAM was applied to localize tongue fissures in each image. The neural network model not only determined whether tongue fissures existed, but also allowed users to learn about the tongue fissure regions.
Conclusions: This study demonstrated how to apply transfer learning using the ImageNet-pretrained ResNet50 model for the identification and localization of tongue fissures and regions. The neural network model built in this study provided interpretability and intuitiveness, (often lacking in general neural network models), and improved the feasibility for clinical application.
... Fu et al. [46] discussed the challenging aspects of capturing the nature of a tongue's coating; they proposed using neural networks that combine the characteristics of basic image processing and deep learning, based on a standard and balanced tongue image dataset. Hou et al. [47] also proposed a method to combine deep learning with a classification system that recognizes tongue color with a high accuracy, which is essential for effective tongue diagnosis in a new era of big data. Zhang et al. [48] constructed an auxiliary pulse sensor management system consisting of acquisition equipment, software, and an Android app for daily TCM data management via mobile terminals. ...
Big data and traditional Chinese medicine (TCM) is a new interdisciplinary field quietly emerging in Chinese society. Internet of Things (IoT) sensor system technology is currently being developed to gather large volumes of personal data so that TCM, specifically herbal pharmaceutics, can be applied to treat acute and chronic diseases alike utilizing low cost, safe, and effective treatment protocols that have been prescribed in clinical practice for thousands of years. Through a survey of existing literature, this paper investigates what a future state of medicine may look like with the deployment of a big data TCM system as a means to enhance humankind’s health and quality of life.
... Especially, in the view of algorithms, these AI-assisted techniques can be recognized by two different approaches: pattern classification and knowledge mining. e former technology attempts to recognize the correct pathological information such as pulse condition [9][10][11][12][13][14] and tongue diagnosis [15] of an individual patient. However, the later one, knowledge mining, mainly focuses on finding out various kinds of hidden relationships in the knowledge, for example, the relationships between symptom and symptom, symptom and syndrome, and syndrome and disease [16][17][18][19][20]. ...
In this work, an ontology-based model for AI-assisted medicine side-effect (SE) prediction is developed, where three main components, including the drug model, the treatment model, and the AI-assisted prediction model, of the proposed model are presented. To validate the proposed model, an ANN structure is established and trained by two hundred forty-two TCM prescriptions. .ese data are gathered and classified from the most famous ancient TCM book, and more than one thousand SE reports, in which two ontology-based attributions, hot and cold, are introduced to evaluate whether the prescription will cause SE or not. .e results preliminarily reveal that it is a relationship between the ontology-based attributions and the corresponding predicted indicator that can be learnt by AI for predicting the SE, which suggests the proposed model has a potential in AI-assisted SE prediction. However, it should be noted that the proposed model highly depends on the sufficient clinic data, and hereby, much deeper exploration is important for enhancing the accuracy of the prediction.
... Especially, in the view of algorithms, these AI-assisted techniques can be recognized by two different approaches: pattern classification and knowledge mining. e former technology attempts to recognize the correct pathological information such as pulse condition [9][10][11][12][13][14] and tongue diagnosis [15] of an individual patient. However, the later one, knowledge mining, mainly focuses on finding out various kinds of hidden relationships in the knowledge, for example, the relationships between symptom and symptom, symptom and syndrome, and syndrome and disease [16][17][18][19][20]. ...
In this work, an ontology-based model for AI-assisted medicine side-effect (SE) prediction is developed, where three main components, including the drug model, the treatment model, and the AI-assisted prediction model, of proposed model are presented. To validate the proposed model, an ANN structure is established and trained by two hundred and forty-two TCM prescriptions that are gathered and classified from the most famous ancient TCM book and more than one thousand SE reports, in which two ontology-based attributions, hot and cold, are simply introduced to evaluate whether the prediction will cause a SE or not. The results preliminarily reveal that it is a relationship between the ontology-based attributions and the corresponding indicator that can be learnt by AI for predicting the SE, which suggests the proposed model has a potential in AI-assisted SE prediction. However, it should be noted that, the proposed model highly depends on the sufficient clinic data, and hereby, much deeper exploration is important for enhancing the accuracy of the prediction.
Traditional Chinese medicine distinguishes tongue features such as tongue color, fur color, tongue shape and crack mainly through the visual observation and empirical analysis of traditional Chinese medicine doctors. Therefore, the judgment standard will be affected by the subjective factors of doctors and surrounding environment. These factors restrict the application and development of tongue diagnosis. Therefore, objectifying tongue diagnosis information and standardizing diagnosis is an important direction of tongue diagnosis automation research. This paper presents a classification method of TCM tongue image based on multi feature fusion. By constructing a multi feature fusion model, two sub networks are used to classify the different features of the tongue image, so as to realize the task of multi feature classification of the tongue image. The model classifies the tongue image into tongue color classification, fur color classification, tongue shape classification and crack classification, and outputs the color parameters of tongue color and fur color while outputting the classification results. The model adds the method of transfer learning, which can reduce the demand for the amount of tongue image data, At the same time, the accuracy of the model is improved.
This chapter starts from primary machine learning concepts and deals with cutting-edge meta-learning models in the context of diverse modalities of healthcare data, including medical imaging, electrocardiography, time-series data with nonuniform time intervals, multimodal data, facial imaging captured from a video camera, and others. This chapter investigates 15 interesting contributions to the field, all of which can be divided into three taxonomies according to their tasks to solve: medical imaging analysis (image classification, lesion classification, image segmentation, and image reconstruction), computing based on EHRs, and applications in emerging areas (cardiac and disease diagnosis). In-depth overviews are offered of studies per practice area: breast, chest, cardiac, tongue, abdominal, pulmonary, and dermatological. Finally, future direction and open questions of meta-learning for medical and healthcare applications are discussed.
Objective
To propose two novel methods based on deep learning for computer-aided tongue diagnosis, including tongue image segmentation and tongue color classification, improving their diagnostic accuracy.
Methods
LabelMe was used to label the tongue mask and Snake model to optimize the labeling results. A new dataset was constructed for tongue image segmentation. Tongue color was marked to build a classified dataset for network training. In this research, the Inception + Atrous Spatial Pyramid Pooling (ASPP) + UNet (IAUNet) method was proposed for tongue image segmentation, based on the existing UNet, Inception, and atrous convolution. Moreover, the Tongue Color Classification Net (TCCNet) was constructed with reference to ResNet, Inception, and Triple-Loss. Several important measurement indexes were selected to evaluate and compare the effects of the novel and existing methods for tongue segmentation and tongue color classification. IAUNet was compared with existing mainstream methods such as UNet and DeepLabV3+ for tongue segmentation. TCCNet for tongue color classification was compared with VGG16 and GoogLeNet.
Results
IAUNet can accurately segment the tongue from original images. The results showed that the Mean Intersection over Union (MIoU) of IAUNet reached 96.30%, and its Mean Pixel Accuracy (MPA), mean Average Precision (mAP), F1-Score, G-Score, and Area Under Curve (AUC) reached 97.86%, 99.18%, 96.71%, 96.82%, and 99.71%, respectively, suggesting IAUNet produced better segmentation than other methods, with fewer parameters. Triplet-Loss was applied in the proposed TCCNet to separate different embedded colors. The experiment yielded ideal results, with F1-Score and mAP of the TCCNet reached 88.86% and 93.49%, respectively.
Conclusion
IAUNet based on deep learning for tongue segmentation is better than traditional ones. IAUNet can not only produce ideal tongue segmentation, but have better effects than those of PSPNet, SegNet, UNet, and DeepLabV3+, the traditional networks. As for tongue color classification, the proposed network, TCCNet, had better F1-Score and mAP values as compared with other neural networks such as VGG16 and GoogLeNet.
In Traditional Chinese Medicine (TCM), tongue diagnosis is essential for symptom differentiation and treatment selection. Compared with traditional tongue diagnostic instruments, deploying a tongue diagnosis system on mobile devices is more convenient to monitor general health and facilitates the development of telemedicine. However, limited by both the quality and quantity of tongue images taken by mobile devices, extracting tongue features of the images maintains a great challenge. In this paper, we present a tongue feature extraction method on mobile devices, containing a high-accuracy tongue segmentation method based on Moment Invariants with Data Augmentation (DAMI) and an efficient and lightweight feature classification model with an attention mechanism. Meanwhile, we construct a novel tongue image dataset from mobile devices for extracting tongue features, significantly, first including sublingual images which are beneficial to extracting sublingual vein features. Extensive experiments on two datasets demonstrate the effectiveness and robustness of our method. Furthermore, our method greatly reduces the computing and storage demands compared with other current methods, providing a good prerequisite for deployment on mobile devices. Finally, to demonstrate the potential application of our proposed method, we develop a TCM intelligence tongue diagnosis application, which can be accessed through the WeChat Mini Program or web version, exhibiting its great potential in clinical diagnosis and health monitoring.
In the field of traditional Chinese medicine (TCM) informatics, Chinese word segmentation and syndrome differentiation are two crucial analysis tasks. Owing to the ambiguity of the Chinese language and the peculiarities of syndrome differentiation, these tasks face huge challenges. Notably, from previous studies and investigations, these two tasks have a high correlation, which makes them fit the idea of multi-task joint learning (MTL). By sharing the underlying parameters and adding two different task loss functions, we proposed a novel MTL method to perform segmentation and classification of medical records in this research. Moreover, two classic deep neural network (Bidirectional LSTM (Bi-LSTM) and TextCNN) are fused into the MTL to conduct these two tasks simultaneously. As far as we know, our approach is the first attempt to combine these tasks with the idea of MTL. We used our proposed method to conduct a large number of comparative experiments. Through experimental comparison, it can be proved that our method is superior to other methods on both tasks. Therefore, this research can help to realize the modernization of TCM and the intelligent differentiation of TCM.KeywordsTraditional Chinese medicineChinese word segmentationSyndrome differentiationMulti-task joint learningDeep learning
Traditional Chinese Medicine (TCM), as an effective alternative medicine, utilizes tongue diagnosis as a major method to assess the patient’s health status by examining the tongue’s color, shape, and texture. Tongue images can also give the pre-disease indications without any significant disease symptoms, which provides a basis for preventive medicine and lifestyle adjustment. However, traditional tongue diagnosis has limitations, as the process may be subjective and inconsistent. Hence, computer-aided tongue diagnoses have a great potential to provide more consistent and objective health assessments. This paper reviewed the current trends in TCM tongue diagnosis, including tongue image acquisition hardware, tongue segmentation, feature extraction, color correction, tongue classification, and tongue diagnosis system. We also present a case of TCM constitution classification based on tongue images.
Traditional Chinese Medicine (TCM) is a well-established medical system with a long history. Currently, artificial intelligence (AI) is rapidly expanding in many fields including TCM. AI will significantly improve the reliability and accuracy of diagnostics, thus increasing the use of effective therapeutic methods for patients. This systematic review provides an updated overview on the major breakthroughs in the field of AI-assisted TCM four diagnostic methods, syndrome differentiation, and treatment. AI-assisted TCM diagnosis is mainly based on digital data collected by modern electronic instruments, which makes TCM diagnosis more quantitative, objective, and standardized. As a result, the diagnosis decisions made by different TCM doctors exhibit more consistency, accuracy, and reliability. Meanwhile, the therapeutic efficacy of TCM can be evaluated objectively. Therefore, AI is promoting TCM from experience to evidence-based medicine, a genuine scientific revolution. Furthermore, huge and non-uniform knowledge on formula-syndrome relationships and the combination rules of herbal TCM formulae could be better standardized with the help of AI analysis, which is necessary for the clinical efficacy evaluation and further optimization on the standardized TCM formulae. AI bridges the gap between TCM and modern science and technology. AI may bring clinical TCM diagnostics closer to western medicine. With the help of AI, more scientific evidence about TCM will be discovered. It can be expected that more unified guidelines for specific TCM syndromes will be issued with the development of AI-assisted TCM therapies in the future.
Tongue diagnosis is one of the primary clinical diagnostic methods in Traditional Chinese Medicine. Recognizing the tooth‐marked tongue and the crackled tongue plays an essential role in evaluating the status of patients. Previous methods mainly focus on identifying whether a tongue image is a tooth‐marked tongue (cracked tongue) or not, while cannot provide more details. In this study, we propose a weakly supervised method for training the tooth‐mark and crack detection model by leveraging fully bounding‐box level annotated and coarse image‐level annotated tongue images. The proposed model is extended from the YOLO object detection model, and we add several classification branches for recognizing the tooth‐marked tongue and cracked tongue. The classification branch aims to predict the coarse label for both coarse‐labeled data and fully annotated data. The detection branch is used to locate the position of tooth marks and cracks from the fully annotated data. Finally, we utilize a multitask loss function for training the model. Experimental results on a challenging tongue image dataset demonstrate the effectiveness of our proposed weakly supervised method.
Automatic tongue image segmentation and tongue classification are two crucial tongue characterization tasks in traditional Chinese medicine (TCM). Due to the complexity of automatic tongue segmentation and the fine-grained traits of tongue classification, both tasks are challenging. However, as discussed in the introduction section, these two tasks are interrelated, making them highly compatible with the idea of multitask joint learning (MTL). By sharing the underlying parameters and adding two different task objective functions, a MTL method for segmenting and classifying tongue images is proposed in this paper. Moreover, two state-of-the-art deep neural network variants (UNET and discriminative filter bank (DFL)) are fused into the MTL to perform the tongue segmentation and tongue classification tasks, respectively. To the best of our knowledge, our method is the first attempt to manage both tasks simultaneously with MTL. We conducted extensive experiments on reliable and quality assured datasets. The experimental results show that our joint method outperforms both the existing tongue segmentation methods and the existing tongue classification methods. Visualizations and ablation studies are provided to aid in understanding our approach, which suggest that our method is highly consistent with human perception
Purpose
Studies have shown the association between tongue color and diseases. To help clinicians make more objective and accurate decisions quickly, we take unsupervised learning to deal with the basic clustering of tongue color in a 2D way.
Methods
A total of 595 typical tongue images were analyzed. The 3D information extracted from the image was transformed into 2D information by principal component analysis (PCA). K-Means was applied for clustering into four diagnostic groups. The results were evaluated by clustering accuracy (CA), Jaccard similarity coefficient (JSC), and adjusted rand index (ARI).
Results
The new 2D information totally retained 89.63% original information in the L*a*b* color space. And our methods successfully classified tongue images into four clusters and the CA, ARI, and JSC were 89.04%, 0.721, and 0.890, respectively.
Conclusions
The 2D information of tongue color can be used for clustering and to improve the visualization. K-Means combined with PCA could be used for tongue color classification and diagnosis. Methods in the paper might provide reference for the other research based on image diagnosis technology.
With the improvement of living standards, people are paying more attention to healthcare, but there is still a long way to go to improve healthcare. A usable, intelligent aided diagnosis measure can be helpful for people to achieve daily health management. Several studies suggested that tongue features can directly reflect a person’s physical state. In this paper, we apply tongue diagnosis to daily health management. To this end, this paper proposes and implements a classification model of tongue image syndromes based on convolutional neural network and carries out an experiment to verify the feasibility and stability of the model. Finally, a tongue diagnosis platform that can be used for daily health management is implemented. In the two-class experiment, our model has achieved a good result. In addition, our model performs better on classifying the tongue image syndrome compared with traditional machine learning methods.
The vast majority of indoor navigation algorithms either rely on manual scene augmentation and labelling or exploit multi-sensor fusion techniques in achieving simultaneous localization and mapping (SLAM), leading to high computational costs, hardware complexities and robustness deficiencies. This paper proposes an efficient and robust deep learning-based indoor navigation framework for robots. Firstly, we put forward an end-to-end trainable siamese deep convolutional neural network (DCNN) which decomposes navigation into orientation and localization in one branch, while achieving semantic scene mapping in another. In mitigating the computational costs associated with DCNNs, the proposed model design shares a significant amount of convolutional operations between the two branches, streamlining the model and optimizing for efficiency in terms of memory and inference latency. Secondly, a transfer learning regime is explored in demonstrating how such siamese DCNNs can be efficiently trained for high convergence rates without extensive manual dataset labelling. The resulting siamese framework combines semantic scene understanding with orientation estimation towards predicting collision-free and optimal navigation paths. Experimental results demonstrate that the proposed framework achieves accurate and efficient navigation and outperforms existing "navigation-by-classification" variants.
Background and objective:
Yin and Yang, two concepts adapted from classical Chinese philosophy, play a diagnostic role in Traditional Chinese Medicine (TCM). The Yin and Yang in harmonious balance indicate health, whereas imbalances to either side indicate unhealthiness, which may result in diseases. Yin-yang disharmony is considered to be the cause of pathological changes. Syndrome differentiation of yin-yang is crucial to clinical diagnosis. It lays a foundation for subsequent medical judgments, including therapeutic methods, and formula, among many others. However, because of the complexities of the mechanisms and manifestations of disease, it is difficult to exactly point out which one, yin or yang, is disharmonious. There has been inadequate research conducted on syndrome differentiation of yin and yang from a computational perspective. In this study, we present a computational method, viz. an end-to-end syndrome differentiation of yin deficiency and yang deficiency.
Methods:
Unlike most previous studies on syndrome differentiation, which use structured datasets, this study takes unstructured texts in medical records as its inputs. It models syndrome differentiation as a task of text classification. This study experiments on two state-of-the-art end-to-end algorithms for text classification, i.e. a classic convolutional neural network (CNN) and fastText. These two systems take the n-grams of several types of tokens as their inputs, including characters, terms, and words.
Results:
When evaluated on a data set with 7326 modern medical records in TCM, it is observed that CNN and fastText generally give rise to comparable performances. The best accuracy rate of 92.55% comes from the system taking inputs as raw as n-grams of characters. It implies that one can build at least a moderate system for the differentiation of yin deficiency and yang deficiency even if he has no glossary or tokenizer at hand.
Conclusions:
This study has demonstrated the feasibility of using end-to-end text classification algorithms to differentiate yin deficiency and yang deficiency on unstructured medical records.
The latest generation of convolutional neural networks (CNNs) has achieved impressive results in the field of image classification. This paper is concerned with a new approach to the development of plant disease recognition model, based on leaf image classification, by the use of deep convolutional networks. Novel way of training and the methodology used facilitate a quick and easy system implementation in practice. The developed model is able to recognize 13 different types of plant diseases out of healthy leaves, with the ability to distinguish plant leaves from their surroundings. According to our knowledge, this method for plant disease recognition has been proposed for the first time. All essential steps required for implementing this disease recognition model are fully described throughout the paper, starting from gathering images in order to create a database, assessed by agricultural experts. Caffe, a deep learning framework developed by Berkley Vision and Learning Centre, was used to perform the deep CNN training. The experimental results on the developed model achieved precision between 91% and 98%, for separate class tests, on average 96.3%.
Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models. The framework is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Caffe fits industry and internet-scale media needs by CUDA GPU computation, processing over 40 million images a day on a single K40 or Titan GPU (approx 2 ms per image). By separating model representation from actual implementation, Caffe allows experimentation and seamless switching among platforms for ease of development and deployment from prototyping machines to cloud environments.
Caffe is maintained and developed by the Berkeley Vision and Learning Center (BVLC) with the help of an active community of contributors on GitHub. It powers ongoing research projects, large-scale industrial applications, and startup prototypes in vision, speech, and multimedia.
When a large feedforward neural network is trained on a small training set,
it typically performs poorly on held-out test data. This "overfitting" is
greatly reduced by randomly omitting half of the feature detectors on each
training case. This prevents complex co-adaptations in which a feature detector
is only helpful in the context of several other specific feature detectors.
Instead, each neuron learns to detect a feature that is generally helpful for
producing the correct answer given the combinatorially large variety of
internal contexts in which it must operate. Random "dropout" gives big
improvements on many benchmark tasks and sets new records for speech and object
recognition.
We present a fast, fully parameterizable GPU implementation of Convolutional Neural Network variants. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Our deep hierarchical architectures achieve the best published results on benchmarks for object classification (NORB, CIFAR10) and handwritten digit recognition (MNIST), with error rates of 2.53%, 19.51%, 0.35%, respectively. Deep nets trained by simple back-propagation perform better than more shallow ones. Learning is surprisingly rapid. NORB is completely trained within five epochs. Test error rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs, respectively.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif- ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implemen- tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called dropout that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry
Training Deep Neural Networks is complicated by the fact that the
distribution of each layer's inputs changes during training, as the parameters
of the previous layers change. This slows down the training by requiring lower
learning rates and careful parameter initialization, and makes it notoriously
hard to train models with saturating nonlinearities. We refer to this
phenomenon as internal covariate shift, and address the problem by normalizing
layer inputs. Our method draws its strength from making normalization a part of
the model architecture and performing the normalization for each training
mini-batch}. Batch Normalization allows us to use much higher learning rates
and be less careful about initialization. It also acts as a regularizer, in
some cases eliminating the need for Dropout. Applied to a state-of-the-art
image classification model, Batch Normalization achieves the same accuracy with
14 times fewer training steps, and beats the original model by a significant
margin. Using an ensemble of batch-normalized networks, we improve upon the
best published result on ImageNet classification: reaching 4.9% top-5
validation error (and 4.8% test error), exceeding the accuracy of human raters.
Assisting the training of deep neural networks with applications to computer vision
- A Romero
Study on tongue color analysis of digital tongue [D]
- Chen Song-He
Study on Color - and Texture - Based Retrieval Technology of Chinese [D]
- N I Hao
Fine-tuning deep convolutional networks for plant recognition
- A K Reyes
- J C Caicedo
- J E Camargo
Pattern Recognition of Tongue Colors [D]
- W U Xia