CN115634147A - Hand acupuncture point identification method based on lightweight deep learning network - Google Patents
Hand acupuncture point identification method based on lightweight deep learning network Download PDFInfo
- Publication number
- CN115634147A CN115634147A CN202211329098.6A CN202211329098A CN115634147A CN 115634147 A CN115634147 A CN 115634147A CN 202211329098 A CN202211329098 A CN 202211329098A CN 115634147 A CN115634147 A CN 115634147A
- Authority
- CN
- China
- Prior art keywords
- hand
- acupoint
- model
- palm
- acupuncture point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a hand acupuncture point identification method based on a lightweight deep learning network; the method comprises the following steps: s1: image acquisition, namely acquiring hand data in real time; s2: identifying a palm area and a palm area based on a traditional image processing method; s3: inputting the recognized palm region data into a hand acupoint recognition model based on deep learning, wherein the hand acupoint recognition model outputs 16 hand acupoint points and 2 hand acupoint regions in total; wherein: the hand acupuncture point recognition model based on deep learning is a mobilenetv 3-based ultra-light quantification model which is obtained by carrying out knowledge distillation and quantification processing on a main network mobilenetv3 model. According to the method, based on the methods such as the lightweight deep learning network and the geometric space calculation, each key acupuncture point hitting point of the hand is finally obtained, and compared with the traditional acupuncture point recognition method, the accuracy and speed are remarkably improved.
Description
Technical Field
The invention belongs to the technical field of computer vision and biomedical image processing, and particularly relates to a hand acupuncture point identification method based on a lightweight deep learning network.
Background
Different points on the human hand represent different organs of the body. Usually, we can relieve the discomfort of the body by reflecting points through the acupuncture points of the hands. For the traditional Chinese medicine treatment method, firstly, the labor cost is high, and correspondingly experienced related practitioners are less and less. And a medical practitioner who is not specialized may not be able to quickly find the corresponding position to treat the patient, reducing the effect of acupoint massage.
In recent years, the field of artificial intelligence has a lot of outstanding results on medical images, and the medical image processing direction of the technology combined with deep learning is also a popular field in recent years. However, these studies have common problems, and the models trained by using the conventional large-scale network are too huge, complex in calculation, difficult to deploy, and have no practical industrialization method.
Therefore, the device capable of providing accurate hand acupuncture point massage in real time is very important. Firstly, the Chinese medicine culture can be propagated; secondly, the speed is high, so that the pain of the patient can be relieved conveniently in real time; thirdly, in the aspect of precision, the accuracy of the acupuncture points ensures that the massage effect is more guaranteed.
Chinese patent publication No. CN 113486758A provides a method for identifying acupoints on the back of the palm and hand, which comprises 1) collecting image data of the palm and the back of the hand of the left and right hands to form an original sample; (2) calculating position information of key points of the hand; (3) Labeling the hand image, wherein the palm is marked as 0, and the back of the hand is marked as 1; (4) Carrying out data preprocessing on an original sample, classifying and dividing a training set and a verification set according to a proportion to form a training sample; (5) constructing and training a palm/back of hand classification model; obtaining a palm/back of hand discrimination result according to the model output value; (6) According to the positions of the key points and the classification result, the relative positions of the acupuncture points among the key points are determined by combining the description of the traditional Chinese medicine acupuncture point selecting positions, the coordinate values of the acupuncture points are calculated by utilizing the space geometric principle, and the positions and the names of the acupuncture points are marked in the image. The invention can solve the problems of inaccurate positioning, low speed and the like caused by certain subjectivity in manual point selection.
The publication number is CN 108938396A, an ear recognition device based on deep learning and a method thereof, use a traditional deep learning architecture, combine a traditional image processing method in deployment, and have no good effect of a lightweight deep learning model on speed and precision of the model.
However, the hand acupuncture point identification method cannot realize real-time acupuncture point identification in speed, and cannot be deployed at a mobile terminal to be popularized and used as a product. In terms of precision, the acupuncture point deducing method based on geometric calculation cannot realize accurate acupuncture point judgment, and key points may drift due to the fact that palm areas in pictures are too small when acupuncture point identification is directly carried out.
Disclosure of Invention
In order to solve the problems, the invention provides a hand acupuncture point identification method based on a lightweight deep learning network. The hand acupuncture point recognition device can solve the problem that the speed and the precision effect of the traditional hand acupuncture point recognition are poor.
According to the invention, after a palm picture is shot by a camera, palm region detection is carried out, hand acupuncture point detection is carried out by using a hand acupuncture point recognition model, regression and thermodynamic diagram prediction are carried out on the acupuncture point points of the hand, and finally 18 key acupuncture point points of the hand (including 16 hand key acupuncture points and 2 hand special acupuncture points) are output. The invention combines the traditional Chinese medicine with the current advanced learning technology to realize the acupuncture point identification of the palm.
The technical scheme of the invention is specifically introduced as follows.
A hand acupuncture point identification method based on a lightweight deep learning network comprises the following steps:
s1: image acquisition, namely acquiring hand data in real time;
s2: identifying a palm area and a palm area based on a traditional image processing method;
s3: inputting the recognized palm region data into a hand acupoint recognition model based on deep learning, wherein the hand acupoint recognition model outputs 16 hand acupoint points and 2 hand acupoint regions in total; wherein: 16 hand acupuncture points include: lung meridian, shaoshan, respiratory area, daling acupoint, large intestine meridian, pericardium meridian, triple energizer meridian, heart meridian, large intestine, heart acupoint, lung acupoint, kidney acupoint, small intestine, triple energizer, liver acupoint, and Mingmen, 2 hand acupoints including: ear, pharyngeal area and palpitations area; wherein: the hand acupuncture point recognition model based on deep learning is an ultra-light quantification model based on mobilenetv3, the hand acupuncture point recognition model is obtained by taking the mobilenetv3 model as a main network and further carrying out knowledge distillation and quantification processing on the mobilenetv3 model, logic knowledge distillation is directly carried out on output normalized coordinates in a knowledge distillation method, a knowledge distillation teacher model is resnet50, and model parameter types are changed from float32 to int8 in a quantification method. .
In the present invention, in step S2, the specific method for identifying the palm region and the palm region is as follows:
converting the data picture to be identified into a YCrCb mode, wherein Cr and Cb values meet the following conditions: cr is more than or equal to 133 and less than or equal to 173, cb is more than or equal to 77 and less than or equal to 127, namely, the skin color area is a part similar to the color of the palm, a black and white picture is generated, the black and white picture is used for obtaining the maximum outline and generating an outline picture, an approximate ellipse is obtained, and the outline picture is the palm area;
and simultaneously rotating the original picture, the black-and-white picture and the outline picture according to the ellipse angle so as to vertically place the palm, obtaining the palm center based on distance transformation according to the black-and-white picture by using the black-and-white picture as a mask according to the original picture and the black-and-white picture, and drawing an inscribed circle of the palm according to the maximum radius to obtain a palm center region.
In the invention, in step S3, when training and testing the hand acupuncture point recognition model based on deep learning, the input palm region data is preprocessed, and the preprocessing comprises the following steps: and carrying out normalization processing, random rotation and scaling for data enhancement. The normalization and standardization of data enable the model to be more rapid and better in convergence; the training data is rotated and scaled to enhance the data, which is beneficial to increasing the robustness of the data.
In the invention, in step S3, a lightweight deep neural network model mobilenetv3 is adopted in the hand acupuncture point recognition model, and the model is characterized in that depth separable convolution, linear bottlenecks, a reverse residual structure, a se module and NAS are used for carrying out module-level search and NetAdapt local search, besides, the model structure of the model is adjusted to enable the parameter number to be smaller, so that the hand acupuncture point recognition model is convenient to deploy at a moving end and recognize hand acupuncture point regions (16 acupuncture point points +2 acupuncture point regions) in real time.
In the invention, an onehand10k open source data set is adopted to train a hand acupuncture point recognition model.
In the invention, when the hand acupuncture point recognition model is trained, the DeepPose regression algorithm is adopted for convergence when the hand acupuncture point recognition model is trained, the accuracy of the model is obtained through the proportion pck correctly estimated by the key points, and the loss function is a wingloss function so as to optimize the model.
Compared with the prior art, the invention has the beneficial effects that:
according to the method, the palm area is recognized, then the palm area data enters the hand acupuncture point recognition model, most of the area of the image is the palm, the palm is divided, and then the acupuncture points are recognized, namely, priori knowledge is added to the input hand image, and the final hand acupuncture point recognition accuracy is improved.
The invention applies the lightweight deep neural network to the acupuncture point recognition and trains the reliability by a deep learning-based method
The deep learning network model is an innovative practice of subject fusion by analyzing the function of each acupoint in combination with the traditional Chinese medicine knowledge.
The method adopts a knowledge distillation and quantification method besides the lightweight model mobilenetv3, so that the model speed is higher. The model after the quantitative distillation is an ultra-light quantitative model, the model reasoning process is 2ms, the requirement of real-time detection is met, and the popularization and the use of the product are easy.
Drawings
Fig. 1 is a hand point diagram.
Fig. 2 is a schematic flow chart of the working steps of the hand acupuncture point recognition device system.
Detailed Description
The invention is further described below with reference to the figures and examples.
Example 1
As shown in fig. 1, a hand acupuncture point identification method based on a lightweight deep learning network includes the following steps:
s01 real-time acquisition of hand images
Specifically, through a Jeson Nano core development board, an IMX219 camera is matched to collect hand images in real time. (the data is data of the right hand being flattened, and only a part of the acupuncture points can be recognized if there is a fist-making posture)
And S02, processing the shot picture by using a traditional image to obtain a palm area.
Specifically, the processing method comprises the following steps: converting a data picture to be identified into an ycrcb mode, acquiring a part similar to the palm color according to the skin color in the ycrcb, generating a black-white picture, acquiring the maximum outline by using the black-white picture, generating a contour picture and obtaining an approximate ellipse, wherein the contour picture is the palm area. And simultaneously rotating the original picture, the black-white picture and the outline picture according to the ellipse angle so as to vertically place the palm as much as possible, obtaining the center of the palm based on distance transformation according to the black-white picture and the original picture by using the black-white picture as a mask, and drawing an inscribed circle of the palm according to the maximum radius.
And S03, sending the picture with the human palm as the main area to a hand acupuncture point network for recognition. 16 hand keypoints and 2 acupoint regions are identified and output. 16 hand acupuncture points include: lung meridian, shaoshan, respiratory region, daling acupoint, large intestine meridian, pericardium meridian, triple energizer meridian, heart meridian, large intestine, heart acupoint, lung acupoint, kidney acupoint, small intestine, triple energizer, liver acupoint, and Mingmen, 2 hand acupoint regions including: ear, pharyngeal area and palpitations area.
Specifically, the training process of the lightweight model comprises data processing, data enhancement, lightweight model architecture selection, lightweight model knowledge distillation and quantification and model output correction.
Specifically, the data processing section normalizes the input data (-1-1) so that the model is easily converged. The data enhancement part adopts data enhancement methods such as rotation, zooming, contrast adjustment brightness adjustment and the like to improve the robustness of the model
In particular, a lightweight model selection part adopts a lightweight model mobilenetv3. The model integrates the depth separable convolution of the mobilenetv1, greatly reduces the complexity of the model, and integrates the inverse residual structure of the linear bottleneck of the mobilenetv2 and the SE lightweight attention model. Moreover, the model adds a complementary searching technology combination, in particular to a resource-limited NAS module level searching method, so as to automatically search for proper optimal training parameters. Netadapt performs fine tuning of parameters after module determination. The model also omits the convolution layer of the last average pooling layer of the Mobilenetv2, and the complexity of the model is greatly reduced. The mobilenetv3 model is subjected to lightweight treatment, and the following two methods are included:
1) The knowledge distillation method comprises the following steps: and directly carrying out logic knowledge distillation on the output normalized coordinates. The knowledge distillation teacher model is resnet50.
(2) The quantization method comprises the following steps: the pair changes the model parameter type from float32 to int8.
When the hand acupuncture point network is trained, the onehand10k open source data set is adopted in the data set. The data set contains virtual scene hand point images and real scene hand point images. The specific conditions are shown in table 1:
TABLE 1
PCK is a general key point detection index, and is defined as the proportion of PCK representing the correct estimation of the key point, namely, the proportion of the normalized distance between the detected key point and the corresponding group route which is smaller than a set threshold value is calculated.
S05 visual traditional Chinese medicine acupoint function
Specifically, the output 16 acupoint points and 2 acupoint regions are marked on the palm picture. Meanwhile, the corresponding function of each key acupoint is displayed. For example, shao Shang acupoint can be used to treat fire-heat syndrome such as sore throat, swelling and pain of gum, and swelling and pain of eye due to blood stasis. The lung meridian has more functions, can be used for treating throat diseases, chest diseases and head diseases by regulating lung meridian acupoints, and can achieve the treatment purpose by massaging the lung meridian for treating cough, wheezing, chest distress, short breath, dyspnea, suffocation and other symptoms. The large intestine generally has the function of treating symptoms such as head, face, five sense organs and the like. Can massage large intestine channel frequently, and has effects of eliminating edema, reducing arm fat, and improving constipation. And so on.
S06 acupoint stimulation device
In particular, in addition to the function of visualizing the respective acupuncture points, supporting the user to input a corresponding condition to the device, such as stomach pain, the corresponding acupuncture points are stimulated to relieve the pain of the patient.
According to another embodiment of the invention, the method can also be used for identifying the acupuncture points on the soles of the feet, and the lightweight network can be used for applying a hand acupuncture point identification scheme to all parts of the body and can be deployed on small-sized mobile equipment, so that the method is convenient for mass production and popularization. The acupuncture points of all parts of the body can be identified and combined into a body acupuncture point identification system, so that one-stop acupuncture point identification is realized, and the popularization and the use of the traditional Chinese medicine acupuncture point theory are facilitated. The identification method adopted by the invention is 2d key point identification, and in order to ensure the accuracy in a 3d scene, the identification of 3d acupuncture points can also be concerned, so that the accuracy of acupuncture point identification in a three-dimensional space is improved.
It will be apparent to those skilled in the art that various changes and modifications can be made without departing from the inventive concept thereof, and these changes and modifications can be made without departing from the spirit and scope of the invention.
Claims (6)
1. A hand acupuncture point identification method based on a lightweight deep learning network is characterized by comprising the following steps:
s1: image acquisition, namely acquiring hand data in real time;
s2: identifying a palm area and a palm area based on a traditional image processing method;
s3: inputting the recognized palm region data into a hand acupoint recognition model based on deep learning, wherein the hand acupoint recognition model outputs 16 hand acupoint points and 2 hand acupoint regions in total; wherein: 16 hand acupuncture points include: lung meridian, shaoshan, respiratory area, daling acupoint, large intestine meridian, pericardium meridian, triple energizer meridian, heart meridian, large intestine, heart acupoint, lung acupoint, kidney acupoint, small intestine, triple energizer, liver acupoint, and Mingmen, 2 hand acupoints including: ear, pharyngeal and palpitations areas; wherein: the hand acupuncture point recognition model based on deep learning is a mobile lentetv 3-based ultra-light quantification model, the mobile lentetv 3 model is used as a main network, knowledge distillation and quantification processing are carried out on the mobile lentetv 3 model, logic knowledge distillation is directly carried out on output normalized coordinates in a knowledge distillation method, a knowledge distillation teacher model is resnet50, and model parameter types are changed from float32 to int8 in a quantification method.
2. The hand acupoint identification method of claim 1, wherein in step S2, the specific method for identifying the palm and palm region is as follows:
and converting the data picture to be recognized into a YCrCb mode, wherein Cr and Cb values meet the following conditions: the skin color area with Cr more than or equal to 133 and less than or equal to 173 and Cb more than or equal to 77 and less than or equal to 127 is a part similar to the color of the palm, a black-white picture is generated, the black-white picture is used for obtaining the maximum outline and generating an outline picture and obtaining an approximate ellipse, and the outline picture is the palm area;
and simultaneously rotating the original picture, the black-and-white picture and the outline picture according to the ellipse angle so as to vertically place the palm, obtaining the palm center based on distance transformation according to the black-and-white picture by using the black-and-white picture as a mask according to the original picture and the black-and-white picture, and drawing an inscribed circle of the palm according to the maximum radius to obtain a palm center region.
3. The method of claim 1, wherein hand acupoint recognition is trained and tested
When the model is used, firstly, preprocessing the input palm area data, wherein the preprocessing comprises the following steps: and carrying out normalization processing, random rotation and scaling to carry out data enhancement.
4. The hand acupoint recognition method of claim 1, wherein onehand10k open source data is used
Training a hand acupuncture point recognition model.
5. The hand acupoint recognition method of claim 1, wherein a DeepPose regression algorithm is adopted for convergence when training the hand acupoint recognition model, the accuracy of the model is obtained through the proportion pck of the correct key point estimation, and the loss function is a wingloss loss function so as to optimize the model.
6. The hand acupoint recognition method of claim 1, wherein the hand acupoint recognition model is output
Regression and thermodynamic diagrams were used.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211329098.6A CN115634147A (en) | 2022-10-27 | 2022-10-27 | Hand acupuncture point identification method based on lightweight deep learning network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211329098.6A CN115634147A (en) | 2022-10-27 | 2022-10-27 | Hand acupuncture point identification method based on lightweight deep learning network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115634147A true CN115634147A (en) | 2023-01-24 |
Family
ID=84947482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211329098.6A Pending CN115634147A (en) | 2022-10-27 | 2022-10-27 | Hand acupuncture point identification method based on lightweight deep learning network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115634147A (en) |
-
2022
- 2022-10-27 CN CN202211329098.6A patent/CN115634147A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ploumpis et al. | Towards a complete 3D morphable model of the human head | |
| CN110464633A (en) | Acupuncture point recognition methods, device, equipment and storage medium | |
| CN105118023B (en) | Real-time video human face cartoon generation method based on human face characteristic point | |
| CN113486758B (en) | Automatic hand acupoint positioning method | |
| de San Roman et al. | Saliency driven object recognition in egocentric videos with deep CNN: toward application in assistance to neuroprostheses | |
| WO2022022551A1 (en) | Method and device for analyzing video for evaluating movement disorder having privacy protection function | |
| CN111127444A (en) | Method for automatically identifying radiotherapy organs at risk in CT image based on depth semantic network | |
| US20090042661A1 (en) | Rule based body mechanics calculation | |
| CN110930374A (en) | Acupoint positioning method based on double-depth camera | |
| CN113537057B (en) | Facial acupuncture point automatic positioning detection system and method based on improved cycleGAN | |
| CN111933275A (en) | Depression evaluation system based on eye movement and facial expression | |
| CN106780653A (en) | The generation method of collaterals of human and acupuncture points on the human body Visual Graph | |
| CN110189324A (en) | A kind of medical image processing method and processing unit | |
| Lanz et al. | Automated classification of therapeutic face exercises using the Kinect | |
| CN117636446B (en) | Face acupoint positioning method, acupuncture robot and storage medium | |
| Wang et al. | Hand vein recognition based on improved template matching | |
| CN114092634A (en) | Channel and acupoint positioning method and device, electronic equipment and storage medium | |
| CN111312363B (en) | Double-hand coordination enhancement system based on virtual reality | |
| CN115634147A (en) | Hand acupuncture point identification method based on lightweight deep learning network | |
| CN113780250B (en) | End-to-end facial acupoint positioning method for small sample and electronic equipment | |
| CN115601339A (en) | Tongue picture inquiry moxibustion scheme generation system and method based on artificial intelligence | |
| KR20230141175A (en) | Location of the acupuncture point recognition system based on machine learning and the method thereof | |
| Lee et al. | Design of A Hand Back Acupoint Massage Aid | |
| Sun et al. | Hand acupoint detection from images based on improved hrnet | |
| Kuang et al. | A survey of multimedia technologies and robust algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |