CN114712220B - Acupuncture point detection method and device and electronic equipment - Google Patents
Acupuncture point detection method and device and electronic equipment Download PDFInfo
- Publication number
- CN114712220B CN114712220B CN202210062379.3A CN202210062379A CN114712220B CN 114712220 B CN114712220 B CN 114712220B CN 202210062379 A CN202210062379 A CN 202210062379A CN 114712220 B CN114712220 B CN 114712220B
- Authority
- CN
- China
- Prior art keywords
- detected
- acupoint
- thermodynamic diagram
- target image
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001467 acupuncture Methods 0.000 title claims abstract description 127
- 238000001514 detection method Methods 0.000 title claims abstract description 50
- 238000010586 diagram Methods 0.000 claims abstract description 129
- 238000002372 labelling Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 12
- 238000010276 construction Methods 0.000 claims description 2
- 238000000034 method Methods 0.000 abstract description 28
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000007689 inspection Methods 0.000 abstract 1
- 238000004364 calculation method Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H39/00—Devices for locating or stimulating specific reflex points of the body for physical therapy, e.g. acupuncture
- A61H39/02—Devices for locating such points
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0064—Body surface scanning
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
- A61B5/015—By temperature mapping of body part
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
- A61B5/7425—Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
- A61B5/743—Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
- A61B5/7445—Display arrangements, e.g. multiple display units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Abstract
The invention provides an acupoint detection method, an acupoint detection device and electronic equipment; relates to the technical field of artificial intelligence; can automated inspection go out the acupuncture point, reduce the manpower time cost. The method comprises the steps of collecting a target image containing a part to be detected; predicting a thermodynamic diagram of the part to be detected based on the target image, wherein pixel values at acupuncture points are different from pixel values at non-acupuncture points in the thermodynamic diagram; and determining the acupoint to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupoint to be detected in the target image.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an acupuncture point detection method and device and electronic equipment.
Background
The acupuncture point therapy is carried out by doctors with quality for acupuncture point diagnosis and treatment, the accuracy of the acupuncture point positioning of different doctors has individual difference, and the treatment effect can be influenced by inaccurate acupuncture point positioning. Moreover, there are hundreds of points in the body, so it is a big task for the physician to find the points.
Disclosure of Invention
The invention provides an acupoint detection method, an acupoint detection device and electronic equipment, which are used for solving the problem of large workload of doctors caused by mainly depending on manual acupoint finding in the prior art.
In a first aspect, the present invention provides a method for detecting acupuncture points, including:
collecting a target image containing a part to be detected;
predicting a thermodynamic diagram of the part to be detected based on the target image, wherein pixel values at acupuncture points are different from pixel values at non-acupuncture points in the thermodynamic diagram;
and determining the acupoint to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and labeling the acupoint to be detected in the target image.
According to an exemplary embodiment of the present invention, the acquiring a target image including a part to be detected includes:
shooting the part to be detected through a camera to obtain an original image shot by the camera;
detecting contour points of a part to be detected in the original image;
and extracting the target image of the part to be detected from the original image according to the contour points.
According to an exemplary embodiment of the present invention, predicting a thermodynamic diagram of the part to be detected based on the target image includes:
inputting the target image into a trained acupoint prediction model, and obtaining a plurality of thermodynamic diagrams output by the acupoint prediction model, wherein the number of the thermodynamic diagrams is the same as the number of the acupoints to be detected contained in the part to be detected.
According to an exemplary embodiment provided by the present invention, the determining, according to the pixel value of each pixel point in the thermodynamic diagram, an acupoint to be detected of the part to be detected, and labeling the acupoint to be detected in the target image includes:
acquiring a target pixel point with the maximum pixel value in each thermodynamic diagram, and taking the target pixel point as an acupoint to be detected;
and marking the position corresponding to the target pixel point in the target image corresponding to the thermodynamic diagram.
According to an exemplary embodiment of the present invention, the method further includes:
acquiring a plurality of first images containing the parts to be detected and the labeling information of the first images, wherein the labeling information is used for indicating the positions of the acupuncture points to be detected in the first images;
constructing Gaussian distribution with the acupuncture points to be detected as the center, and generating a truth value thermodynamic diagram corresponding to the coordinates of each acupuncture point according to the Gaussian distribution of each acupuncture point to be detected, wherein the pixel value of each pixel point in the truth value thermodynamic diagram is the probability value of the Gaussian distribution;
inputting a plurality of first images into the acupoint prediction model to train the acupoint prediction model until the loss between the predicted thermodynamic diagrams output by the acupoint prediction model aiming at the first images and the truth thermodynamic diagrams corresponding to the first images meets the preset requirement, and finishing training.
According to an exemplary embodiment of the present invention, the site to be detected includes a hand.
In a second aspect, the present invention further provides an acupuncture point detecting device, including:
the image input module is used for acquiring a target image containing a part to be detected;
the image prediction module is used for predicting a thermodynamic diagram of the part to be detected based on the target image, wherein the pixel values of acupuncture points in the thermodynamic diagram are different from the pixel values of non-acupuncture points;
and the acupoint output module is used for determining the to-be-detected acupoint of the to-be-detected part according to the pixel value of each pixel point in the thermodynamic diagram and marking the to-be-detected acupoint in the target image.
In a third aspect, the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above acupuncture point detection methods.
In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the acupoint detection method as described in any of the above.
In a fifth aspect, the present invention also provides a computer program product comprising a computer program, which when executed by a processor, implements the method for detecting acupuncture points as described in any one of the above.
According to the acupoint detection method, the acupoint detection device and the electronic equipment, thermodynamic diagrams can be obtained through prediction according to target images of parts to be detected, pixel values of the acupoints in the thermodynamic diagrams are different from those of non-acupoints, and therefore the acupoints can be determined according to the thermodynamic diagrams obtained through prediction. The acupuncture points are marked in the target image, so that reference of the acupuncture points can be provided for a user (such as a doctor), the acupuncture points can be quickly positioned instead of manpower, and the workload of the manpower is reduced. In addition, the acupuncture points are automatically detected according to the collected target images, manual participation is not needed, the problem of inaccurate acupuncture point positioning caused by individual difference can be avoided, and therefore the accuracy of acupuncture point positioning is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of the acupoint detection method provided by the present invention;
FIG. 2 is a schematic diagram of a system architecture of the acupoint detection method provided by the present invention;
FIG. 3 is a second schematic flow chart of the acupoint detection method provided by the present invention;
fig. 4 is a third schematic flow chart of the acupoint detection method provided by the present invention;
FIG. 5 is a schematic diagram of acupoint labeling in the acupoint detection method provided by the present invention;
fig. 6 is a schematic view of an application scenario of the acupoint detection method provided in the present invention;
fig. 7 is a schematic structural diagram of the acupoint detecting device provided by the invention;
fig. 8 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
At present, acupuncture point determination mainly depends on doctors, and few automatic acupuncture point detection methods are used, namely, facial acupuncture points are determined based on facial key points. This approach requires manually designing some key points of the face, recognizing these key points through the image, and then calculating the positions of the acupuncture points of the face based on the identified key points. The method needs the characteristic of manual design, and the accuracy of detecting the acupuncture points is low due to the large difference of human bodies. Another way is to perform point location based on the specificity of the electrical impedance of the points. However, the method needs to depend on specific equipment, so that the method is poor in universality and is not suitable for application and popularization.
Based on this, the present invention first provides an acupoint detecting method. The method can be executed by electronic equipment such as a mobile phone, a personal computer, a tablet computer or a server, and the expression form of the electronic equipment is not particularly limited by the invention.
The technical scheme of the acupoint detection method of the invention is described below with reference to fig. 1-6.
As shown in fig. 1, the acupoint detection method may include the following steps:
step S10: a target image including a portion to be detected is acquired.
The part needing acupuncture point detection can be photographed through a camera, and the photographed picture is a target image of the part to be detected. Illustratively, the site to be detected may be any site of the human body, such as the head, neck, limbs, hands, etc. The acupuncture points of the hand are most concentrated and related to organs of the whole body, and the hand does not involve privacy, is convenient to operate and is one of the most common parts for acupuncture and moxibustion. In the following embodiments, the site to be detected is exemplified by a hand.
Fig. 2 is a system architecture diagram illustrating the acupoint detection method of the present invention. As shown in fig. 2, the system architecture includes a camera 210, a workbench 220, and a computer 230. The working table refers to a table top, such as a table top, a bed, etc., on which a user performs acupuncture point treatment. The computer 230 may be an electronic device that performs the acupoint detection method of the present invention. The user can place a hand on the workbench 220, and then the computer 230 controls the control camera 210 to take a picture of the hand on the workbench 220, so as to obtain a target image containing the hand of the user.
In an exemplary embodiment, the original image collected by the camera may be segmented, and only the minimum bounding rectangle of the part to be detected is extracted as the target image. Therefore, the size of the image acquired by the camera can be reduced, the calculation rate when the target image is processed is improved, and the calculation resources are saved; and reduce the memory usage. Specifically, a part to be detected is photographed by a camera to obtain an original image photographed by the camera. Detecting contour points of the part to be detected in the original image; and then extracting a target image of the part to be detected from the original image according to the contour points of the part to be detected.
The contour points in the original image can be detected by a contour point detection algorithm, for example, all contour points in the original image are detected by a suzuki contour tracing algorithm. And then calculating the minimum enclosing rectangle of all contour points of the part to be detected, and taking the area in the minimum enclosing rectangle as a target image.
In an exemplary embodiment, the process of obtaining the target image is illustrated in FIG. 3. In step S31, the original image is converted to YCrCb color space. The converted image is denoted as an image T1. The original image may be an image of a user's hand. The color space of the original image may be RGB, which is converted from RGB to YCrCb. It is very effective to perform skin color detection in the YCrCb color space. In step S32, a Cr channel is extracted. Namely, the Cr channel of the image T1 obtained in the previous step is extracted to obtain an extracted image T2. After the original image is converted into a YCrCb color space, the skin color has obvious characteristics in a Cr channel, so that the Cr channel is extracted, and an extracted image T2 is obtained. Next, in step S33, OTSU threshold division is performed. In the step, OTSU threshold segmentation is carried out on the image T2 obtained after the last step of processing, and a binary image is obtained after the processing. OTSU is an automatic thresholding method that uses the maximum between-class variance. By the method, a threshold value of the pixel value of the image T2 can be determined, so that the pixel value of the image T2 is divided into 0 or 255 according to the threshold value to obtain a binary image. In step S34, hand contour point extraction is performed. Namely, the contour point extraction is carried out on the binary image obtained by the previous step processing. The contour points of the hand can be obtained through a suzuki contour tracing algorithm. In step S35, a hand minimum bounding rectangle is calculated. In step S36, the image within the minimum bounding rectangle is cut out as the target image.
In the embodiment, the hand region can be clearer and more obvious through color space conversion, and the interference of the color difference of the image can be reduced through image binarization, so that the accuracy of contour point detection is favorably improved. The minimum hand surrounding rectangular frame is cut out from the image after the hand contour points are detected, so that the size of the image can be reduced, resources required by image processing are reduced, and the processing speed is increased.
Step S20: and predicting a thermodynamic diagram of the part to be detected on the basis of the target image, wherein pixel values at acupuncture points are different from pixel values at non-acupuncture points in the thermodynamic diagram.
The thermodynamic diagram refers to a diagram in which acupuncture points and non-acupuncture points are displayed in a differentiated manner. That is, the pixel values at the acupuncture points in the thermodynamic diagram are different from the pixel values at the non-acupuncture points. Illustratively, the thermodynamic diagram may display acupuncture points in a particularly highlighted form. That is, the pixel values at the acupuncture points in the thermodynamic diagram may be greater than the pixel values at the non-acupuncture points.
A model for predicting the thermodynamic diagram of the image can be trained through a machine learning algorithm, and the model is an acupuncture point prediction model. Specifically, the acupoint prediction model needs to be trained first, and the method for obtaining the acupoint prediction model through training is shown in fig. 4.
In step S41, a plurality of first images including the to-be-detected portion and annotation information of the first images are acquired.
Multiple pictures can be taken as the first image from different angles for the hands of different users in advance. And then adding label information to each first image, wherein the label information is used for indicating the positions of the acupuncture points to be detected of the hand in the first image. For example, a professional such as an acupuncturist can determine the acupuncture point of the hand in each first image, and add the annotation information to the acupuncture point. The labeling information may be a special color, graphic, etc., for example, labeling with "dots" at acupuncture points in the first image of the hand, etc. As shown in fig. 5, each acupoint on the hand may be marked with a "dot".
In addition, the number of the first images may be determined according to actual requirements, for example, 1000, 10000, etc., and this embodiment is not particularly limited thereto.
In step S42, gaussian distribution with the acupuncture points to be detected as a center is constructed, and a truth thermodynamic diagram corresponding to each acupuncture point coordinate is generated according to the gaussian distribution of each acupuncture point to be detected, where a pixel value of each pixel in the truth thermodynamic diagram is a probability value of the gaussian distribution.
The thermodynamic diagram is the same size as its corresponding image. That is, the truth thermodynamic diagram corresponding to the first image is an image with the same size as the first image, that is, the first image and the corresponding truth thermodynamic diagram have the same number of pixels. For example, if the first image is 30 × 30, the corresponding thermodynamic diagram is also 30 × 30.
The pixel point coordinates are used as variables, and a Gaussian distribution is set by taking the to-be-detected cave point as a center, so that the pixel values of the to-be-detected cave point and the pixel points within a certain radius near the to-be-detected cave point are in Gaussian distribution. The probability value of the Gaussian distribution is a pixel value at the pixel point coordinate, so that a true thermodynamic diagram is generated according to the pixel point coordinate and the pixel value at the pixel point coordinate.
Illustratively, the pixel values in a truth thermodynamic diagram obey a two-dimensional Gaussian distribution centered on the acupoint to be examined
Wherein, X P ，Y P Is the coordinate of the acupoint to be detected>
Three parameters rho can be manually set according to actual requirements and are combined with each other>
Which represents the variance of the gaussian distribution in the x, y directions, respectively, ρ represents the correlation coefficient for the x and y axes, and ρ may be set to 0. A variable, namely a probability value corresponding to a pixel point coordinate can be calculated according to an expression of two-dimensional Gaussian distribution, and a true value thermodynamic diagram can be obtained by taking the probability value as a pixel value at the pixel point coordinate.
It can be understood that if the acupuncture points to be detected on the hand include a plurality of acupuncture points, each acupuncture point to be detected is taken as a center, and a thermodynamic diagram corresponding to each acupuncture point to be detected can be obtained, so that the truth value thermodynamic diagrams with the same number as the acupuncture points to be detected can be generated according to the first image.
In step S43, inputting the plurality of first images into the acupoint prediction model to train the acupoint prediction model, and completing training until a loss between prediction data output by the acupoint prediction model for the first images and a truth thermodynamic diagram corresponding to the first images meets a preset requirement.
For example, the acupoint prediction model may be a High-Resolution Network (HRNet) model. The HRNet model comprises a plurality of prediction channels, the number of the prediction channels can be set to be the same as that of acupuncture points needing to be detected, and each prediction channel is responsible for predicting one acupuncture point. All acupuncture points to be detected may be numbered in advance, e.g., 1, 2, 3, etc., and then a plurality of truth thermodynamic diagrams of the first image may be determined in the order of the acupuncture point numbers. After the first image is input into the HRNet model, a group of thermodynamic diagrams, namely predicted thermodynamic diagrams, can be output through prediction of a plurality of prediction channels. The predicted thermodynamic diagrams output by the HRNet model correspond to the sequence of the acupuncture point numbers. The error between the predictive thermodynamic diagram output by each predictive channel and the corresponding true thermodynamic diagram is first calculated, for example, the error between the first predictive thermodynamic diagram and the first true thermodynamic diagram of the first image is calculated. The sum of the errors of all predicted channels is then calculated as the penalty. The error between the predictive thermodynamic diagram and the corresponding true thermodynamic diagram can be calculated by the difference between the pixel values of each pixel of the predictive thermodynamic diagram and the true thermodynamic diagram. For example, the pixel value A1 of the first pixel point in the predictive thermodynamic diagram is subtracted from the pixel value A2 of the first pixel point in the true thermodynamic diagram, the difference of the pixel values corresponding to each pixel point is calculated, and then the sum of each pixel point is obtained, and the obtained result is used as the loss. And when the sum of the losses is smaller than a preset value, determining that the loss of the HRNet model meets a preset requirement, and finishing training.
In an exemplary embodiment, the loss may also be calculated by other means. Specifically, for a set of predictive thermodynamic diagrams of the first image, the mean square error is calculated pixel by pixel and summed between the predictive thermodynamic diagram and the true thermodynamic diagram for each prediction channel, as shown in equation (1) below:
wherein e is i Representing the mean square error of the ith prediction channel; z represents the set of all pixel positions in the image of the ith prediction channel; the predicted thermodynamic diagram of the HRNet model for the first image prediction is h; h (z) represents the pixel value at z; the truth thermodynamic diagram for the first image is h.
And after the mean square error of each prediction channel is obtained through calculation, normalizing the size of a group of errors obtained through calculation to be 0-1 by utilizing a Softmax function. And taking the normalized value as the weight of the prediction channel, thereby obtaining the weight of each prediction channel. Then, the mean square error of each predicted channel is multiplied by the weight of the predicted channel, and all predicted channels are summed to obtain the final loss. Expressed as the following equation (2):
wherein, L represents a loss; i represents the ith prediction channel; n is the total number of the predicted channels; w is a i The weight of the ith predicted channel.
Through weighting the loss of each prediction channel, the weight of the point corresponding to the more difficult detection points can be increased, so that the model training is more focused on the acupuncture points with larger training loss, and the accuracy of the model is improved.
Furthermore, the manner of calculating the loss may include a variety of ways, such as: the loss can be reduced by
The calculation, i.e., L calculated above, can be further divided by the total number of channels n to obtain the average loss per channel. Or by>
To calculate losses, etc.
After obtaining the loss, the parameters of the HRNet model can be adjusted by using an error back propagation method with the minimum loss as an optimization target until the optimization target is reached. In addition, the preset condition to be satisfied by the loss may also include other conditions, such as the loss being less than a preset value, or the rate of change of the loss being less than a set threshold, and so on. And iterating for multiple times until the loss of the HRNet model meets the preset condition, wherein the accuracy of the trained HRNet model can meet the requirement, so that the accuracy of acupoint detection is ensured. In addition, the HRNet model does not depend on the characteristics set artificially, can directly predict thermodynamic diagrams corresponding to acupuncture points, and has good robustness.
After the training of the HRNet model is completed, the trained HRNet model can be stored. When acupuncture point detection is needed, the trained HRNet model can be used for acupuncture point detection. For a target image needing to be detected, inputting the target image into a trained HRNet model (namely an acupuncture point prediction model), and obtaining a plurality of thermodynamic diagrams output by the acupuncture point prediction model. The number of the thermodynamic diagrams is the same as the number of the acupuncture points to be detected contained in the part to be detected. Each thermodynamic diagram can be used to determine an acupoint to be tested.
Step S30: and determining the acupoint to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupoint to be detected in the target image.
The pixel value of the acupuncture point in the thermodynamic diagram is different from the pixel value of the non-acupuncture point, the closer the acupuncture point is to the acupuncture point, the larger the pixel value is, and the pixel value of the acupuncture point is the largest. And acquiring a target pixel point with the maximum pixel value in each thermodynamic diagram, and taking the target pixel point as an acupoint to be detected. And marking the position corresponding to the target pixel point in the target image corresponding to the thermodynamic diagram. In the embodiment, the acupuncture points of the part to be detected, such as the hand, are predicted according to the thermodynamic diagram, and the method is independent of special equipment, high in usability and more beneficial to popularization.
Specifically, a plurality of thermodynamic diagrams can be predicted for the target image. For example, if 10 acupuncture points to be detected on the hand are detected, the thermodynamic diagram corresponding to the target image is 10. And traversing the pixel value of each pixel point in each thermodynamic diagram, and taking the pixel point with the maximum pixel value in each thermodynamic diagram as a target pixel point to obtain the target pixel points with the same number as that of the thermodynamic diagrams, namely 10 target pixel points. And then, labeling the same coordinate position in the target image according to the coordinate of the target pixel point. For example, if the coordinates of the target pixel point in the thermodynamic diagram a are (10, 10), the pixel point at (10, 10) in the target image is labeled. The effect of automatically marking out the acupuncture points can be realized after the acupuncture points in the image are predicted through the model, and labor and time can be saved. For example, the position of the target pixel in the target image may be labeled by a specific color, a specific graph, and the like, for example, a "dot" is labeled at the position of the target pixel in the target image, and the position of the target pixel is labeled as a red highlight, and the like, which is not limited in this embodiment.
Target pixel points are marked in the target image, and then the marked target image can be displayed, so that the user is prompted that the marked positions in the target image are positions of the acupuncture points to be detected, and the user is assisted in positioning the hand acupuncture points. Compared with manual positioning of acupuncture points, the automatic acupuncture point positioning method can automatically position acupuncture points, reduce manual workload and reduce labor time cost.
Fig. 6 shows an application scenario of the acupoint detection method of the present invention. As shown in fig. 6, the target image 601 may be input into an acupoint prediction model 602, and the acupoint prediction model 602 may predict acupoints in the target image and output a plurality of predicted thermodynamic diagrams. Such as thermodynamic diagram 603. And determining the position of a target pixel point according to each thermodynamic diagram so as to obtain the positions of a plurality of target pixel points. The position of each target pixel point is labeled in the target image 601, so that a labeled image 604 can be obtained, and the labeled image 604 can be displayed as an acupuncture point detection result, so that a user can directly see the positions of all acupuncture points on the hand, and the time for the user to search the acupuncture points one by one is saved.
The invention also provides an acupuncture point detection device which can be used for executing the acupuncture point detection method. The acupuncture point detecting device provided by the present invention is described below.
As shown in fig. 7, the acupoint detecting device 70 may include an image input module 71, an image prediction module 72, and an acupoint output module 73.
Specifically, the image input module 71 is configured to acquire a target image including a to-be-detected portion. The image prediction module 72 is configured to predict a thermodynamic diagram of the to-be-detected part, where pixel values at acupuncture points are different from pixel values at non-acupuncture points, based on the target image. The acupoint output model 73 is configured to determine an acupoint to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and label the acupoint to be detected in the target image.
In an exemplary embodiment of the present invention, the image input module 71 specifically includes a photographing module, a contour detection module, and an image extraction module. The photographing module is used for photographing the part to be detected through a camera to obtain an original image photographed by the camera; the contour detection module is used for detecting contour points of the part to be detected in the original image; and the image extraction module is used for extracting the target image of the part to be detected from the original image according to the contour points.
In an exemplary embodiment of the present invention, the image prediction module 72 may be configured to input the target image into a trained acupoint prediction model, and obtain a plurality of thermodynamic diagrams output by the acupoint prediction model, where the number of the thermodynamic diagrams is the same as the number of the acupoints to be detected contained in the part to be detected.
In an exemplary embodiment of the present invention, the acupoint output module 73 specifically includes a pixel value determining module and an acupoint labeling module. The pixel value determining module is used for acquiring a target pixel point with the maximum pixel value in each thermodynamic diagram and taking the target pixel point as an acupoint to be detected. And the acupoint labeling module is used for labeling the position corresponding to the target pixel point in the target image corresponding to the thermodynamic diagram.
In an exemplary embodiment of the present invention, the acupoint detecting device 70 further includes an image labeling module, a thermodynamic diagram building module, and a model training module. Specifically, the image labeling module is used for acquiring a plurality of first images containing the parts to be detected and labeling information of the first images, and the labeling information is used for indicating the positions of the acupuncture points to be detected in the first images. The thermodynamic diagram construction module is used for constructing Gaussian distribution with the acupuncture points to be detected as the center, generating a truth thermodynamic diagram corresponding to the coordinates of each acupuncture point according to the Gaussian distribution of each acupuncture point to be detected, and the pixel value of each pixel point in the truth thermodynamic diagram is the probability value of the Gaussian distribution. The model training module is used for inputting the first images into the acupoint prediction model so as to train the acupoint prediction model until the loss between the predicted thermodynamic diagram output by the acupoint prediction model aiming at the first images and the truth thermodynamic diagram corresponding to the first images meets a preset requirement, and then the training is completed.
In an exemplary embodiment of the invention, the site to be detected comprises a hand.
As the functional modules of the acupoint detection device in the exemplary embodiment of the present invention correspond to the steps of the exemplary embodiment of the method of the acupoint detection device, please refer to the above embodiment of the method of the acupoint detection device for details that are not disclosed in the embodiments of the device of the present invention.
Fig. 8 illustrates a physical structure diagram of an electronic device, and as shown in fig. 8, the electronic device may include: a processor (processor) 810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may invoke logic instructions in the memory 830 to perform the above-described acupoint detection device method, the method comprising: s10, collecting a target image containing a part to be detected; step S20, predicting a thermodynamic diagram of the part to be detected based on the target image, wherein pixel values of acupuncture points in the thermodynamic diagram are different from pixel values of non-acupuncture points; and S30, determining the acupuncture point to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupuncture point to be detected in the target image.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, when the computer program is executed by a processor, a computer can execute the acupoint detection device method provided by the above methods, the method includes: s10, collecting a target image containing a part to be detected; step S20, predicting a thermodynamic diagram of the part to be detected based on the target image, wherein the pixel values of acupuncture points in the thermodynamic diagram are different from the pixel values of non-acupuncture points; and S30, determining the acupuncture point to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupuncture point to be detected in the target image.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for an acupuncture point detection device provided by the above methods, the method including: s10, collecting a target image containing a part to be detected; step S20, predicting a thermodynamic diagram of the part to be detected based on the target image, wherein the pixel values of acupuncture points in the thermodynamic diagram are different from the pixel values of non-acupuncture points; and S30, determining the acupuncture point to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupuncture point to be detected in the target image.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (7)
1. An acupuncture point detection method, comprising:
collecting a target image containing a part to be detected;
predicting a thermodynamic diagram of the part to be detected based on the target image, wherein pixel values at acupuncture points are different from pixel values at non-acupuncture points in the thermodynamic diagram;
determining the acupuncture point to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and marking the acupuncture point to be detected in the target image;
predicting a thermodynamic diagram of the to-be-detected part based on the target image, comprising:
inputting the target image into a trained acupoint prediction model, and obtaining a plurality of thermodynamic diagrams output by the acupoint prediction model, wherein the number of the thermodynamic diagrams is the same as that of the acupoints to be detected contained in the part to be detected;
the acupoint detection method further comprises the following steps:
acquiring a plurality of first images containing the parts to be detected and the labeling information of the first images, wherein the labeling information is used for indicating the positions of the acupuncture points to be detected in the first images;
constructing Gaussian distribution taking the acupuncture points to be detected as the center, and generating a truth value thermodynamic diagram corresponding to each acupuncture point coordinate according to the Gaussian distribution of each acupuncture point to be detected, wherein the pixel value of each pixel point in the truth value thermodynamic diagram is the probability value of the Gaussian distribution;
inputting a plurality of first images into the acupoint prediction model to train the acupoint prediction model until the loss between the predicted thermodynamic diagram output by the acupoint prediction model aiming at the first images and a truth thermodynamic diagram corresponding to the first images meets a preset requirement, and finishing training; the loss is:
wherein L represents loss, i represents the ith prediction channel, n is the total number of prediction channels, and w i Weight for the ith predicted channel, e i Representing the mean square error of the ith prediction channel.
2. The acupoint detection method according to claim 1, wherein the acquiring of the target image including the part to be detected comprises:
shooting the part to be detected through a camera to obtain an original image shot by the camera;
detecting contour points of a part to be detected in the original image;
and extracting the target image of the part to be detected from the original image according to the contour points.
3. The acupoint detection method according to claim 1, wherein the determining the acupoint to be detected of the part to be detected according to the pixel value of each pixel point in the thermodynamic diagram, and labeling the acupoint to be detected in the target image comprises:
acquiring a target pixel point with the maximum pixel value in each thermodynamic diagram, and taking the target pixel point as an acupoint to be detected;
and marking the position corresponding to the target pixel point in the target image corresponding to the thermodynamic diagram.
4. The acupoint detection method of any one of claims 1 to 3, wherein the site to be detected comprises a hand.
5. An acupuncture point detecting device, comprising:
the image input module is used for acquiring a target image containing a part to be detected;
the image prediction module is used for predicting a thermodynamic diagram of the part to be detected based on the target image, wherein the pixel values of acupuncture points in the thermodynamic diagram are different from the pixel values of non-acupuncture points;
the acupoint output module is used for determining the to-be-detected acupoint of the to-be-detected part according to the pixel value of each pixel point in the thermodynamic diagram and marking the to-be-detected acupoint in the target image;
the image prediction module is further configured to input the target image into a trained acupoint prediction model to obtain a plurality of thermodynamic diagrams output by the acupoint prediction model, wherein the number of the thermodynamic diagrams is the same as the number of the acupoints to be detected contained in the part to be detected;
the acupuncture point detection device further comprises:
the image labeling module is used for acquiring a plurality of first images containing the parts to be detected and labeling information of the first images, and the labeling information is used for indicating the positions of the acupuncture points to be detected in the first images;
the thermodynamic diagram construction module is used for constructing Gaussian distribution with the acupuncture points to be detected as the center, generating a truth thermodynamic diagram corresponding to each acupuncture point coordinate according to the Gaussian distribution of each acupuncture point to be detected, and the pixel value of each pixel point in the truth thermodynamic diagram is the probability value of the Gaussian distribution;
the model training module is used for inputting a plurality of first images into the acupoint prediction model so as to train the acupoint prediction model until the loss between a predicted thermodynamic diagram output by the acupoint prediction model aiming at the first images and a truth thermodynamic diagram corresponding to the first images meets a preset requirement;
the loss is:
wherein L represents loss, i represents the ith prediction channel, n is the total number of prediction channels, and w i Weight for the ith predicted channel, e i Representing the mean square error of the ith prediction channel.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the acupoint detection method of any one of claims 1 to 4 when executing the program.
7. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the acupoint detection method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062379.3A CN114712220B (en) | 2022-01-19 | 2022-01-19 | Acupuncture point detection method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210062379.3A CN114712220B (en) | 2022-01-19 | 2022-01-19 | Acupuncture point detection method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114712220A CN114712220A (en) | 2022-07-08 |
| CN114712220B true CN114712220B (en) | 2023-03-28 |
Family
ID=82235068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210062379.3A Active CN114712220B (en) | 2022-01-19 | 2022-01-19 | Acupuncture point detection method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114712220B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110464633A (en) * | 2019-06-17 | 2019-11-19 | 深圳壹账通智能科技有限公司 | Acupuncture point recognition methods, device, equipment and storage medium |
| CN113486758A (en) * | 2021-06-30 | 2021-10-08 | 浙江大学 | Automatic hand acupuncture point positioning method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202288798U (en) * | 2011-06-08 | 2012-07-04 | 美国亚德诺半导体公司 | Meridian measurement system |
| TWI495454B (en) * | 2013-06-11 | 2015-08-11 | Univ Nat Taiwan Science Tech | Device for detecting surface microcirculation of acupoint |
| CN110327202A (en) * | 2019-05-05 | 2019-10-15 | 王立群 | The method and system at acupuncture point is found in a kind of Traditional Chinese medicine massage robot thermal imaging |
| CN111444896B (en) * | 2020-05-09 | 2023-06-30 | 北京碧拓科技有限公司 | Far infrared thermal imaging human body meridian key point positioning method |
| CN112691029A (en) * | 2020-12-25 | 2021-04-23 | 深圳市元征科技股份有限公司 | Meridian data processing method, device, equipment and storage medium |
| CN113807414A (en) * | 2021-08-30 | 2021-12-17 | 中科尚易健康科技(北京)有限公司 | Human body meridian automatic labeling method and device based on deep learning, equipment and storage medium |
| CN113842116B (en) * | 2021-10-14 | 2022-09-27 | 北京鹰之眼智能健康科技有限公司 | Automatic positioning method and device for human acupuncture points and electronic equipment |
-
2022
- 2022-01-19 CN CN202210062379.3A patent/CN114712220B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110464633A (en) * | 2019-06-17 | 2019-11-19 | 深圳壹账通智能科技有限公司 | Acupuncture point recognition methods, device, equipment and storage medium |
| CN113486758A (en) * | 2021-06-30 | 2021-10-08 | 浙江大学 | Automatic hand acupuncture point positioning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114712220A (en) | 2022-07-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11487995B2 (en) | Method and apparatus for determining image quality | |
| US20210192758A1 (en) | Image processing method and apparatus, electronic device, and computer readable storage medium | |
| US11915514B2 (en) | Method and apparatus for detecting facial key points, computer device, and storage medium | |
| CN107679466B (en) | Information output method and device | |
| CN111310705A (en) | Image recognition method and device, computer equipment and storage medium | |
| WO2016089529A1 (en) | Technologies for learning body part geometry for use in biometric authentication | |
| CN110930296B (en) | Image processing method, device, equipment and storage medium | |
| KR20210028185A (en) | Human posture analysis system and method | |
| CN108229301B (en) | Eyelid line detection method and device and electronic equipment | |
| EP4181059A1 (en) | Medical image processing method, apparatus, device, storage medium, and product | |
| CN111242109B (en) | Method and device for manually fetching words | |
| CN111126254A (en) | Image recognition method, device, equipment and storage medium | |
| CN111401318A (en) | Action recognition method and device | |
| CN113298158A (en) | Data detection method, device, equipment and storage medium | |
| CN112287765A (en) | Face living body detection method, device and equipment and readable storage medium | |
| CN111079676A (en) | Human eye iris detection method and device | |
| CN113591763A (en) | Method and device for classifying and identifying face shape, storage medium and computer equipment | |
| CN111723688B (en) | Human body action recognition result evaluation method and device and electronic equipment | |
| CN114712220B (en) | Acupuncture point detection method and device and electronic equipment | |
| CN112597847A (en) | Face pose estimation method and device, electronic equipment and storage medium | |
| CN112801238B (en) | Image classification method and device, electronic equipment and storage medium | |
| CN116798087A (en) | Employee on-duty state detection method and system | |
| CN114723659A (en) | Acupuncture point detection effect determining method and device and electronic equipment | |
| CN113724237A (en) | Tooth mark recognition method and device, computer equipment and storage medium | |
| CN111783519A (en) | Image processing method, image processing device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |