CN115527235B - Method and device for identifying Mongolian medical hand acupoints based on image processing - Google Patents
Method and device for identifying Mongolian medical hand acupoints based on image processing Download PDFInfo
- Publication number
- CN115527235B CN115527235B CN202211178680.7A CN202211178680A CN115527235B CN 115527235 B CN115527235 B CN 115527235B CN 202211178680 A CN202211178680 A CN 202211178680A CN 115527235 B CN115527235 B CN 115527235B
- Authority
- CN
- China
- Prior art keywords
- hand
- point
- image
- acupoints
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012545 processing Methods 0.000 title claims abstract description 21
- 238000001467 acupuncture Methods 0.000 claims abstract description 21
- 238000010606 normalization Methods 0.000 claims description 14
- 238000012360 testing method Methods 0.000 claims description 6
- 238000012549 training Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 210000004932 little finger Anatomy 0.000 claims description 5
- 210000003811 finger Anatomy 0.000 claims description 4
- 210000000707 wrist Anatomy 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 3
- 210000003813 thumb Anatomy 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims 1
- 230000009193 crawling Effects 0.000 claims 1
- 238000004519 manufacturing process Methods 0.000 claims 1
- 241001133760 Acoelorraphe Species 0.000 description 22
- 210000001503 joint Anatomy 0.000 description 20
- 239000003814 drug Substances 0.000 description 18
- 244000060701 Kaempferia pandurata Species 0.000 description 10
- 235000016390 Uvaria chamae Nutrition 0.000 description 10
- 210000004247 hand Anatomy 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000002560 therapeutic procedure Methods 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000000811 metacarpophalangeal joint Anatomy 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Finger-Pressure Massage (AREA)
Abstract
A method for identifying Mongolian medical hand acupoints based on image processing identifies palm and back of hand, and obtains hand key points of manually marked hand images in a database; acquiring the position relation between the hand key points and the acupuncture points of the hand image of the manual annotation in the database; and positioning the hand key points of the hand image to be identified, and calculating the point positions of the acupuncture points of the hand image to be identified based on the position relation. The invention also provides a device for identifying the Mongolian hand acupuncture points based on image processing, and compared with the prior art, the device can accurately and real-timely detect the positions of the Mongolian hand acupuncture points.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a device for identifying Mongolian medical hand acupoints based on image processing.
Background
At present, most of acupoint identification methods are researched aiming at traditional Chinese medicine, and few Mongolian medicine acupoints are researched. It is well known that the specific positions of Mongolian acupoints play a key role in learning and clinical operation of Mongolian acupuncture therapy, and the accuracy of Mongolian acupuncture point positioning directly affects the curative effect of Mongolian acupuncture therapy. At present, the related data of Mongolian acupoints are less, modern tools for teaching and clinical application of Mongolian acupoint theory are less, the acupoint positioning is very complex, long-time professional training is required to find the positions of the acupoints, and problems often occur to beginners or doctors with low experience. However, experience teaching is still adopted in the aspect of Mongolian acupuncture point positioning at present, so that the teaching difficulty is increased, and the development of Mongolian acupuncture is not facilitated.
The traditional Chinese medicine acupoints are mainly distributed at special point areas on the meridian line of the human body, the main positioning methods include a body surface marking method and an index positioning method, and the traditional Chinese medicine positioning has accurate medical description, but the Mongolian medicine acupoints are distributed randomly, the positions of the Mongolian medicine acupoints are not accurately described, and the positions of the Mongolian medicine acupoints cannot be determined by adopting the positioning method of the traditional Chinese medicine acupoints.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention aims to provide a method and a device for identifying Mongolian hand acupoints based on image processing.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
A method for identifying Mongolian medical hand acupoints based on image processing, comprising:
Step 1, recognizing palm and back of hand, wherein five Mongolian medical hand acupoints to be recognized are arranged on the palm, two of the acupoints are distributed on the back of hand, and the other three acupoints are distributed on the back of hand;
step 2, obtaining hand key points of the manual annotation hand image in the database;
Step 3, acquiring the position relation between the hand key points and the acupuncture points of the hand image of the manual annotation in the database;
and 4, positioning the hand key points of the hand image to be identified, and calculating the point positions of the acupuncture points of the hand image to be identified based on the position relation.
In one embodiment, in the step 1, the recognition model adopts a YOLOv model, the palm image and the back of the hand are crawled from the internet through web crawler software, the palm and the back of the hand are marked, a palm and back of the hand recognition dataset is manufactured, the palm and back of the hand recognition dataset is divided into a training set and a testing set, and the recognition model is trained by using the dataset.
In one embodiment, in the step 2, the database is google open source database MEDIAPIPE, which manually marks 21 key points of the hand in the hand image, the wrist number is 0, the joints are numbered from 1 to 4 in sequence from thumb root to fingertip, the joints are numbered from index finger root to fingertip, the joints are numbered from 5 to 8 in sequence from middle finger root to fingertip, the joints are numbered from 9 to 12 in sequence from ring finger root to fingertip, the joints are numbered from 13 to 16 in sequence from small finger root to fingertip, and the joints are numbered from 17 to 20 in sequence, thereby obtaining 21 key points K 0～K20 of the hand.
In one embodiment, the step 3 is to calculate the position relationship by using an average value method or a polynomial fitting method;
The average value method comprises the following steps:
Step (1), acquiring coordinates (x i (K),yi (K)) of an ith hand key point K i of an artificial annotation hand image and coordinates (x j (A),yj (A)) of a jth acupoint A j in a database, selecting two key points K k、Kl closest to the A j, connecting the K k、Kl, and taking the midpoint of the connecting line as coordinates (x j (C),yj (C)) of a datum point C j,Cj of the A j, wherein K is less than l;
Step (2), the position relation of A j and the reference point C j thereof is calculated, wherein the position relation comprises the distance between A j and C j One included angle/>The included angle/>The included angle between the line segment C jKk and the line segment C jAj;
Step (3): for a pair of And/>Performing normalization, namely taking half of the connecting line of the key point K k、Kl as L j, and performing normalization onCarrying out normalization treatment; taking a key point K 0 as an origin, taking an included angle theta oj between a key point K 0-K5 connecting line and a key point K 0-Kl connecting line, and changing the key point K 0-Kl connecting line into a K 0-K17 connecting line if l=5; pair/>The normalization process is performed, and the distance d 'c,j and the angle θ' j (A) after normalization are expressed by the following formulas:
Obtaining;
step (4): the distances d 'c,j and the angles theta' j (A) of all the manually marked hand images in the database are respectively averaged, and the average distance between A j and the reference point C j thereof is calculated Average angle/>
Wherein n is the total number of manually marked hand images in the database;
Step (5): according to average distance Average angle/>The relation between the distance j and the Angle j between the unknown point A and the reference point in the input hand image is obtained, and the calculation formula is as follows:
The polynomial fitting method finds reference points according to the key points and the point positions of the acupoints, and obtains the distances between the reference points and the point positions in all manual labeling images And angle/>L j、θoj is selected, polynomial fitting is carried out on the distance between the acupoint and the datum point and the angular relation between the acupoint and the datum point as well as the angular relation between the acupoint and the key point, and then the positional relation between the acupoint and the key point is obtained.
In one embodiment, in the step 4, the point coordinate point (x "j (A),y″j (A)) is obtained by the following formula:
Wherein (x j (C),yj (C)) is the reference point coordinate corresponding to the point of the acupoint, length j is the distance between the point of the acupoint and the reference point, and Angle j is the Angle between the point of the acupoint and the reference point.
The invention also provides a device for identifying the hand acupoints of the Mongolian medicine based on image processing, which comprises:
The image acquisition module is used for acquiring hand pictures of the object, wherein the hand pictures comprise palm pictures and back pictures;
The first recognition module is used for recognizing the hand picture as a palm picture or a back hand picture;
the second recognition module is used for recognizing hand key points of the palm picture or the back hand picture;
the positioning module is used for positioning the hole sites of the hand picture according to the position relation between the hand key points and the hole sites;
the marking module is used for marking the acupuncture points on the hand picture according to the positioning of the positioning module;
and the output module is used for outputting the marked hand picture.
In one embodiment, the image acquisition module is a camera and a flat plate with pure background, the flat plate is used for placing the hands of a user, and the camera takes a picture of the hands.
In one embodiment, the first identification module, the second identification module, the first positioning module, the second positioning module and the marking module are integrated in the processor, and the first identification module completes identification based on YOLOv models or completes identification based on definitions when the image acquisition module acquires the images; and the second recognition module completes recognition based on the manually marked image data of the Google open source library MEDIAPIPE.
In one embodiment, the output module is a display device coupled to the processor.
Compared with the prior art, the invention has the beneficial effects that: the prior art has less research on the identification of hand acupoints of Mongolian medicine. The acupoint detection method can detect the acupoint position of the hand of the Mongolian medical person in real time, and detect the acupoint position of the hand through a camera or uploading pictures.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic representation of 21 hand keypoints annotation of manually annotated hand images in a database.
FIG. 3 is a schematic representation of the results of one test of the present invention; the method comprises the steps of (a) manually marking an image of a hand acupoint, (b) obtaining a comparison image of the acupoint point obtained by a method for obtaining the relationship between the acupoint point and the key point by using an average value and the manually marked point, and (c) obtaining a comparison image of the acupoint point obtained by a method for obtaining the relationship between the acupoint point and the key point by using polynomial fitting and the manually marked point.
FIG. 4 is a schematic representation of another test result of the present invention; the method comprises the steps of (a) manually marking an image of a hand acupoint, (b) obtaining a comparison image of the acupoint point obtained by a method for obtaining the relationship between the acupoint point and the key point by using an average value and the manually marked point, and (c) obtaining a comparison image of the acupoint point obtained by a method for obtaining the relationship between the acupoint point and the key point by using polynomial fitting and the manually marked point.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.
In order to solve the problem of difficult positioning of the hand acupoints of the traditional Mongolian medicine, the invention firstly provides a method for identifying the hand acupoints of the Mongolian medicine based on image processing, so that the positions of the acupoints can be quickly and accurately identified through a camera or uploading a hand picture. As shown in fig. 1, it mainly comprises the following steps:
step 1, recognizing the palm and back of the hand.
Five acupoints on the hand are commonly used in Mongolian medicine, and the aim of the invention is to identify the five acupoints. Two acupoints are distributed on the palm, and three acupoints are distributed on the back of hand. Because the hand acupoints of Mongolian medicine are distributed at different positions on the palm and back of the hand, the palm and back of the hand need to be identified before the hand acupoints are positioned. The YOLOv model is adopted as the identification model.
The YOLOv model can be trained as follows: the Hand image is crawled from the Internet through webpage crawler software, the palm and the back of the Hand are marked, a palm back of the Hand recognition data set hand_B_P is produced, the more the crawled images are, the better, but the 1800 images are selected by the invention in consideration of the operation amount. Second, the hand_b_p dataset is partitioned into training and test sets, which may be a conventional 4:1 partition ratio. Finally, the data set is used to train the pattern recognition model.
And 2, acquiring key points and coordinates of the hand.
The key points of the hand refer to hand points with obvious characteristics, which are defined as joint points of the hand in the invention, and the joint points are often obvious and are easy to locate and identify.
The present invention adopts google open source library MEDIAPIPE, in MEDIAPIPE picture data, hand key points have been explicitly marked, as shown in fig. 2, wherein the hand key points are 21 in total, the wrist number is 0, the joints are numbered from thumb root to fingertip in sequence from 1 to 4, the index finger root to fingertip, the joints are numbered from 5 to 8, the middle finger root to fingertip, the joints are numbered from 9 to 12, the ring finger root to fingertip, the joints are numbered from 13 to 16, the small finger root to fingertip, and the joints are numbered from 17 to 20, thereby obtaining 21 hand key points K 0～K20.
The key points finally obtained and the serial numbers thereof are as follows:
Wrist K 0, thumb-carpometacarpal joint KK1, thumb-metacarpophalangeal joint K 2, thumb-interphalangeal joint K 3, thumb-fingertip joint K 4, Index finger-metacarpophalangeal joint K 5, index finger-proximal interphalangeal joint K 6, index finger-distal interphalangeal joint K 7, index finger-fingertip joint K 8, Middle finger-metacarpophalangeal joint K 9, middle finger-proximal interphalangeal joint K 10, middle finger-distal interphalangeal joint K 11, middle finger-fingertip joint K 12, Ring finger-metacarpophalangeal joint K 13, ring finger-proximal interphalangeal joint K 14, ring finger-distal interphalangeal joint K 15, ring finger-fingertip joint K 16, little finger metacarpophalangeal joint K 17, little finger proximal interphalangeal joint K 18, little finger distal interphalangeal joint K 19, little finger fingertip joint K 20.
And step 3, acquiring the point position relation between the key points of the hand and the acupuncture points.
In order to obtain the positional relationship between the hand key points and the points of the Mongolian hand acupoints, a datum point needs to be found from the hand key points, the distance between the points of the acupoints and the datum point is found, and the angular relationship among the points of the acupoints, the datum point and the key points is obtained. Thus, a reference point is defined for each acupoint. In the present invention, the fiducial point is defined as the midpoint of the line connecting the two keypoints nearest to the corresponding hole site.
For convenience of distinction, in the present invention, the points are denoted by the symbols (a), (C) and (K) as reference points.
In the method, when the distance relation and the angle relation between the acupoint and the key point are calculated, an average value method and a polynomial fitting method are adopted respectively.
The specific method of the average method is as follows:
Step (1): find the reference point (C).
Calling an API of a Google open source library MEDIAPIPE, acquiring coordinates (x i (K),yi (K)) of an ith hand key point K i and coordinates (x j (A),yj (A)) of a jth acupoint site A j in a hand image of manual annotation, selecting two key points K k、Kl closest to A j, connecting the K k、Kl, and taking the midpoint of the connecting line as coordinates (x j (C),yj (C)) of a datum point C j,Cj of A j, wherein K is less than l.
Step (2): calculate the position relation of A j and the reference point C j, the position relation includes the distance between A j and C j One included angle/>Included angle/>Is the angle between the line segment C jKk and the line segment C jAj.
And/>The calculation formulas of (a) are shown as formula (1) and formula (2).
Step (3): for a pair ofAnd/>And (5) carrying out normalization processing.
The data is given the same metric by the normalization process, since the hands of each person are different in size. First, half of the connecting line of the key point K k、Kl is taken as L j, and the distance is calculatedCarrying out normalization treatment; taking the key point K 0 as the origin, taking the included angle theta oj between the connection line of the key point K 0-K5 and the connection line of the key point K 0-Kl (taking the key point 17 if the key point K l is the key point 5, namely changing the connection line of the key point K 0-Kl into the connection line of the key point K 0-K17), and determining the included angle/>And (5) carrying out normalization processing. The distance d 'c,j and the angle θ' j (A) after normalization are obtained by the formulas (3) and (4):
Step (4): the distances d 'c,j and the angles theta' j (A) in all the manually marked hand images are respectively averaged, and the average distance between A j and the reference point C j thereof is calculated Average angle/>As in formula (5) and formula (6):
where n is the total number of manually annotated hand images.
Step (5): according to average distanceAverage angle/>And (3) solving the relation between the distance j and the Angle j between the unknown acupoint and the reference point in the input hand image. Calculation formula (7) and formula (8):
the specific method of polynomial fitting method is as follows:
the polynomial used in the method is:
Where M is the order of the polynomial, x j denotes the power of x, and w= (w 0,w1,……wM) denotes the coefficients of the polynomial.
The method uses a polynomial fitting method to predict the positional relationship of the hole sites and the key points. Finding out proper reference points according to the positions of the key points and the acupuncture points, and solving the distance between the reference points and the acupuncture points in all manual labeling images by the same method as the method for obtaining the relation between the acupuncture points and the key points by an average methodAnd the angle theta j (A) is used for selecting the length L j and the angle theta oj, and performing polynomial fitting on the distance between the acupoint point and the datum point and the angle between the acupoint point and the datum point and between the acupoint point and the datum point respectively, so as to obtain the position relation between the acupoint point and the datum point.
And 4, calculating the position of the acupoint.
According to the position relation between the acupoint points and the key points obtained by the two methods in the step 3, the datum point is taken as the origin of coordinates, and the theory is deduced according to the chord length formula of the conic curve:
the method can be deduced:
xa＝xb±|PaPb|*cosα (12)
xa＝xb±|PaPb|*cosα (13)
Where, |p aPb | represents the distance between two points (two end points of line segment |p aPb | of P a(xa,ya)、Pb(xb,yb), and α is the inclination angle of the straight line P aPb.
The position relations of the acupoint points and the key points are put into a formula, and the coordinate points (x' j (A),y″j (A)) of the acupoint points can be obtained:
Wherein (x j (C),yj (C)) is the reference point coordinate corresponding to the point of the acupoint, length j is the distance between the point of the acupoint and the reference point, and Angle j is the Angle between the point of the acupoint and the reference point.
The invention also provides a device for identifying the hand acupoints of the Mongolian medicine based on image processing, which comprises:
The image acquisition module is used for acquiring hand pictures of the object, wherein the hand pictures comprise palm pictures and back pictures. The image acquisition module is a camera and a flat plate with pure background, and the flat plate is used for placing hands of a user so that the camera can take pictures of the hands to obtain hand pictures, and when the web camera is adopted, real-time positioning can be realized.
The first recognition module is used for recognizing whether the hand picture is a palm picture or a back hand picture, obviously, the recognition can be realized based on the YOLOv model obtained through the training, and the definition can be performed when the picture is acquired, and the recognition can be directly completed through the definition.
And the second recognition module is used for recognizing the hand key points of the palm picture or the back hand picture, wherein the hand key points are the characteristic of remarkable hand, so that the hand key points can be easily recognized and obtained, and the recognition can be realized based on the manually marked image data of the google open source library MEDIAPIPE.
And the positioning module is used for positioning the acupoint positions of the hand pictures according to the position relation between the hand key points and the acupoint positions. The positional relationship is the conclusion obtained in the step 3.
And the marking module is used for marking the acupuncture points on the hand picture according to the positioning of the positioning module. In the present invention, the first identification module, the second identification module, the positioning module and the marking module may be integrated into the processor.
And the output module is used for outputting the marked hand picture. By way of example, it may be a display device coupled to a processor.
In order to better examine the experimental results, the method compares the results of the two mentioned acupoint detection methods. And comparing the identification results of the two experimental methods with the manually marked Mongolian hand acupoint positions respectively. The three aspects of area offset error, euclidean distance error and offset error are respectively compared during comparison.
The area offset error is a rectangular area S formed by connecting an artificially marked acupoint (x j (A),yj (A)) with a detected acupoint (x' j (A),y″j (A)) as an oblique diagonal line, and the obtained data is averaged to obtain the average area offset error. The average area offset error E S is defined as:
Where m is the total number of experimental images.
The average Euclidean distance error E D is given by:
Because of the different proportions of the images, the error needs to be normalized, i.e. offset error, in order to evaluate the accuracy of the experiment more objectively. The average offset error E R is defined as:
where d 0,9 is the distance between keypoint 0 and keypoint 9, i.e. the palm root to middle finger root distance.
The average area offset error E S, the average euclidean distance error E D, and the average offset error E R are shown in table 1. The data in Table 1 are the error values for each hole site under the two methods. As can be seen from table 1, the method of locating the positions of the acupoints using the average value method for acupoints a 1、A2 is more accurate than the polynomial fitting method, and the method of locating the positions of the acupoints using the polynomial fitting method for acupoints a 3、A4、A5 is more accurate than the average value method. The final experimental result shows that from the aspect of area deviation error, the deviation of the five acupoint positions is smaller than the pressing area of the finger of the human body; from the average Euclidean distance error, the deviation of the five acupoint positions is smaller than the width of the index finger of the human body; the average distance from the palm root to the middle finger root of a normal adult is 12 cm, the width of the index finger is 1 cm, the deviation error formula (17) is carried into, the result is 0.083, and the deviation of the five hole sites is less than 0.083. The experimental results were evaluated by three error analysis methods, and the five point deviations were all within acceptable ranges. Experimental results show that the two methods for acquiring the images can accurately identify the hand acupoint positions of the Mongolian medicine.
Table 1 error values for each acupoint under two methods
Note that: the average area offset error value and the average euclidean distance error value in the table are both pixel values in the 320 x 320 image.
Fig. 3 and 4 show the test results of the present invention, wherein fig. 3 is the back of the hand, fig. 4 is the palm of the hand, and it is apparent from (a), (b), (c) in fig. 3 and (a), (b), (c) in fig. 4 that the present invention has an accurate positioning effect.
In summary, the invention obtains 21 hand key point coordinates by MEDIAPIPE, uses YOLOv3 model to make a judgment of palm or back of hand, obtains the distance and angle relation between the hand acupoint of the Mongolian medicine and the hand key point by calculation, and finally obtains the hand acupoint coordinates of the Mongolian medicine. Experiments show that the method has high recognition rate and high robustness, and the positions of the Mongolian hand acupoints can be obtained in real time through the web camera, so that the Mongolian hand acupuncture teaching is more visual and accurate.
Claims (5)
1. A method for identifying Mongolian medical hand acupoints based on image processing, comprising:
Step 1, recognizing palm and back of hand, wherein five Mongolian medical hand acupoints to be recognized are arranged on the palm, two of the acupoints are distributed on the back of hand, and the other three acupoints are distributed on the back of hand; the method comprises the steps of adopting a YOLOv model as an identification model, crawling hand images from the Internet through web crawler software, marking the palm and the back of the hand, manufacturing a palm and back of the hand identification dataset, dividing the palm and back of the hand identification dataset into a training set and a testing set, and training the identification model by utilizing the dataset;
Step 2, obtaining hand key points of the manual annotation hand image in the database; the database is google open source database MEDIAPIPE, which is manually marked with 21 hand images, the number of key points of the hand is 0, the number of the wrist is 0 from the root of the thumb to the fingertip, the number of the joints is from 1 to 4 in sequence, the number of the joints is from 5 to 8 in sequence, the number of the joints is from 9 to 12 in sequence, the number of the joints is from the root of the ring finger to the fingertip, the number of the joints is from 13 to 16 in sequence, the number of the joints is from the root of the little finger to the fingertip, and the number of the joints is from 17 to 20 in sequence, thereby obtaining 21 hand key points K 0～K20;
Step 3, acquiring the position relation between the hand key points and the acupuncture points of the hand image of the manual annotation in the database; wherein, the position relation is obtained by adopting an average value method or a polynomial fitting method;
The average value method comprises the following steps:
Step (1), acquiring coordinates (x i (K),yi (K)) of an ith hand key point K i of an artificial annotation hand image and coordinates (x j (A),yj (A)) of a jth acupoint A j in a database, selecting two key points K k、Kl closest to the A j, connecting the K k、Kl, and taking the midpoint of the connecting line as coordinates (x j (C),yj (C)) of a datum point C j,Cj of the A j, wherein K is less than l;
Step (2), the position relation of A j and the reference point C j thereof is calculated, wherein the position relation comprises the distance between A j and C j One included angle/>The included angle/>The included angle between the line segment C jKk and the line segment C jAj;
Step (3): for a pair of And/>The normalization processing is carried out, firstly, half of the connecting line of the key point K k、Kl is taken as L j, and the pair/>Carrying out normalization treatment; taking a key point K 0 as an origin, taking an included angle theta oj between a key point K 0-K5 connecting line and a key point K 0-Kl connecting line, and changing the key point K 0-Kl connecting line into a K 0-K17 connecting line if l=5; pair/>Normalization processing is carried out, and the distance d c′,j and the angle/>, after normalizationThe formula is:
Obtaining;
Step (4): distance d c′,j and angle of hand image for all manual labels in database Respectively averaging and calculating the average distance/>, between A j and the reference point C j thereofAverage angle/>
Wherein n is the total number of manually marked hand images in the database;
Step (5): according to average distance Average angle/>The relation between the distance j and the Angle j between the unknown point A and the reference point in the input hand image is obtained, and the calculation formula is as follows:
The polynomial fitting method finds reference points according to the key points and the point positions of the acupoints, and obtains the distances between the reference points and the point positions in all manual labeling images Selecting L j、θoj from the angle theta j (A), and performing polynomial fitting on the distance between the acupoint and the datum point and the angular relationship between the acupoint and the datum point and between the acupoint and the key point respectively to obtain the position relationship between the acupoint and the key point;
step 4, for the hand image to be identified, positioning the hand key points of the hand image to be identified, and calculating the point positions of the acupuncture points of the hand image to be identified based on the position relation; wherein, the point coordinates point The method is calculated by the following formula:
Wherein the method comprises the steps of Length j is the distance between the point and the reference point, angle j is the Angle between the point and the reference point.
2. An apparatus for identifying Mongolian hand acupoints based on image processing, for implementing the method for identifying Mongolian hand acupoints based on image processing as described in claim 1, comprising:
The image acquisition module is used for acquiring hand pictures of the object, wherein the hand pictures comprise palm pictures and back pictures;
the first recognition module is used for recognizing the hand picture as a palm picture or a back hand picture; the first recognition module completes recognition based on YOLOv model or completes recognition based on definition when the image acquisition module acquires;
the second recognition module is used for recognizing hand key points of the palm picture or the back hand picture; the second recognition module completes recognition based on the manually marked image data of the google open source library MEDIAPIPE;
the positioning module is used for positioning the hole sites of the hand picture according to the position relation between the hand key points and the hole sites;
the marking module is used for marking the acupuncture points on the hand picture according to the positioning of the positioning module;
and the output module is used for outputting the marked hand picture.
3. The device for identifying Mongolian medical hand acupoints based on image processing according to claim 2, wherein the image acquisition module is a camera and a flat plate with pure background, the flat plate is used for placing the hand of a user, and the camera is used for taking a picture of the hand.
4. The device for identifying Mongolian medical hand acupoints based on image processing of claim 2, wherein the first identification module, the second identification module, the first positioning module, the second positioning module and the marking module are integrated in a processor.
5. The device for identifying Mongolian medical hand acupoints based on image processing of claim 2, wherein the output module is a display device connected to a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178680.7A CN115527235B (en) | 2022-09-27 | 2022-09-27 | Method and device for identifying Mongolian medical hand acupoints based on image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211178680.7A CN115527235B (en) | 2022-09-27 | 2022-09-27 | Method and device for identifying Mongolian medical hand acupoints based on image processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115527235A CN115527235A (en) | 2022-12-27 |
| CN115527235B true CN115527235B (en) | 2024-06-21 |
Family
ID=84698702
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211178680.7A Active CN115527235B (en) | 2022-09-27 | 2022-09-27 | Method and device for identifying Mongolian medical hand acupoints based on image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115527235B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116469519B (en) * | 2023-03-23 | 2024-01-26 | 北京鹰之眼智能健康科技有限公司 | Human body acupoint obtaining method based on infrared image |
| CN116864079A (en) * | 2023-06-28 | 2023-10-10 | 成都中医药大学 | Method, system and storage medium for measuring same body size of human body by transverse finger |
| CN116958244A (en) * | 2023-09-21 | 2023-10-27 | 深圳市德壹医疗科技有限公司 | Patient acupoint positioning method, equipment and medium based on visual recognition system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113486758A (en) * | 2021-06-30 | 2021-10-08 | 浙江大学 | Automatic hand acupuncture point positioning method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146879B (en) * | 2018-09-30 | 2021-05-18 | 杭州依图医疗技术有限公司 | Method and device for detecting bone age |
| CN110648361B (en) * | 2019-09-06 | 2022-01-11 | 深圳市华汉伟业科技有限公司 | Real-time pose estimation method and positioning and grabbing system of three-dimensional target object |
| CN111598949B (en) * | 2020-04-10 | 2023-03-24 | 上海中医药大学 | Automatic positioning method for head and neck acupoints of human body |
| CN112416134A (en) * | 2020-12-10 | 2021-02-26 | 华中科技大学 | Device and method for quickly generating hand key point data set |
-
2022
- 2022-09-27 CN CN202211178680.7A patent/CN115527235B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113486758A (en) * | 2021-06-30 | 2021-10-08 | 浙江大学 | Automatic hand acupuncture point positioning method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115527235A (en) | 2022-12-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115527235B (en) | Method and device for identifying Mongolian medical hand acupoints based on image processing | |
| Capecci et al. | The kimore dataset: Kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation | |
| Smeragliuolo et al. | Validation of the Leap Motion Controller using markered motion capture technology | |
| CN108764120B (en) | Human body standard action evaluation method | |
| CN110045823B (en) | Motion guidance method and device based on motion capture | |
| CN104207931B (en) | Face facial point precise positioning and acupuncture and moxibustion prescription, acumoxibustion prescription learning method | |
| CN112184705A (en) | Human body acupuncture point identification, positioning and application system based on computer vision technology | |
| CN109091380B (en) | Traditional Chinese medicine system and method for realizing acupoint visualization by AR technology | |
| WO2019196099A1 (en) | Method for positioning boundaries of target object in medical image, storage medium, and terminal | |
| CN111598949B (en) | Automatic positioning method for head and neck acupoints of human body | |
| WO2022068717A1 (en) | Abdominal acupuncture point positioning system, method and apparatus, control device, and storage medium | |
| CN113052138A (en) | Intelligent contrast correction method for dance and movement actions | |
| CN109685695A (en) | It is a kind of based on the teaching of Chinese medicine acupuncture simulated training method and system | |
| CN113616512A (en) | Human body acupuncture point positioning instrument based on traditional Chinese medicine meridian acupuncture point big data | |
| Sharma et al. | iYogacare: real-time Yoga recognition and self-correction for smart healthcare | |
| CN114187234A (en) | Method and system for locating acupoints | |
| CN115035037A (en) | Limb rehabilitation training method and system based on image processing and multi-feature fusion | |
| Klatzky et al. | Representing spatial location and layout from sparse kinesthetic contacts. | |
| Zhang et al. | Faceatlasar: Atlas of facial acupuncture points in augmented reality | |
| CN116740026A (en) | Acupoint positioning method based on whole body key point recognition and instance segmentation | |
| CN116704603A (en) | Action evaluation correction method and system based on limb key point analysis | |
| CN115530814A (en) | Child motion rehabilitation training method based on visual posture detection and computer deep learning | |
| Matsumoto et al. | Estimation of grip strength using monocular camera for home-based hand rehabilitation | |
| Wen et al. | An auricular division method based on ASM algorithm | |
| Chen et al. | Posenet based acupoint recognition of blind massage robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |