CN110210526A - Predict method, apparatus, equipment and the storage medium of the key point of measurand - Google Patents
Predict method, apparatus, equipment and the storage medium of the key point of measurand Download PDFInfo
- Publication number
- CN110210526A CN110210526A CN201910399587.0A CN201910399587A CN110210526A CN 110210526 A CN110210526 A CN 110210526A CN 201910399587 A CN201910399587 A CN 201910399587A CN 110210526 A CN110210526 A CN 110210526A
- Authority
- CN
- China
- Prior art keywords
- key point
- mark
- correlation
- measurand
- definition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Abstract
The present invention provides method, apparatus, electronic equipment and the storage medium of a kind of key point for predicting measurand, method therein includes: to obtain multiple mark set that mark is defined based on key point, wherein, the mark set is for recording one of mark personnel to the coordinate parameters of each key point mark of a measurand；The correlation between the coordinate parameters of key point is calculated according to the mark set of acquisition；Determined whether to update the key point definition according to the correlation being calculated.To be conducive to the definition that enhancing key point defines, reduces different labeled personnel and define the deviation of understanding to same key point, and improve the availability of key point, important foundation stone is established in the raising for the development efficiency and model quality of deep learning project.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to predict the method, apparatus of the key point of measurand, set
Standby and storage medium.
Background technique
In the deep learning for having supervision, it usually needs data are labeled, for example, for critical point detection task,
The mark of data is the key point coordinate of measurand in image.Wherein, the definition and selection of key point are usually to be led by project
To, by taking human body critical point detection as an example, more common COCO model just includes 18 human body key points at present, these passes
Key point includes the point on the joints such as point, such as elbow, wrist, knee, ankle on human body major joint.Select these
Key point is as detection target mainly there are two reason: first, the location information of this kind of key point to judge the posture of human body with
It acts extremely important；Second, this kind of key point has obvious feature in the picture, and define more clear.But for big
For most new projects, people generally require to redefine key point according to the target of project, this is defined key point
Process be usually subjective, it is therefore more likely that can because key point define it is indefinite and occur the key point that detection obtains can
With the not strong problem of property.Also, such issues that generally can data preparation completion after in addition data training complete after just by
It was found that huge time and economic loss can be brought to project development.
Summary of the invention
Based on this, method, apparatus, electronic equipment and the storage that the present invention provides a kind of key point for predicting measurand are situated between
Matter.
According to a first aspect of the embodiments of the present invention, the present invention provides a kind of sides of key point for predicting measurand
Method, which comprises
It obtains and defines multiple mark set of mark based on key point, wherein the mark set is for record wherein one
Coordinate parameters of a mark personnel to each key point mark of a measurand；
The correlation between the coordinate parameters of key point is calculated according to the mark set of acquisition；
Determined whether to update the key point definition according to the correlation being calculated.
According to a second aspect of the embodiments of the present invention, the present invention provides a kind of dresses of key point for predicting measurand
It sets, described device includes:
Module is obtained, for obtaining the multiple mark set for defining mark based on key point, wherein the mark collection shares
In recording one of mark personnel to the coordinate parameters of each key point mark of measurand；
Computing module, for calculating the correlation between the coordinate parameters of key point according to the mark set of acquisition；
Determining module updates the key point definition for determining whether according to the correlation being calculated.
The third aspect according to embodiments of the present invention, the present invention provides a kind of electronic equipment comprising:
Processor；
Memory, for storing the computer program that can be executed by the processor；
Wherein, the step of the method for the key point of the prediction measurand is realized when the processor executes described program
Suddenly.
Fourth aspect according to embodiments of the present invention, the present invention provides a kind of machine readable storage mediums, are stored thereon with
Program；The step of method of the key point of the prediction measurand is realized when described program is executed by processor.
Relative to the relevant technologies, the embodiment of the present invention at least produces following advantageous effects:
The embodiment of the present invention is determined whether fixed to key point by the correlation between the coordinate parameters according to key point
Justice is updated, (can be with thus in the case where realizing that correlation between the coordinate parameters of key point is unsatisfactory for preset condition
It is larger to reflect that key point defines the indefinite understanding difference for causing different labeled personnel to define same key point, and/or closes
Key point is not applicable), key point definition is updated, so that key point definition tends to be clear, is conducive to reduce different labeled personnel to same
One key point defines the deviation of understanding, improves the availability of key point, and is the development efficiency and model matter of deep learning project
Important foundation stone is established in the raising of amount.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
It can the limitation present invention.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention
Example, and be used to explain the principle of the present invention together with specification.
Fig. 1 is a kind of present invention stream of the method for the key point for predicting measurand shown according to an exemplary embodiment
Cheng Tu；
Fig. 2 is the distance phase of all key points corresponding to present invention left side image shown according to an exemplary embodiment
The statistics schematic diagram of closing property；
Fig. 3 is a kind of present invention scatter plot of inter-class correlation shown according to an exemplary embodiment；
Fig. 4 is a kind of present invention knot of the device for the key point for predicting measurand shown according to an exemplary embodiment
Structure block diagram；
Fig. 5 is the hardware structure diagram of present invention a kind of electronic equipment shown according to an exemplary embodiment.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistented with the present invention.On the contrary, they be only with it is such as appended
The example of device and method being described in detail in claims, some aspects of the invention are consistent.
It is only to be not intended to limit the invention merely for for the purpose of describing particular embodiments in terminology used in the present invention.
It is also intended in the present invention and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the present invention
A little information should not necessarily be limited by these terms.These terms are only used to for same type of information being distinguished from each other out.For example, not departing from
In the case where the scope of the invention, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as
One information.Depending on context, word as used in this " if " can be construed to " ... when " or " when ...
When " or " in response to determination ".
In deep learning task, Yao Xunlian obtains object module, and it is suitable according to learning tasks object definition in advance to need
Label, for example, Yao Xunlian obtains relevant object module in human body posture Detection task, with the prediction based on object module
As a result the detection to human body posture is realized, the required label defined is usually the key point of partes corporis humani position and the position of key point
Set coordinate.Wherein, label is usually to be marked according to the definition of the key point of partes corporis humani position to tested human body by mark personnel
Note obtains, therefore obtained label is affected by human factor；And since the quality of label can be largely affected by base
In the prediction effect for the model that this label training obtains, therefore it is necessary to the key points of explication partes corporis humani position, and to key
The definition and importance of point are judged, so that key point definition tends to be clear, reduce different labeled personnel to same key
The deviation that point definition understands, and the availability of key point is improved, to be the development efficiency and model quality of deep learning project
Raising establish important foundation stone.
Based on this, as shown in Figure 1, the embodiment of the invention provides a kind of method of key point for predicting measurand, it can
To be applied in terminal, also can be applied in server, which comprises
S011 obtains multiple mark set that mark is defined based on key point, wherein the mark set is for recording it
In a mark personnel to the coordinate parameters of each key point mark of measurand；
S012 calculates the correlation between the coordinate parameters of key point according to the mark set of acquisition；
S013 determines whether that updating the key point defines according to the correlation being calculated.
Among the above, the measurand is object targeted in the deep learning task based on key point, including is based on
Testee's unspecified angle shoots resulting image, for example, when the deep learning task is human body posture Detection task, it is described
Measurand may include that unspecified angle based on tested body shoots resulting image, e.g., direct picture including several human bodies,
Side image and/or back side image.
The key point definition marks corresponding to all key points marked needed for personnel measurand for indicating
Definition, for example, for human body front position, can wrap containing 20 be predefined in human body posture Detection task
Key point；For human body sideway stance, can wrap containing 13 key points being predefined；It, can for human body back side position
To include 18 key points being predefined.It is to be understood that each mark personnel can be scheduled based on human body front
The definition of 20 key points of justice, marks out 20 key points from every direct picture；It can be scheduled based on human body side
13 key points definition of justice, marks out 13 key points from every side image；It can be predefined based on the human body back side
18 key point definition, mark out 18 key points from every back side image.
It, can be according to need to increase or decrease it should be noted that the quantity for the key point that above-mentioned each face is included is unlimited.Separately
Outside, above-mentioned all key points can be defined according to the posture index of required calculating, for example, in the detection of human body posture, people
Point in body right and left shoulders can be used to quantify high-low shoulder degree, and some point on earhole and some point on shoulder can be with
It is used to quantify head inclination degree.Based on this, left shoulder, right shoulder can be pre-defined, on earhole according to the demand
Key point.It therefore, in practical applications, can be according to the corresponding key point of posture index definition of required evaluation, in the present invention
The selection of key point and definition are not repeated one by one in embodiment.
Each mark personnel can be labeled measurand based on the key point definition pre-defined as a result, with
Under, the direct picture (have N direct pictures) that includes N number of testee with measurand, human body front are corresponding with n key point
For illustratively a mark personnel to the annotation process of measurand:
For every human body direct picture, mark personnel can the n key point according to defined in human body front determine
Justice carries out key point to a direct picture and marks to obtain the coordinate parameters of n key point.It then, can be by a front elevation
As saving as a mark set by the coordinate parameters that a mark personnel mark resulting n key point.
It follows that after a mark personnel carry out key point mark to N direct pictures, the available and N
Open the one-to-one N number of mark set of direct picture, each coordinate parameters of the mark set comprising n key point.Also, one
It is available to be corresponded with the Z mark personnel after direct picture carries out key point mark respectively by Z mark personnel
Z mark set, it is each mark set include n key point coordinate parameters, based on this, a direct picture can correspond to
Have and marks resulting Z mark set by different labeled personnel.
In one embodiment, can also side image to N number of testee and back side image carry out key point mark,
The mark set of the key point in side image and back side image to respectively obtain each testee, so as to using more
Data go to judge the definition and importance of key point, further increase judging result and final selected key point
Availability.
In one embodiment, the direct picture, side image and back of N number of testee can be obtained respectively by picture pick-up device
Face image, and after obtaining all images, by image transmitting to can be marked in the terminal that personnel are labeled processing.By
This, all mark personnel can define according to key point and respectively carry out key point mark to all images, and then, the present invention is implemented
The image that the executing subject of example the method can be marked according to each mark personnel, identification obtain all keys in each image
The coordinate parameters of point, and it is stored as corresponding mark set.One mark personnel of each mark set record are in an image
Each key point mark coordinate parameters.
As a result, during executing the method for the embodiment of the present invention, it can be obtained according to aforesaid way based on key
Multiple mark set of point definition mark.After obtaining the multiple mark set, so that it may execute step S012, i.e., according to obtaining
The mark set obtained calculates the correlation between the coordinate parameters of key point, and in one embodiment, the correlation may include
Distance correlation is based on this, under the premise of each measurand corresponding two marks set, it can be understood as each tested
Under the premise of object carries out key point mark by two mark personnel, the seat that key point is calculated according to the mark set of acquisition
Mark the correlation between parameter, comprising:
S0121 calculates each measurand according to coordinate parameters of each key point in two mark set
The distance of each key point；
S0122, defines the distance of identical key point based on all measurands, calculate define identical key point away from
From the distance between correlation.
Hereinafter, for example illustrating to adjust the distance the meter of correlation by the step S0121 and the step S0122
Calculation process:
Assuming that there is N number of testee, for each testee, its direct picture, left side image, right image are taken in respectively
And back side image；Then it is found that for N number of testee, a shared N direct pictures, N left side images, N right images and N
Open back side image.Assuming that two mark personnel are according to the understanding respectively defined to key point, independent (mutually having no AC deposition)
The image of all intakes is labeled to obtain the mark set of every image.It is with i-th image in N direct pictures
Example, it is assumed that one of mark personnel carry out the mark collection that n key point marks to i-th direct picture and are combined into [(xi11,
yi11), (xi12, yi12)…(xi1n, yi1n)], another mark personnel carry out what n key point marked to i-th direct picture
Mark collection is combined into [(xi21, yi21), (xi22, yi22)…(xi2n, yi2n)]；Wherein, i is integer and 1≤i≤N；N be integer and n >=
1；(xi1n, yi1n) indicate the coordinate that the first mark personnel are labeled n-th of key point of i-th direct picture,
(xi2n, yi2n) indicate the coordinate that the second mark personnel are labeled n-th of key point of i-th direct picture.
After every image is marked respectively by two mark personnel as a result, two groups of mark set can be all generated, comparison is passed through
This two groups of mark set of every image can be obtained in this two groups of mark set between two coordinate parameters of each key point
Distance, the distance includes Euclidean distance, horizontal distance and vertical range.Wherein, with the key point n of i-th direct picture
Illustrate the calculating process of the distance of each key point:
Since the key point n of i-th direct picture joins in two coordinates marked in set of i-th direct picture
Number is respectively (xi1n, yi1n) and (xi2n, yi2n)；, can be by formula 1. based on this ---The key point n of i-th direct picture is calculated by two
The secondary Euclidean distance d marked between obtained pointlin, can be by formula 2. --- dxin=| xi1n-xi2n| it is calculated described
The horizontal distance d between point that the key point n of i-th direct picture is marked twicexin, can be by formula 3. ---
dyin=| yi1n-yi2n| it is calculated vertical between the point that the key point n of i-th direct picture is marked twice
Distance dyin。
It can be seen from the above, can according to above-mentioned formula 1., formula each of every image is 3. 2. calculated with formula
Euclidean distance, horizontal distance and the vertical range between point that key point obtains after being marked twice.
After obtaining Euclidean distance, horizontal distance and the vertical range of each key point in every image, it can calculate
The distance between all distances of identical key point correlation is defined, in the present embodiment, the distance correlation includes Europe
Formula is apart from mean value, horizontal distance mean value and vertical range mean value, it is possible to understand that are as follows: for key point n, based on it in N fronts
It is respectively d that the Euclidean distance that the coordinate of resulting point is calculated is marked in imagel1n、dl2n…dlNn, horizontal distance is respectively
dx1n、dx2n…dxNn, vertical range is respectively dy1n、dy2n…dyNn；Then can be by formula 4. ---It calculates
Obtain the Euclidean distance mean value of key point nCan be by formula 5. ---The key is calculated
The horizontal distance mean value of point nCan be by formula 6. ---The vertical of key point n is calculated
Apart from mean value
It can be seen from the above, can by formula 4., 5. 6. formula is calculated in the N direct pictures with formula and defines
Euclidean distance mean value, horizontal distance mean value and the vertical range mean value of identical key point, the Euclidean distance of each key point are equal
Value, horizontal distance mean value and vertical range mean value are used to characterize the distance correlation that the key point corresponds to all images.
Similarly, can be calculated according to above-mentioned calculating process the distance correlation of each key point in N back side images,
The distance correlation of each key point and N open the distance correlation of each key point in right images in N left side images.
In another embodiment, the number for marking personnel can be not limited to two, for example, can be two or more.Base
Key point n of every direct picture can be calculated first between the key point n that every two mark personnel are marked in this
Euclidean distance, horizontal distance and vertical range, then seek key point n repeatedly marked in an image it is resulting all
The third mean value of first mean value of Euclidean distance, the second mean value of all horizontal distances and all vertical ranges；Subsequently, for N
Euclidean distance is calculated according to formula 4. based on all first mean values of the key point n in the key point n for opening direct picture
According to formula 5. based on all second mean values of the key point n horizontal distance mean value is calculated, 6. according to formula in mean value
All third mean values based on the key point n, are calculated vertical range mean value.
After obtaining the distance correlation of each key point, in one embodiment, in order to improve the distance phase of each key point
The visualization of closing property, can be by the distance phase for all key points being calculated based on the resulting image of same angle shot
Closing property is depicted as statistical chart, for example, as shown in Fig. 2, Fig. 2 is present invention image institute in the left side shown according to an exemplary embodiment
The statistics schematic diagram of the distance correlation of corresponding all key points can clearly learn the distance of each key point from Fig. 2
The size cases of correlation.In one embodiment, it can also be arranged each key point in statistical chart with certain arrangement regulation
Distance correlation, as shown in Fig. 2, being to be with the size of the Euclidean distance mean value in the distance correlation of each key point in Fig. 2
Foundation, according to the sequence of Euclidean distance mean value from small to large, the distance correlation of each key point of sequential.
After obtaining the distance correlation of each key point, it can be determined and be corresponded to based on the distance correlation of each key point
Key point define whether accurately, it is possible to understand that are as follows: each key can be understood based on the distance correlation of each key point
The mark levels of precision of point, while can also learn the source direction of mark difference.Wherein it is possible to first according to the Europe of each key point
Size of the formula apart from mean value judges the mark levels of precision of each key point, for example, if the Euclidean distance mean value of key point is small
In preset first threshold, then it can indicate that the error of key point is smaller, belongs to negligible error, then can recognize
It is accurate enough for the definition of the key point, it does not need to be updated.But if the Euclidean distance mean value of key point is greater than or equal to
The first threshold can then indicate that the error of key point is larger, belongs to the error that can not ignore, then it is considered that the key
The definition of point is not accurate enough, needs to be updated.In addition, being greater than or equal to the key of first threshold for Euclidean distance mean value
Point further can cause crucial point tolerance according to the size of the horizontal distance mean value of key point and vertical range mean value to learn
Biggish source, for example, if the horizontal distance mean value of key point is much larger than vertical range mean value or is greater than preset second threshold
Value, then it represents that error source is mainly horizontally oriented.
Based on this, in one embodiment, it can export and update the prompt that the biggish key point of error defines, export prompt
Content may include at least one of: key point title, key point define, the error source of key point.In another embodiment
In, voluntarily the definition of key point can also be updated.Wherein, to improve the definition and precision that key point defines,
In one embodiment, the key point define may include the key point horizontal coordinate parameter definition and/or vertical coordinate
The definition of parameter；When determining that updating the key point defines, the method can also include: S0141, related according to the distance
Property updates the definition of the horizontal coordinate parameter of the key point and/or the definition of vertical coordinate parameter.
In the step S0141, the key point of first threshold is greater than or equal to for Euclidean distance mean value, if it is horizontal
It is greater than or equal to second threshold apart from mean value and vertical range mean value is greater than or equal to third threshold value, then to the water of the key point
The definition of flat coordinate parameters and the definition of vertical coordinate parameter are updated；If horizontal distance mean value is greater than or equal to the second threshold
It is worth and vertical range mean value is less than third threshold value, then only the definition of the horizontal coordinate parameter of the key point is updated；Such as
Fruit vertical range mean value is greater than or equal to third threshold value and horizontal distance mean value is less than second threshold, then only to the key point
The definition of vertical coordinate parameter is updated.
It among the above, can be by reducing the range of definition and/or vertical coordinate parameter of the horizontal coordinate parameter of key point
The range of definition, for example in the definition of horizontal coordinate parameter and/or the definition of vertical coordinate parameter the key point to be added attached with it
The description of positional relationship between nearly object of reference, so that the definition of horizontal coordinate parameter and determining for vertical coordinate parameter of key point
Justice tends to precisely, so that different labeled personnel define understanding having the same to same key point, can guarantee anyone in this way
It can mark to obtain accurate key point in the picture based on key point definition, to obtain the mark of accurate model training
Label.
In another embodiment, directly whether can need to carry out the update that key point defines by artificial judgment.With
Under be illustrated how by artificial judgment whether to need to carry out the update that key point defines based on Fig. 2: as can be seen from Figure 2, in Fig. 2
Shown in 12 key points, the Euclidean distance mean value of key point 10,11 and 12 is relatively large, and this 3 key points is vertical
It is almost big apart from mean value and Euclidean distance mean value, and horizontal distance mean value is more much smaller than vertical range mean value.Therefore, pass through
Artificial observation Fig. 2 may directly learn that there are biggish errors for key point 10,11 and 12, and these errors are mainly derived from pass
The definition of existing range deviation in key point vertical direction, the vertical coordinate parameter that thus judgement obtains these key points is inadequate
Precisely.Then, it can be updated by the definition of the vertical coordinate parameter manually to these key points, such as in vertical coordinate
The description of the positional relationship between the key point and its neighbouring object of reference is added in the definition of parameter, is defined with to improve key point
Accuracy.
Although the definition that key point defines can be improved by any of the above-described embodiment, different labeled personnel are reduced to same
One key point defines the deviation of understanding, improves availability that key point defines and the label training that defines based on key point
The prediction effect of the model arrived, still, in some tasks, as posture Detection task also needs after obtaining key point coordinate
Posture index is calculated according to the positional relationship of multiple key points.Therefore the result of posture detection depends not only on single key
The position precision of point, and the influence of the relative position between multiple key points is also suffered from, for example, the water of human body or so shoulder
Flat degree is calculated by the coordinate of two key points on the shoulder of left and right, this just also requires the relative position of two key points
It meets the requirements.Therefore, for preferably improve key point availability and model prediction effect, in one embodiment, in addition to away from
Other than correlation, the correlation further includes inter-class correlation, and the inter-class correlation is for assessing different labeled personnel institute
The similarity of the relative position of multiple key points of mark, for example, it is assumed that in the key point that wherein a mark personnel are marked
Key point A and key point B can be used for assessing posture index a, similarly, the key in key point that another mark personnel are marked
Point A and key point B can be used for assessment posture index a, in such event, the similarity of the relative position it is to be understood that
Based on key point A and key point B that wherein a mark personnel are marked the posture index a being calculated and based on another mark people
The similarity between posture index a that the member key point A marked and key point B is calculated, this can be considered as a kind of result
Similarity.Based on this, in the step S012, the correlation between the coordinate parameters of key point is calculated according to the mark set of acquisition
Property, further includes:
S0123, for each mark set of each measurand, according to the coordinate parameters meter of specified several key points
Calculation obtains corresponding metrics evaluation parameter；
S0124, between the metrics evaluation parameter based on the metrics evaluation gain of parameter different labeled personnel being calculated
Inter-class correlation.
Among the above, specified several key points are used for parameter evaluation parameter, it should be noted that the finger of required calculating
The quantity for marking evaluation parameter is identical as the specified group number of several key points, for example, it is assumed that the metrics evaluation of required calculating
Parameter has 3, then can specify 3 groups of key points, every group of key point includes at least two key points, it is possible thereby to be based on 3 groups of passes
The coordinate parameters of key point calculate separately to obtain 3 metrics evaluation parameters.
Hereinafter, for example illustrating the meter by the step S0123 and step S0124 to inter-class correlation
Calculation process:
Assuming that can detecte to obtain I for every side image (left side image or right image) in N side images
A posture metrics evaluation parameter, I are integer, and I >=1；In one example, the value of I can be 7.These posture metrics evaluations
Parameter can be shown as in angle and/or side image between the line and horizontal line of two key points in side image
The lines of three key points be formed by angle.Any angle therein can be joined based on the coordinate of corresponding key point
Number is calculated, and specific calculation can be found in the relevant technologies, herein without repeating.
Based on this, it is assumed that the quantity for marking personnel is 2, is marked based on wherein one mark personnel to i-th side image
Obtained specified several key points, I posture metrics evaluation parameter being calculated is respectively a1i1、a2i1、…aIi1。
Based on specified several key points that another mark personnel mark i-th side image, the I being calculated
Posture metrics evaluation parameter is respectively a1i2、a2i2、…aIi2.Wherein, aIi1In aIIndicate i-th posture metrics evaluation parameter,
aIi1In aIiIndicate the i-th posture metrics evaluation parameter of i-th side image, aIi1Indicate the i-th of first mark personnel
The i-th posture metrics evaluation parameter of side image is opened, label of any posture metrics evaluation parameter can be carried out based on this
Understand.
It follows that for any one posture metrics evaluation parameter, based on any mark personnel to N side image marks
The result that resulting designated key point calculates separately have it is N number of, for example, for one of posture metrics evaluation parameter aj, j
For integer and 1≤j≤I；Based on N side images, the result that any mark personnel generate has N number of, corresponds to one of mark
N number of result of note personnel are as follows: aj11, aj21, aj31... aji1, aj(i+1)1…ajN1；N number of result corresponding to another mark personnel
Are as follows: aj12, aj22, aj32... aji2, aj(i+1)2…ajN2。
It can be seen from the above, K is integer and K based on the designated key point that K mark personnel mark N side images
>=2, obtained posture metrics evaluation parameter ajK × N number of data result can be shown in Table 1:
1 posture metrics evaluation parameter a of tablejTables of data
It should be noted that in table 1, by the posture metrics evaluation parameter a of same mark personnel generationjN number of knot
Fruit data are joined as the column data in same row, by K mark personnel based on the posture metrics evaluation that same side image generates
Number ajK result data as the row data in same a line.
It is based on a posture metrics evaluation parameter a as a result,jN × K result data, can be by formula 7. ---Posture metrics evaluation parameter a is calculatedjInter-class correlation ICCj.It is public
Formula 7. in, MSR is the square of row factor, MSRjFor posture metrics evaluation parameter ajRow factor it is square；MSE is the equal of error
Side, MSEjFor posture metrics evaluation parameter ajError it is square；MSC is the square of column factor, MSCjFor posture metrics evaluation ginseng
Number ajColumn factor it is square.It is possible thereby to pass through between 7. formula is calculated corresponding group of any posture metrics evaluation parameter
Correlation is indicated in this example with ICC (Intraclass Correlation Coefficient, interclass correlation coefficient) between group
Correlation, value range is [0,1], for characterizing the ratio of individual variation degree and total degree of variation, wherein when the value of ICC is 0
When, indicate onrelevant between all results of corresponding posture metrics evaluation parameter；When the value of ICC is 1, corresponding posture is indicated
It is associated with by force between all results of metrics evaluation parameter.
Similarly, it can be calculated according to above-mentioned calculating process between metrics evaluation parameter involved in N direct pictures
Inter-class correlation, the inter-class correlation between metrics evaluation parameter involved in N back side images.
After obtaining the inter-class correlation between the metrics evaluation parameter of different labeled personnel, inter-class correlation can be based on
Determine whether to update key point definition with distance correlation, this is based on, in one embodiment, in the step S013, according to meter
Obtained correlation determines whether that updating the key point defines, and may include:
S0131 obtains the key that the inter-class correlation is less than preset 4th threshold value from specified several key points
Point；
S0132 determines whether to update key point definition according to the corresponding distance correlation of each key point of acquisition.
Among the above, each threshold value can empirically or experimentally gained, herein without repeating.
In one example, the 4th threshold value can be 0.5.
Hereinafter, for example illustrating to be determined whether to define key point according to inter-class correlation and distance correlation to carry out
The process of update:
When inter-class correlation be less than four threshold value when, indicate the corresponding metrics evaluation parameter of the inter-class correlation (with
Be referred to as target indicator evaluation parameter down) all results between onrelevant or weak rigidity (can be understood as institute resultful one
Cause property is unsatisfactory for requiring), then further determine that the distance for calculating key point used in the target indicator evaluation parameter is related
Property determines that key point defines whether accurately, wherein determines that key point defines whether accurately according to the distance correlation of key point
The visible above-mentioned related record of realization process, herein without repeating.
The key point that there is definition inaccuracy is determined when defining, indicate the target indicator evaluation parameter all results it
Between onrelevant or weak rigidity phenomenon be possible as key point and define caused by inaccuracy, be based on this, can be according to upper
It states step S0141 to be updated the key point definition of definition inaccuracy, to improve all of the target indicator evaluation parameter
As a result the consistency between, and then improve the prediction effect of model.
But in practice, there is also determine that the definition for obtaining all key points is all accurate according to the distance correlation of key point
The case where, that is, there is no the key point definition of definition inaccuracy.At this point, indicating all of the target indicator evaluation parameter
As a result onrelevant or weak rigidity phenomenon between are not as key point and define caused by inaccuracy, it may be possible to because crucial
Point does not choose not pair or the accuracy requirement that mark to key point of the target indicator evaluation parameter is excessively high caused, based on this,
In one embodiment, the key point for calculating the target indicator evaluation parameter can be chosen again, alternatively, the target is referred to
Evaluation parameter is marked to delete.With this corresponding, in one embodiment, the method also includes:
S0142, when determining the definition for not updating acquired key point from specified several key points, output is used
It is not suitable for evaluating the prompt information of measurand in the instruction metrics evaluation parameter, or updates the metrics evaluation parameter meter
Key point needed for calculating.
The embodiment of the present invention is by determining whether that updating key point determines in conjunction with distance correlation and inter-class correlation as a result,
Whether justice, the consistency between metrics evaluation parameter and metrics evaluation parameter are reasonable, are conducive to preferably to improve final determining
To the key point reasonability and reliability of the definition and availability and metrics evaluation parameter that define, and then preferably mention
The forecasting accuracy and reliability for the model that high final training obtains, are the development efficiency and model quality of deep learning project
Raising established solid foundation stone.
It in another embodiment, is the intuitive of the resultful inter-class correlation of each metrics evaluation parameter of raising, also
The scatter plot of the corresponding inter-class correlation of each metrics evaluation parameter can be generated, as shown in figure 3, Fig. 3 is that the present invention shows according to one
Example property implements a kind of scatter plot of the inter-class correlation exemplified, and Fig. 3 is with specified based on what is be marked in human body side image
Several key points 7 metrics evaluation parameters being calculated of coordinate corresponding to inter-class correlation be that the one kind exemplified dissipates
The size of inter-class correlation can be divided into 4 according to the degree of strength of the relevance between result as can be seen from Figure 3 by point diagram
Rank, for indicating the degree of strength of the relevance between result.The corresponding codomain of first level be [0.00,0.25), the
The corresponding codomain of two ranks be [0.25,0.50), the corresponding codomain of third level be [0.50,0.75), fourth level is corresponding
Codomain is [0.75,1].Wherein, if inter-class correlation belongs to first level, then it represents that the institute of its corresponding metrics evaluation parameter
Have very weak without association or relevance between result；If inter-class correlation belongs to second level, then it represents that its corresponding index
There is certain association between all results of evaluation parameter, but relevance is weaker；If inter-class correlation belongs to third level,
Indicate that the relevance between all results of its corresponding metrics evaluation parameter is medium；If inter-class correlation belongs to the fourth stage
Not, then it represents that the relevance between all results of its corresponding metrics evaluation parameter is preferable or relevance is strong.
Also, the inter-class correlation of each metrics evaluation parameter shown from Fig. 3 it is found that " index 7 " inter-class correlation
It is 0.389, belongs to second level, then can directly knows the pass between all results of " index 7 " corresponding metrics evaluation parameter
Connection property is weaker, in such event, can go to update key point definition according to above-mentioned related record, or choose new key point, or delete
Except " index 7 " and export the prompt information for being used to indicate " index 7 " and not being suitable for evaluation measurand.
It should be noted that, although the side of being provided for the embodiments of the invention by taking human body posture Detection task as an example among the above
Method is illustrated, but does not indicate that method provided by the embodiment of the present invention can be only applied in human body posture Detection task, this hair
Method provided by bright embodiment can also be applied in other critical point detection tasks other than human body posture Detection task, example
Such as, the coordinate, and/or metrics evaluation parameter that are related to key point are the Detection tasks that the coordinate based on key point is calculated.
Corresponding with the aforementioned prediction method of key point of measurand, the present invention also provides a kind of prediction measurands
The device of the device of key point, the key point of the prediction measurand can be applied in terminal, also can be applied to service
In device.As shown in figure 4, Fig. 4 is a kind of present invention key point for predicting measurand shown according to an exemplary embodiment
The device 200 of key point of the structural block diagram of device, the prediction measurand includes:
Module 201 is obtained, for obtaining the multiple mark set for defining mark based on key point, wherein the mark collection
It shares in recording one of mark personnel to the coordinate parameters of each key point mark of measurand；
Computing module 202, for calculating the correlation between the coordinate parameters of key point according to the mark set of acquisition；
Determining module 203 updates the key point definition for determining whether according to the correlation being calculated.
In one embodiment, under the premise of corresponding two marks of each measurand are gathered, the correlation include away from
When from correlation, the computing module 202 includes:
First computing unit, for marking the seat in gathering at two according to each key point for each measurand
Parameter is marked, the distance of each key point is calculated；
Second computing unit, for calculating and defining phase based on the distance for defining identical key point in all measurands
The distance between the same distance of key point correlation.
In one embodiment, it is based on a upper embodiment, the key point defines the ginseng of the horizontal coordinate including the key point
The definition of several definition and/or vertical coordinate parameter；Described device 200 further include:
First update module, for when the determining module 203 determines that updating the key point defines, according to it is described away from
The definition of definition and/or vertical coordinate parameter from the horizontal coordinate parameter that correlation updates the key point.
In one embodiment, under the premise of corresponding two marks of each measurand are gathered, the correlation include away from
When from correlation and inter-class correlation, the computing module 202 other than including the first computing unit and the second computing unit,
Further include:
Third computing unit, for each mark set for each measurand, according to specified several key points
Coordinate parameters corresponding metrics evaluation parameter is calculated；
4th computing unit, for the metrics evaluation based on the metrics evaluation gain of parameter different labeled personnel being calculated
Inter-class correlation between parameter.
In one embodiment, it is based on a upper embodiment, the key point defines the ginseng of the horizontal coordinate including the key point
The definition of several definition and/or vertical coordinate parameter；Described device 200 further include:
Second update module, for not updated from specified several key points in the determining module 203 determination
When the definition of acquired key point, output is used to indicate the metrics evaluation parameter and is not suitable for evaluating the prompt of measurand
Information, or update the key point needed for the metrics evaluation parameter calculates.
In one embodiment, the measurand includes the human body image of unspecified angle shooting.
The realization process of the function and effect of modules and unit is specifically detailed in right in the above method in above-mentioned apparatus 200
The realization process of step is answered, details are not described herein.
For device embodiment, since it corresponds essentially to embodiment of the method, so related place is referring to method reality
Apply the part explanation of example.The apparatus embodiments described above are merely exemplary, wherein described be used as separation unit
The unit of explanation may or may not be physically separated, and component shown as a unit can be or can also be with
It is not physical unit.
Corresponding with the aforementioned prediction method of key point of measurand, the present invention also provides a kind of prediction measurands
The electronic equipment of the device of key point, the electronic equipment may include:
Processor；
Memory, for storing the computer program that can be executed by the processor；
Wherein, the prediction measurand in aforementioned either method embodiment is realized when the processor executes described program
The step of method of key point.
The embodiment of the device of the key point of prediction measurand provided by the embodiment of the present invention can be applied described
On electronic equipment.It taking software implementation as an example, is the processing by electronic equipment where it as the device on a logical meaning
Computer program instructions corresponding in nonvolatile memory are read into memory what operation was formed by device.From hardware view
Speech, as shown in figure 5, Fig. 5 is the hardware structure diagram of present invention a kind of electronic equipment shown according to an exemplary embodiment, in addition to
Except processor shown in fig. 5, memory, network interface and nonvolatile memory, the electronic equipment can also include real
Other hardware of the method for the key point of existing aforementioned prediction measurand, such as photographing module；Or generally according to the electronic equipment
Actual functional capability can also include other hardware, repeat no more to this.
Corresponding with preceding method embodiment, the embodiment of the present invention also provides a kind of machine readable storage medium, stores thereon
There is program, the key point of the prediction measurand in aforementioned either method embodiment is realized when described program is executed by processor
The step of method.
It includes storage medium (the including but not limited to magnetic of program code that the embodiment of the present invention, which can be used in one or more,
Disk storage, CD-ROM, optical memory etc.) on the form of computer program product implemented.The machine readable storage is situated between
Matter may include: removable or non-removable media permanently or non-permanently.The information of the machine readable storage medium
Store function can be realized by any method or technique that may be implemented.The information can be computer-readable instruction, data
Structure, the model of program or other data.
In addition, the machine readable storage medium includes but is not limited to: phase change memory (PRAM), static random access memory
Device (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory
(ROM), the memory body of electrically erasable programmable read-only memory (EEPROM), flash memory or other memory techniques, read-only
Compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical memory, magnetic tape cassette, tape magnetic
Disk storage or other magnetic storage devices or the other non-transmission mediums that can be used for storing the information that can be accessed by a computing device.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to of the invention its
Its embodiment.The present invention is directed to cover any variations, uses, or adaptations of the invention, these modifications, purposes or
Person's adaptive change follows general principle of the invention and including the undocumented common knowledge in the art of the present invention
Or conventional techniques.The description and examples are only to be considered as illustrative, and true scope and spirit of the invention are by following
Claim is pointed out.
It should be understood that the present invention is not limited to the precise structure already described above and shown in the accompanying drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present invention is limited only by the attached claims.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the present invention.
Claims (10)
1. a kind of method for the key point for predicting measurand, which is characterized in that the described method includes:
Obtain multiple mark set that mark is defined based on key point, wherein the mark set is for recording one of mark
Coordinate parameters of the note personnel to each key point mark of a measurand；
The correlation between the coordinate parameters of key point is calculated according to the mark set of acquisition；
Determined whether to update the key point definition according to the correlation being calculated.
2. gathering the method according to claim 1, wherein each measurand corresponds to two marks, described
The correlation between the coordinate parameters of key point is calculated according to the mark set of acquisition, comprising:
Each key point is calculated according to coordinate parameters of each key point in two mark set for each measurand
Distance；
Based on the distance for defining identical key point in all measurands, calculate between the distance for defining identical key point
Distance correlation.
3. according to the method described in claim 2, it is characterized in that, the key point defines the horizontal seat including the key point
Mark the definition of parameter and/or the definition of vertical coordinate parameter；When determining that updating the key point defines, the method also includes:
The definition of horizontal coordinate parameter and/or determining for vertical coordinate parameter of the key point are updated according to the distance correlation
Justice.
4. according to the method described in claim 2, it is characterized in that, the seat for calculating key point according to the mark set of acquisition
Mark the correlation between parameter, further includes:
For each mark set of each measurand, correspondence is calculated according to the coordinate parameters of specified several key points
Metrics evaluation parameter；
Inter-class correlation between metrics evaluation parameter based on the metrics evaluation gain of parameter different labeled personnel being calculated.
5. according to the method described in claim 4, it is characterized in that, the correlation that the basis is calculated determines whether to update
The key point definition, comprising:
The key point that the inter-class correlation is less than preset threshold is obtained from specified several key points；
Determined whether to update key point definition according to the corresponding distance correlation of each key point of acquisition.
6. according to the method described in claim 5, it is characterized in that, the method also includes:
When determining the definition for not updating acquired key point from specified several key points, output is used to indicate described
Metrics evaluation parameter is not suitable for evaluating the prompt information of measurand, or updates the pass needed for the metrics evaluation parameter calculates
Key point.
7. the method according to claim 1, wherein the measurand includes the human figure of unspecified angle shooting
Picture.
8. a kind of device for the key point for predicting measurand characterized by comprising
Module is obtained, for obtaining the multiple mark set for defining mark based on key point, wherein the mark set is for remembering
One of mark personnel are recorded to the coordinate parameters of each key point mark of a measurand；
Computing module, for calculating the correlation between the coordinate parameters of key point according to the mark set of acquisition；
Determining module updates the key point definition for determining whether according to the correlation being calculated.
9. a kind of electronic equipment characterized by comprising
Processor；
Memory, for storing the computer program that can be executed by the processor；
Wherein, the step of any one of claim 1~7 the method is realized when the processor executes described program.
10. a kind of machine readable storage medium, is stored thereon with computer program；It is characterized in that, described program is by processor
The step of any one of claim 1~7 the method is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910399587.0A CN110210526A (en) | 2019-05-14 | 2019-05-14 | Predict method, apparatus, equipment and the storage medium of the key point of measurand |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910399587.0A CN110210526A (en) | 2019-05-14 | 2019-05-14 | Predict method, apparatus, equipment and the storage medium of the key point of measurand |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110210526A true CN110210526A (en) | 2019-09-06 |
Family
ID=67785877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910399587.0A Pending CN110210526A (en) | 2019-05-14 | 2019-05-14 | Predict method, apparatus, equipment and the storage medium of the key point of measurand |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110210526A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733767A (en) * | 2021-01-15 | 2021-04-30 | 西安电子科技大学 | Human body key point detection method and device, storage medium and terminal equipment |
| CN117609748A (en) * | 2024-01-17 | 2024-02-27 | 沈阳知友网络科技有限公司 | System and method for evaluating reasoning effect of medical image key point detection model |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103390282A (en) * | 2013-07-30 | 2013-11-13 | 百度在线网络技术（北京）有限公司 | Image tagging method and device |
| CN105404896A (en) * | 2015-11-03 | 2016-03-16 | 北京旷视科技有限公司 | Annotation data processing method and annotation data processing system |
| CN108073914A (en) * | 2018-01-10 | 2018-05-25 | 成都品果科技有限公司 | A kind of animal face key point mask method |
| CN108760266A (en) * | 2018-05-31 | 2018-11-06 | 西安交通大学 | The virtual degeneration index building method of mechanical key component based on learning distance metric |
| CN108876858A (en) * | 2018-07-06 | 2018-11-23 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN108876934A (en) * | 2017-12-20 | 2018-11-23 | 北京旷视科技有限公司 | Key point mask method, device and system and storage medium |
| CN108986169A (en) * | 2018-07-06 | 2018-12-11 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN109034095A (en) * | 2018-08-10 | 2018-12-18 | 杭州登虹科技有限公司 | A kind of face alignment detection method, apparatus and storage medium |
| CN109214343A (en) * | 2018-09-14 | 2019-01-15 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating face critical point detection model |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
-
2019
- 2019-05-14 CN CN201910399587.0A patent/CN110210526A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103390282A (en) * | 2013-07-30 | 2013-11-13 | 百度在线网络技术（北京）有限公司 | Image tagging method and device |
| CN105404896A (en) * | 2015-11-03 | 2016-03-16 | 北京旷视科技有限公司 | Annotation data processing method and annotation data processing system |
| CN108876934A (en) * | 2017-12-20 | 2018-11-23 | 北京旷视科技有限公司 | Key point mask method, device and system and storage medium |
| CN108073914A (en) * | 2018-01-10 | 2018-05-25 | 成都品果科技有限公司 | A kind of animal face key point mask method |
| CN108760266A (en) * | 2018-05-31 | 2018-11-06 | 西安交通大学 | The virtual degeneration index building method of mechanical key component based on learning distance metric |
| CN108876858A (en) * | 2018-07-06 | 2018-11-23 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN108986169A (en) * | 2018-07-06 | 2018-12-11 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN109034095A (en) * | 2018-08-10 | 2018-12-18 | 杭州登虹科技有限公司 | A kind of face alignment detection method, apparatus and storage medium |
| CN109214343A (en) * | 2018-09-14 | 2019-01-15 | 北京字节跳动网络技术有限公司 | Method and apparatus for generating face critical point detection model |
| CN109508681A (en) * | 2018-11-20 | 2019-03-22 | 北京京东尚科信息技术有限公司 | The method and apparatus for generating human body critical point detection model |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733767A (en) * | 2021-01-15 | 2021-04-30 | 西安电子科技大学 | Human body key point detection method and device, storage medium and terminal equipment |
| CN117609748A (en) * | 2024-01-17 | 2024-02-27 | 沈阳知友网络科技有限公司 | System and method for evaluating reasoning effect of medical image key point detection model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1871622B (en) | Image collation system and image collation method | |
| CN110188633A (en) | Human body posture index prediction technique, device, electronic equipment and storage medium | |
| CN110188769A (en) | Checking method, device, equipment and the storage medium of key point mark | |
| US9392153B2 (en) | Plenoptic camera resolution | |
| CN110175544B (en) | Target model construction method and device, electronic equipment and storage medium | |
| CN109934847A (en) | The method and apparatus of weak texture three-dimension object Attitude estimation | |
| CN107563995A (en) | A kind of confrontation network method of more arbiter error-duration models | |
| CN108830271A (en) | A kind of digital displaying meter Recognition of Reading method based on convolutional neural networks | |
| CN104408760A (en) | Binocular-vision-based high-precision virtual assembling system algorithm | |
| CN110210526A (en) | Predict method, apparatus, equipment and the storage medium of the key point of measurand | |
| CN109493389B (en) | Camera calibration method and system based on deep learning | |
| CN108961308A (en) | A kind of residual error depth characteristic method for tracking target of drift detection | |
| Muñoz‐Muñoz et al. | Photogrammetry: a useful tool for three‐dimensional morphometric analysis of small mammals | |
| CN110188634B (en) | Human body posture model construction method and device, electronic equipment and storage medium | |
| CN103093237B (en) | A kind of method for detecting human face of structure based model | |
| CN110211670B (en) | Index prediction method, index prediction device, electronic equipment and storage medium | |
| CN107976804A (en) | A kind of design method of lens optical system, device, equipment and storage medium | |
| CN109886337A (en) | Based on adaptively sampled depth measure learning method and system | |
| CN110414101B (en) | Simulation scene measurement method, accuracy measurement method and system | |
| CN109506617A (en) | Sensing data processing method, storage medium, electronic equipment | |
| CN109856946B (en) | Method for manufacturing laser holographic projection sand table | |
| CN111652244A (en) | Pointer type meter identification method based on unsupervised feature extraction and matching | |
| CN107295214B (en) | Interpolated frame localization method and device | |
| CN110619665A (en) | Calibration and verification method of binocular camera in virtual simulation environment | |
| CN109561384A (en) | Wireless sensor network node positioning method under composite noise condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |