CN110147701A - Key point mask method, device, computer equipment and storage medium - Google Patents
Key point mask method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110147701A CN110147701A CN201810678758.9A CN201810678758A CN110147701A CN 110147701 A CN110147701 A CN 110147701A CN 201810678758 A CN201810678758 A CN 201810678758A CN 110147701 A CN110147701 A CN 110147701A
- Authority
- CN
- China
- Prior art keywords
- key point
- samples pictures
- mark
- instruction
- marked
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of key point mask method, device, computer equipment and storage mediums, belong to human-computer interaction technique field.The method passes through in annotation process, while showing samples pictures to be marked, the display of mark reference picture is also provided, so that labeler being capable of position based on each key point in mark reference picture and number information etc., to instruct its annotation process, labeler is assisted to judge the correctness of mark, the accuracy and authenticity of mark can be effectively improved, to improve the recognition accuracy of training pattern.
Description
Technical field
The present invention relates to human-computer interaction technique field, in particular to a kind of key point mask method, device, computer equipment
And storage medium.
Background technique
Gesture identification is the critically important technology of one of human-computer interaction, and user can trigger calculating by gesture operation
Machine equipment executes certain functions.In order to identify the gesture operation of user, then depth of the training for being identified to gesture is needed
Learning model is spent, needs a large amount of training data in training, which, which can be, marks gesture key point
Therefore how picture after note it is most important carries out key point mark to picture.
Currently, main with movement simulation engine knowledge and corresponding programming technology, simulation is not when carrying out key point mark
Movement of the same personage in different scenes, can directly obtain the position of personage's gesture key point in different scenes, and then marking
After note, multiple key points can be attached to obtain key point lines, indicate gesture skeleton with lines.
When due to using computer technology to simulate personage, institute simulated person object gesture has certain with true gesture motion
Difference, and poor based on this authenticity, the accuracy of the gesture key point that are labeled simulated, also result in it is trained
The recognition accuracy of the model arrived also can be relatively low.
Summary of the invention
The embodiment of the invention provides a kind of key point mask method, device, computer equipment and storage mediums, can solve
Certainly authenticity and accuracy problem of the relevant technologies when key point marks, so that the identification for improving training pattern is accurate
Degree.The technical solution is as follows:
On the one hand, a kind of key point mask method is provided, which comprises
Display mark reference picture and samples pictures, the mark reference picture include multiple first key points and each
The number information of first key point, the number information of each first key point are used to indicate the mark for carrying out key point mark
Sequentially；
According to multiple trigger actions in the samples pictures, multiple second key points are marked in the samples pictures；
Based on multiple second key points marked in the samples pictures, the annotation results text of the samples pictures is obtained
Part.
On the one hand, a kind of key point annotation equipment is provided, described device includes:
Display module, for showing mark reference picture and samples pictures, the mark reference picture includes multiple first
The number information of key point and each first key point, the number information of each first key point, which is used to indicate, is closed
The mark sequence of key point mark；
Labeling module, for being marked in the samples pictures more according to multiple trigger actions in the samples pictures
A second key point；
Memory module, for obtaining the sample graph based on multiple second key points marked in the samples pictures
The annotation results file of piece.
On the one hand, a kind of computer equipment is provided, the computer equipment includes processor and memory, the storage
At least one instruction is stored in device, described instruction is loaded by the processor and executed to realize such as above-mentioned key point mark side
Operation performed by method.
On the one hand, a kind of computer readable storage medium is provided, at least one instruction is stored in the storage medium,
Described instruction is loaded as processor and is executed to realize the operation as performed by above-mentioned key point mask method.
Technical solution provided in an embodiment of the present invention has the benefit that
By while showing samples pictures to be marked, also providing the aobvious of mark reference picture in annotation process
Show, so that labeler can be based on the position of each key point and number information etc. in reference picture be marked, to instruct
Its annotation process assists labeler to judge the correctness of mark, can effectively improve the accuracy and authenticity of mark,
To improve the recognition accuracy of training pattern.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is the implement scene figure of key point mask method provided in an embodiment of the present invention；
Fig. 2 is a kind of flow chart of key point mask method provided in an embodiment of the present invention；
Fig. 3 is a kind of schematic diagram for marking interface provided in an embodiment of the present invention；
Fig. 4 is the schematic diagram at another mark interface provided in an embodiment of the present invention；
Fig. 5 is a kind of schematic diagram of encrypted characters string provided in an embodiment of the present invention；
Fig. 6 is the displaying surface chart that one kind provided in an embodiment of the present invention has marked picture；
Fig. 7 be a samples pictures to be marked provided in an embodiment of the present invention from import application program to confirmation mark at
The schematic diagram of the process of function；
Fig. 8 is a kind of structural schematic diagram of key point annotation equipment provided in an embodiment of the present invention；
Fig. 9 is a kind of a kind of structural schematic diagram of implementation of key point annotation equipment provided in an embodiment of the present invention；
Figure 10 is a kind of structural schematic diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, below in conjunction with attached drawing to embodiment party of the present invention
Formula is described in further detail.
Fig. 1 is the implement scene figure of key point mask method provided in an embodiment of the present invention.It include extremely in the implement scene
A few computer equipment 101 and at least one server 102, which may include being used for
The computer equipment of key point mark is carried out, can also include the computer equipment etc. of acquisition images of gestures, the embodiment of the present invention
It does not limit this, which may include the server for training gesture identification model, can also wrap
Trained gesture can also be configured with certainly for the server of application gesture identification model by including in above-mentioned computer equipment
Identification model, to realize gesture identification function.Wired or wireless net can be passed through between computer equipment 101 and server 102
Network is communicated, and carries out model training so that the picture marked is sent to server by computer equipment 101.
Fig. 2 is a kind of flow chart of key point mask method provided in an embodiment of the present invention.Referring to fig. 2, which has
Body includes:
201, computer equipment display mark reference picture and samples pictures, the mark reference picture includes multiple first
The number information of key point and each first key point, the number information of each first key point, which is used to indicate, is closed
The mark sequence of key point mark.
It can be equipped in computer equipment for realizing the application program of key point marking Function, be answered when user triggers this
When with the operating instruction of program, computer equipment can show mark reference picture and to be marked in response to the operating instruction
Samples pictures.
In a kind of possible implementation, which can provide a variety of different types of key point marking Functions,
For example, gesture key point marking Function, face key point marking Function, skeleton key point marking Function and other are any
Kind picture marking Function, correspondingly, computer equipment can show the initial interface of application program in response to the operating instruction,
It include multiple function choosing-items in the initial interface, each function choosing-item corresponds to a kind of picture marking Function.User can basis
The mark demand of itself carries out trigger action to any function choosing-item, when computer equipment detects the touching to the function choosing-item
When hair operation, the corresponding mark reference picture of the function choosing-item is shown.For example, with reference to Fig. 3, when computer equipment detects opponent
When the trigger action of the corresponding function choosing-item of gesture key point marking Function, the corresponding mark of display gesture marking Function is with reference to figure
Piece.
The number information of the first key point is indicated in mark reference picture with numeric form, and therefore, numerical values recited is suitable
Sequence can indicate a kind of operation order, can achieve a kind of intuitive prompt effect, user is when carrying out trigger action, it is only necessary to
It is labeled according to the sequence of the first key point in mark reference picture numbered from small to large.
It wherein, may include more on object and object in the mark reference picture when display marks reference picture
A key point also shows the number information of the key point in each key point, still with gesture key point shown in Fig. 3 mark
For function, shown in Fig. 3 is multiple first key points on manpower and manpower, and it is each that multiple first key point has marked hand
The position of key point, for example, wrist, thumbtip, the first interphalangeal joint of thumb, thumb metacarpophalangeal joints, great thenar center, index finger, in
21 key points such as finger, the third finger, the finger tip of little finger of toe, the first interphalangeal joint, the second interphalangeal joint and metacarpophalangeal joints, it is multiple
Number information, such as number 0-20 shown in Fig. 3 are labeled in first key point, and, the portion in multiple first key points
Dividing has line between the first key point, to indicate the default connection relationship between the first key point.Certainly, for gesture key
For point mark, different lines are formed by between the first key point of the mark reference picture can also use different colors
It shows, to distinguish the different branches of manpower skeleton.
In embodiments of the present invention, which can be directed into the application program by user's batch, should
Samples pictures to be marked can be by importing in external storage equipment, can also be from any storage region in the computer equipment
Samples pictures can be stored file corresponding to application program when samples pictures are directed into application program by interior importing
In.After importing successfully, computer equipment can show any samples pictures in the samples pictures that the batch imports.It is showing
When, mark reference picture (the gesture picture in such as Fig. 3 in left figure) can be shown in by interface using the display mode such as Fig. 3
Left side, samples pictures to be marked are shown on the right side at interface, certainly, according to the use habit of user, exchange picture
Display area shows samples pictures to be marked in the left side at interface so that mark reference picture is shown in the right side at interface,
It that is to say, this method further include: when receiving position and exchanging instruction, reference picture and samples pictures will be marked in interface
Display position is exchanged, and by the exchange function, the interface display mode for being more in line with use habit can be provided for user.
Wherein, which can be to be marked true picture, such as manpower picture of object etc., therefore, to substantially increase mark
The authenticity of note.
It certainly, can be in display mark reference picture and figure to be marked in order to enable user knows how to be labeled
When piece, display operation prompt information, the operation prompt information is for indicate how according to pass shown in mark reference picture
Key point is labeled picture to be marked, to achieve the purpose that user being capable of normal use.For example, the operation prompt information
It can be " please successively clicking according to the number order in left figure has corresponding position in figure ".
202, computer equipment is when detecting trigger action in the samples pictures, according to the second key point marked,
Determine the number information of corresponding second key point of the trigger action.
Wherein, trigger action can be is carried out based on input equipment, for example, the trigger action can be by user to mouse
Triggering realize, certainly, for touch screen, which can also be touch operation, that is to say, user can pass through
Finger realizes trigger action to the touch of screen.
Wherein, the second key point marked refers to that having been based on trigger action in the samples pictures has determined coordinate and volume
Second key point of number information.The second key point that the basis has marked determines corresponding second key point of the trigger action
Number information: 1 is added to the maximum number information in second key point marked, it is corresponding to obtain the trigger action
The number information of second key point.
For the first time trigger action in the samples pictures, due to before this, in the samples pictures also not into
Any mark of row, therefore, for first time trigger action, what is marked is the second key point that number information is 1.And
For any secondary trigger action later for first time trigger action, what is marked should be previous marked
At key point next key point, that is to say, the number information of the key point of the new mark should be the key that has marked
A number information in point after maximum number information, for example, the trigger action before this trigger action has marked
At the number information of key point be 10, then it is key point that number information is 11 that this trigger action, which is marked,.
203, the contact point coordinate of the trigger action is determined as second key point in the sample by computer equipment
Coordinate on this picture shows the number letter of second key point and second key point on the trigger point coordinate
Breath.
In embodiments of the present invention, trigger action can correspond to a contact point coordinate in the samples pictures, for mouse
For mark operation, which refers to click coordinate of the cursor of mouse in the samples pictures, for touch operation
For, which refers to that finger touches the touch point coordinate in the samples pictures.Wherein, the identified coordinate
It can indicate are as follows: the key point that number information is 1: (X1, Y1), so that the number information of key point and coordinate can correspond to.
When being labeled, user can be according to the position of multiple first key points in mark reference picture, to be marked
Samples pictures on corresponding position on carry out trigger action, computer equipment is according to the trigger action detected and has marked
The second key point, it is corresponding second crucial that next key point of the second key point marked is determined as the trigger action
The contact point coordinate of the trigger action is determined as the coordinate of second key point in samples pictures by point, and computer equipment exists
The coordinate corresponding position of the second key point of this in samples pictures shows second key point and the number letter of second key point
Breath.
It should be noted that in above process, second key point is often marked by trigger action, then it can be in sample
Show that second key point marked allows user to check mark in order to provide intuitive visual effect on this picture
It is whether correct, to be adjusted in time.
Above-mentioned steps 202 to 203 are computer equipments according to multiple trigger actions in the samples pictures, in the sample
The process of multiple second key points is marked on this picture, is only that how pass is specifically marked to a trigger action in this process
It is illustrated for key point, in practical applications, needs to be performed both by the above process to each trigger action detected, with complete
The mark of key point in paired samples picture.
A kind of schematic diagram for marking interface is provided for example, with reference to Fig. 4, Fig. 4.In gesture key point annotation process, use
Family can be according to the skeleton joint position of multiple first key points in mark reference picture, phase in samples pictures to be marked
It answers and carries out trigger action on skeleton joint position, computer equipment is according to the trigger action detected and the second pass marked
Next key point of the second key point marked is determined as corresponding second key point of the trigger action, by this by key point
The trigger point coordinate of trigger action is determined as the coordinate of second key point on picture, and computer equipment is shown on interface should
Second key point.It is shown in the original figure layer that second key point can be covered in the samples pictures in a manner of floating layer
Show.In addition, second key point can be indicated using colored dot, such as green etc..
204, the computer equipment is according to the default connection relationship between multiple second key point, by default connection relationship
The second indicated key point line and curve connection obtains at least one key point lines, shows at least in the samples pictures
One key point line.
It is to have default connection relationship for the second key point with different number informations, between number information
, it that is to say that there is default connection relationship between multiple second key point, which key is the default connection relationship for embodying
Point belongs to the same skeleton branches.It therefore, can be based on default when multiple second key points to samples pictures, which mark, to be completed
Connection relationship gets up multiple second key point line and curve connections.Certainly, the connection procedure can be in annotation process into
Row, one the second key point of every mark then close second key point with having second marked of default connection relationship
Key point line and curve connection, and be shown in samples pictures.Optionally, the key point line being in the same branch of skeleton can be adopted
It being indicated with same color, the key point line being in skeleton difference branch can be shown using different colors, so as to
Distinguish the different branches of skeleton.
It can be known by above description, the display of above-mentioned annotation process and key point line can carry out simultaneously, namely
It is that, when user carries out trigger action to the picture, computer equipment shows corresponding second key point of the trigger action,
If there is default connection relationship, computer equipment with the second key point marked in corresponding second key point of the trigger action
They are connected, key point lines are obtained, while showing the key point lines.
205, when receiving mark completion instruction, which is based on multiple second key point on the picture
Coordinate encrypted, obtain encrypted characters string, which stored to the annotation results file of the samples pictures.
Computer equipment uses preset-key and Encryption Algorithm, the coordinate of each second key point that mark is obtained into
Row encryption, obtains encrypted characters string, to guarantee the safety of labeled data.The Encryption Algorithm can be des encryption algorithm, when
So, other Encryption Algorithm can also be used, it is not limited in the embodiment of the present invention.
When user completes mark to a samples pictures, the annotation results of the samples pictures can be stored to mark and be tied
In fruit file.The annotation results of every picture can store as an annotation results file, certainly, the annotation results of batch picture
Also it can store to an annotation results file.The sample is shown when user opens some annotation results file referring to Fig. 5
The encrypted characters string that the coordinate of picture obtains after encryption.
The embodiment of the present invention is illustrated for the coordinate marked is encrypted and stored, in some embodiments
In, when computer equipment, which receives mark, completes instruction, the ciphering process in above-mentioned steps 207 can not also be executed, and it is straight
The annotation results file that the coordinate that will have been marked is stored as samples pictures is connect, to be used for subsequent training process.
206, when computer equipment, which receives picture, checks instruction, to adding in the annotation results file of the samples pictures
Close character string is decrypted, and based on the data obtained after decryption, multiple the second of the samples pictures are shown in the samples pictures
Key point, the number information of each second key point and the lines connected by multiple second key point.
Computer equipment can also provide picture look facility, so that user confirms the correctness of annotation results.Work as user
When the picture triggered in application program checks instruction, computer equipment receives the picture and checks instruction and refer in response to this
It enables, shows the picture annotation results in annotation results file.Referring to Fig. 6, shown in fig. 6 is that multiple the second of the samples pictures are closed
Key point, number information and the lines connected by multiple second key points.
Optionally, when coordinate is stored after encryption, then computer equipment can be based on the picture annotation results file
In encrypted characters string, be decrypted using key that is identical as step 205 or matching, the data after being decrypted.
It should be noted that during this is checked, can the display based on picture and key point, to confirm mark
It is whether correct, if incorrect, annotation results file can be deleted, and be labeled operation again, which can be significantly
The case where reducing mark inferior.
In practical annotation process, instruction can be checked by i.e. triggering picture after the completion of each samples pictures mark, come
It is labeled and whether correctly verifies, can also be after the completion of a collection of samples pictures mark, then trigger picture and check instruction, come one
The samples pictures that a displaying has marked are opened, carry out the verifying of correctness, it is not limited in the embodiment of the present invention.
Above-mentioned steps 201 to 206 be for a samples pictures from mark to data storage again to checking annotation results
The process that sequence is described, and in annotation process, a variety of different processing modes can also be provided, for example, to having marked
Some or whole key point deleted, be switched to next picture and continue to mark, exit annotation process etc., under
A variety of processing modes are introduced in face, point different situations:
(1) some key point marked is deleted.
For any key point marked, with that can be marked again to it per family, it that is to say, retract one
A key point is marked again, then needs to delete former mark, correspondingly, this method further include: when receiving the first instruction,
The number information of previous second key point and the previous gesture key point is deleted, first instruction, which is used to indicate, moves back
It returns a upper key point and re-starts mark.
In key point annotation process, as long as application program is not received by exit instruction, computer equipment can be obtained
Know that user does not complete to the trigger actions of all key points in the picture, can provide and modify to the key point marked
Processing mode.User can work as meter by the trigger action to the first specified button to trigger the first instruction of the application program
When calculation machine equipment receives the first instruction, the coordinate recorded in a upper trigger position and number information are deleted.Wherein, should
First specified button can be P key, it is, of course, also possible to be other preassigned keys, the embodiment of the present invention does not do this
It limits.
(2) the whole key points marked are deleted.Computer equipment can also provide a kind of efficiently deletion side
Formula is deleted with will pass through once-through operation for all key points, correspondingly, this method further include: when receiving the second instruction,
Delete the number information of all second key points and all second key points in the samples pictures, second instruction
For deleting the whole key points marked.
User can work as meter by the trigger action to the second specified button to trigger the second instruction of the application program
When calculation machine equipment receives the second instruction, all labels added in the samples pictures are deleted.Wherein, second specified button
It can be r key, it is, of course, also possible to be other preassigned keys, it is not limited in the embodiment of the present invention.
(3) next picture is switched to continue to mark.When any samples pictures mark is completed, computer equipment
Continuous annotation process can be provided, that is to say, it, can not if a mark is completed or the key point of current image is incomplete
It realizes mark etc., then can skip current samples pictures, and enter the mark of next samples pictures, correspondingly, this method is also
It include: when receiving third instruction, switching next samples pictures of display, the third instruction is for switching picture.User
It can be by the trigger action to third specified button, to trigger the third instruction of the application program, when computer equipment receives
To when third instruction, switching shows next picture to be marked.Wherein, which can be k key, certainly, also
It can be other preassigned keys, it is not limited in the embodiment of the present invention.
(4) annotation process is exited.When user wants to terminate mark, then annotation process, correspondingly, this method can be exited
Further include: it when receiving the 4th instruction, exits annotation process and records the samples pictures not marked, the 4th instruction is used for
Exit annotation process.For this is exited, when starting application program next time, then the sample graph not marked can be directly displayed
Piece.User can be by the trigger action to the 4th specified button, to trigger the 4th instruction of the application program, when computer is set
When for receiving the 4th instruction, annotation process is exited and records not marking picture.Wherein, the 4th specified button can press for q
Key, it is, of course, also possible to be other preassigned keys, it is not limited in the embodiment of the present invention.
(5) mark of current samples pictures is completed and enters the mark of next samples pictures.When the institute of the samples pictures
There is key point to mark completion, then can carry out the mark of next samples pictures, correspondingly, this method further include: when connecing
It when receiving the 5th instruction, saves the samples pictures marked and switches to next samples pictures, the 5th instruction is used
It has completed to mark in the samples pictures.User can be by the trigger action to the 5th specified button, to trigger the application
5th instruction of program, when computer equipment receives the 5th instruction, preservation has marked picture and has switched to next wait mark
Infuse picture.Wherein, the 5th specified button can be s key, it is, of course, also possible to be other preassigned keys, the present invention
Embodiment does not limit this.
And for checking for the process of the samples pictures marked, computer equipment can also provide a variety of different places
Reason mode is described as follows with adapting to the mark demand of user:
(6) it shows next and has marked picture.During any display for having marked picture, it can be grasped by switching
Make to switch next picture and be shown, correspondingly, this method further include: during display in the samples pictures, receive
To when the 6th instruction, next of switching display has marked picture, and the 6th instruction has marked picture for switching.User can lead to
The trigger action to the 6th specified button is crossed, to trigger the 6th instruction of the application program, when computer equipment receives the 6th
When instruction, next of switching display has marked picture.Wherein, which can be n key, it is, of course, also possible to be pre-
First other specified keys, it is not limited in the embodiment of the present invention.
(7) the annotation results file of samples pictures is deleted.Since the mark of key point may need to modify,
The deletion function to annotation results file, correspondingly, this method can also be provided further include: when the display in the samples pictures
In the process, when receiving the 7th instruction, the annotation results file of the picture is deleted, the 7th instruction is for deleting picture
Annotation results file.User can be instructed by the trigger action to the 7th specified button, the to trigger the application program the 7th,
When computer equipment receives the 7th instruction, the annotation results file of the picture is deleted.Wherein, which can be
D key, it is, of course, also possible to be other preassigned keys, it is not limited in the embodiment of the present invention.
It, can be with it should be noted that by step 202 and 203 annotation process and the confirmation of step 206 the two steps
The quantity for substantially reducing mark inferior, improves the quality of training set sample.In practical annotation process, some steps above are
Batch processing.Such as after a samples pictures in office mark successfully, in some embodiments, it is also possible to not immediately enter verifying mark
The step of correctness of note, but the mark of next samples pictures is first carried out, after a collection of samples pictures all mark, then
It is shown, so as to proving correctness.
As shown in fig. 7, Fig. 7 be a samples pictures to be marked provided in an embodiment of the present invention from import application program to
Confirmation marks the schematic diagram of successful process.Referring to Fig. 7, which can be executed based on OpenCV vision library, to create one kind
The mode that people and computer inter are moved, comprising:
1, start application program, display mark reference picture and samples pictures.
2, judge the whether existing annotation results file of the samples pictures, the samples pictures are skipped if having, if not having, are used
Family can be labeled by the trigger action to samples pictures, jump to step 3；
3, in the annotation process to samples pictures, if detecting the trigger action to s key, then it represents that mark is completed,
Jump to step 4；If detecting the trigger action to k key, then it represents that ignore mark, and jump to step 5；The expression of q key is pressed to exit
Program jumps to step 6；
4, it marks successfully, records annotation results, jump to step 7；
5, ignore the mark to the samples pictures, record annotation results, jump to step 7；
6, step 1 is jumped in EP (end of program)；
7, the stage is shown, if detecting the trigger action to d key, then it represents that annotation results need to be deleted, jump to step 2；
If detecting the trigger action to n key, then it represents that the mark that this has marked picture is qualified, can continue to check that next has been marked
Picture is infused, step 8 is jumped to；
8, determine that this picture marks successfully.
In the above process, the mark of a picture is successfully needed by multiple steps, wherein the mark and step 7 of step 3
Confirmation the two steps, the quantity of mark inferior can be substantially reduced, improve the quality of training set sample.
In practical annotation process, some steps above are batch processings.Such as after step 4 picture marks successfully,
In some embodiments, it is also possible to not immediately enter the correctness that step 7 goes verifying to mark, but first carry out next sample graph
The mark of piece after a collection of samples pictures all mark, then is shown, so as to proving correctness.
By while showing samples pictures to be marked, also providing the aobvious of mark reference picture in annotation process
Show, so that labeler can be based on the position of each key point and number information etc. in reference picture be marked, to instruct
Its annotation process assists labeler to judge the correctness of mark, can effectively improve the accuracy and authenticity of mark,
To improve the recognition accuracy of training pattern.
Further, when labeler is labeled key point, connection forms skeleton, energy between adjacent key point
Enough be conducive to assist labeler judgement mark correctness.Further, a variety of interactive keys are additionally provided to mark to realize
A variety of processing functions in journey and after marking successfully allow labeler to tell computer equipment to carry out by interaction key
Different disposal, for example, retract a step, restart or exit the program etc..Further, the result for marking generation can turn
It is melted into encrypted characters string then to save, to guarantee the safety of labeled data.After the completion of entire annotation process, by showing
Journey confirms the correctness of data again.The embodiment of the present invention proposes a kind of efficient gesture skeleton mask method as a result, can
Better training data is provided for depth learning technology.
Fig. 8 is a kind of structural schematic diagram of key point annotation equipment provided in an embodiment of the present invention, referring to Fig. 8, the dress
It sets and includes:
Display module 801, for showing mark reference picture and samples pictures, the mark reference picture includes multiple the
The number information of the number information of one key point and each first key point, each first key point is used to indicate progress
The mark sequence of key point mark；
Labeling module 802, for being marked in the samples pictures according to multiple trigger actions in the samples pictures
Multiple second key points；
Memory module 803, for obtaining the sample based on multiple second key points marked in the samples pictures
The annotation results file of picture.
In some embodiments, the labeling module 802, is used for:
When detecting trigger action in the samples pictures, according to the second key point marked, the touching is determined
Hair operates the number information of corresponding second key point；
By the contact point coordinate of the trigger action, it is determined as seat of second key point in the samples pictures
Mark, shows the number information of second key point and second key point on the trigger point coordinate.
In some embodiments, the labeling module 802, is used for:
When detecting trigger action in the samples pictures, the maximum in second key point marked is compiled
Number information adds 1, obtains the number information of corresponding second key point of the trigger action.
In some embodiments, the display module 801 is also used to showing that multiple second in the samples pictures close
When key point, at least one key point line is shown in the samples pictures.
In some embodiments, the display module 801 is also used to according to default between the multiple second key point
Second key point line and curve connection indicated by default connection relationship is obtained described at least one crucial dotted line by connection relationship
Item.
In some embodiments, which further includes the first command process module, is used for:
When receiving the first instruction, the number of previous second key point and previous second key point is deleted
Information, first instruction, which is used to indicate, retracts a key point and re-starts mark；Or,
When receiving the second instruction, all second key points and all second passes in the samples pictures are deleted
The number information of key point, second instruction is for deleting all the second key points marked.
In some embodiments, which further includes the second command process module, is used for:
When receiving third instruction, switching next samples pictures of display, the third instruction is for switching picture；
Or,
It when receiving the 4th instruction, exits annotation process and records the samples pictures not marked, the 4th instruction is used
In exiting annotation process；Or,
When receiving the 5th instruction, saves the samples pictures marked and switch to next samples pictures, institute
The 5th instruction is stated to have completed to mark for the samples pictures.
In some embodiments, the memory module 803 is used for the multiple second key point in the samples pictures
Coordinate encrypted, obtain encrypted characters string, by the encrypted characters string be stored as the samples pictures annotation results text
Part.
In some embodiments, described device further include: deciphering module is used for:
When receiving picture and checking instruction, the encrypted characters string in the annotation results file of the samples pictures is carried out
Decryption；
Based on the data obtained after decryption, show that multiple the second of the samples pictures are crucial in the samples pictures
Point.
In some embodiments, which is also used to when the display samples pictures and the samples pictures
On multiple second key points when, the lines that are connected by the multiple second key point are shown in the samples pictures.
In some embodiments, which further includes third command process module, is used for:
During the display of the samples pictures, when receiving the 6th instruction, what next of switching display had marked
Samples pictures, the 6th instruction have marked picture for switching；Or,
During the display of the samples pictures, when receiving the 7th instruction, the mark of the samples pictures is deleted
Destination file, the described 7th instructs the annotation results file for deleting the samples pictures.
All the above alternatives can form the alternative embodiment of the disclosure, herein no longer using any combination
It repeats one by one.
It should be understood that key point annotation equipment provided by the above embodiment is when key point marks, only with above-mentioned each
The division progress of functional module can according to need and for example, in practical application by above-mentioned function distribution by different function
Energy module is completed, i.e., the internal structure of equipment is divided into different functional modules, to complete whole described above or portion
Divide function.In addition, key point annotation equipment provided by the above embodiment and key point mask method embodiment belong to same design,
Its specific implementation process is detailed in embodiment of the method, and which is not described herein again.
By taking a kind of specific implementation of above-mentioned key point annotation equipment as an example, the apparatus may include following functions moulds
Block:
Constants block indicates the color of the connection relationship and connecting line between mark samples pictures, each key point；The constant
Module can be provided as the partial function in display module, and each point is particularly shown form when determining display.
The module of monitoring mouse action records the position pressed as mark position when pressing left mouse button；
The module of monitoring keyboard action once marks when labeler needs to return, marks from the beginning, skips picture, temporary
When logging off or saving annotation results, by the corresponding position on lower keyboard；
The module of above-mentioned detection operation can be provided as the partial function of labeling module, to supervise to mouse action
It listens, to be labeled, and keyboard action is monitored, to the processing such as modify to mark.
Memory module can be provided as the encryption store function of above-mentioned memory module, and annotation results are converted to character string lattice
Formula sets key des encryption algorithm for encryption, is then saved in destination file；
Deciphering module is solved when reading data from destination file with key pair result same as encrypting module
It is close；
Mark display module, in the entire period of mark, record currently marked as a result, and show with mark
On the graphic interface of note person interaction；
Annotation results display module after mark, data deciphering is read from destination file, is shown to graphic interface
On.
Above-mentioned mark display module and annotation results display module can be provided as the partial function of display module, use
To be shown to need information to be shown.
Certain independence is kept between the modules of this annotation tool, different tasks, but they are completed in the division of labor
Between again have certain relation of interdependence, the topological diagram of dependence such as Fig. 9 is shown.From the figure, it can be seen that marking
Journey needs constants block, monitors the cooperation between mouse action module, monitoring keyboard action module, and displaying process needs constant mould
Block, the cooperation for monitoring keyboard action module, since the coordinate of each key point is not it has been determined that therefore, this process needs to supervise
Survey the module of mouse action.
Figure 10 is a kind of structural schematic diagram of computer equipment provided in an embodiment of the present invention, which can
Bigger difference is generated because configuration or performance are different, may include one or more processors (central
Processing units, CPU) 1001 and one or more memory 1002, wherein it is deposited in the memory 1002
At least one instruction is contained, at least one instruction is loaded by the processor 1001 and executed to realize above-mentioned each method
The method that embodiment provides.Certainly, which can also have wired or wireless network interface, keyboard and input defeated
The components such as outgoing interface, to carry out input and output, which can also include other portions for realizing functions of the equipments
Part, this will not be repeated here.
In the exemplary embodiment, a kind of computer readable storage medium is additionally provided, the memory for example including instruction,
Above-metioned instruction can be executed by the processor in terminal to complete the key point mask method in following embodiments.For example, the meter
Calculation machine readable storage medium storing program for executing can be ROM, random access memory (RAM), CD-ROM, tape, floppy disk and optical data storage and set
It is standby etc..
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (15)
1. a kind of key point mask method, which is characterized in that the described method includes:
Display mark reference picture and samples pictures, the mark reference picture include multiple first key points and each first
The number information of key point, the mark that the number information of each first key point is used to indicate progress key point mark are suitable
Sequence；
According to multiple trigger actions in the samples pictures, multiple second key points are marked in the samples pictures；
Based on multiple second key points marked in the samples pictures, the annotation results file of the samples pictures is obtained.
2. the method according to claim 1, wherein multiple triggerings according in the samples pictures are grasped
Make, multiple second key points are marked in the samples pictures includes:
When detecting trigger action in the samples pictures, according to the second key point marked, the triggering behaviour is determined
Make the number information of corresponding second key point；
By the contact point coordinate of the trigger action, it is determined as coordinate of second key point in the samples pictures,
The number information of second key point and second key point is shown on the trigger point coordinate.
3. according to the method described in claim 2, it is characterized in that, described ought detect trigger action in the samples pictures
When, according to the second key point marked, determine that the number information of corresponding second key point of the trigger action includes:
When detecting trigger action in the samples pictures, to the maximum number letter in second key point marked
Breath plus 1, obtains the number information of corresponding second key point of the trigger action.
4. the method according to claim 1, wherein the method also includes:
When showing multiple second key points in the samples pictures, at least one key point is shown in the samples pictures
Line.
5. according to the method described in claim 4, it is characterized in that, the display at least one in the samples pictures is crucial
Putting line includes:
According to the default connection relationship between the multiple second key point, by the second key point indicated by default connection relationship
With line and curve connection, at least one key point lines are obtained.
6. the method according to claim 1, wherein the method also includes:
When receiving the first instruction, the number letter of previous second key point and previous second key point is deleted
Breath, first instruction, which is used to indicate, retracts a key point and re-starts mark；Or,
When receiving the second instruction, all second key points and all second key points in the samples pictures are deleted
Number information, it is described second instruction for deletes marked whole the second key points.
7. the method according to claim 1, wherein the method also includes:
When receiving third instruction, switching next samples pictures of display, the third instruction is for switching picture；Or,
It when receiving the 4th instruction, exits annotation process and records the samples pictures not marked, the 4th instruction is for moving back
Annotation process out；Or,
When receiving the 5th instruction, saves the samples pictures marked and simultaneously switch to next samples pictures, described the
Five instructions have been completed to mark for the samples pictures.
8. the method according to claim 1, wherein the method also includes:
Coordinate of the multiple second key point in the samples pictures is encrypted, encrypted characters string is obtained, it will be described
Encrypted characters string is stored as the annotation results file of the samples pictures.
9. according to the method described in claim 8, it is characterized in that, the method also includes:
When receiving picture and checking instruction, the encrypted characters string in the annotation results file of the samples pictures is solved
It is close, based on the data obtained after decryption, multiple second key points of the samples pictures are shown in the samples pictures.
10. according to the method described in claim 9, it is characterized in that, the method also includes:
When showing multiple second key points in the samples pictures and the samples pictures, shown in the samples pictures
Show the lines connected by the multiple second key point.
11. according to the method described in claim 10, it is characterized in that, the method also includes:
During the display of the samples pictures, when receiving the 6th instruction, the switching next sample marked of display
Picture, the 6th instruction have marked picture for switching；Or,
During the display of the samples pictures, when receiving the 7th instruction, the annotation results of the samples pictures are deleted
File, the described 7th instructs the annotation results file for deleting the samples pictures.
12. a kind of key point annotation equipment, which is characterized in that described device includes:
Display module, for showing that mark reference picture and samples pictures, the mark reference picture include multiple first crucial
The number information of point and each first key point, the number information of each first key point are used to indicate carry out key point
The mark sequence of mark；
Labeling module, for marking multiple in the samples pictures according to multiple trigger actions in the samples pictures
Two key points；
Memory module, for obtaining the samples pictures based on multiple second key points marked in the samples pictures
Annotation results file.
13. device according to claim 12, which is characterized in that the labeling module is used for:
When detecting trigger action in the samples pictures, according to the second key point marked, the triggering behaviour is determined
Make the number information of corresponding second key point；
By the contact point coordinate of the trigger action, it is determined as coordinate of second key point in the samples pictures,
The number information of second key point and second key point is shown on the trigger point coordinate.
14. a kind of computer equipment, which is characterized in that the computer equipment includes processor and memory, the memory
In be stored at least one instruction, described instruction is loaded by the processor and is executed to realize as claim 1 to right is wanted
Ask operation performed by 11 described in any item key point mask methods.
15. a kind of computer readable storage medium, which is characterized in that be stored at least one instruction, institute in the storage medium
Instruction is stated to be loaded by processor and executed to realize such as claim 1 to the described in any item key point mark sides of claim 11
Operation performed by method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810678758.9A CN110147701A (en) | 2018-06-27 | 2018-06-27 | Key point mask method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810678758.9A CN110147701A (en) | 2018-06-27 | 2018-06-27 | Key point mask method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110147701A true CN110147701A (en) | 2019-08-20 |
Family
ID=67589123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810678758.9A Pending CN110147701A (en) | 2018-06-27 | 2018-06-27 | Key point mask method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110147701A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259822A (en) * | 2020-01-19 | 2020-06-09 | 杭州微洱网络科技有限公司 | Method for detecting key point of special neck in E-commerce image |
| CN111626233A (en) * | 2020-05-29 | 2020-09-04 | 江苏云从曦和人工智能有限公司 | Key point marking method, system, machine readable medium and equipment |
| CN112508127A (en) * | 2020-12-22 | 2021-03-16 | 北京百度网讯科技有限公司 | Data labeling method and device, electronic equipment, medium and product |
| CN112528929A (en) * | 2020-12-22 | 2021-03-19 | 北京百度网讯科技有限公司 | Data labeling method and device, electronic equipment, medium and product |
| CN113011403A (en) * | 2021-04-30 | 2021-06-22 | 恒睿(重庆)人工智能技术研究院有限公司 | Gesture recognition method, system, medium, and device |
| CN113010069A (en) * | 2021-03-12 | 2021-06-22 | 浙江大华技术股份有限公司 | Switching method and device for picture labels, electronic device and storage medium |
| CN113407869A (en) * | 2021-06-02 | 2021-09-17 | 北京爱笔科技有限公司 | Beacon labeling method, device, computer equipment and storage medium |
| CN113539446A (en) * | 2020-04-21 | 2021-10-22 | 杭州普健医疗科技有限公司 | CT image labeling method and system, storage medium and terminal |
| US11361490B2 (en) | 2020-08-18 | 2022-06-14 | Here Global B.V. | Attention guidance for ground control labeling in street view imagery |
| WO2022142783A1 (en) * | 2020-12-29 | 2022-07-07 | 华为云计算技术有限公司 | Image processing method and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0607926A2 (en) * | 1993-01-18 | 1994-07-27 | Sharp Kabushiki Kaisha | Information processing apparatus with a gesture editing function |
| US20100290699A1 (en) * | 2009-05-15 | 2010-11-18 | Google Inc. | Landmarks from Digital Photo Collections |
| CN103390282A (en) * | 2013-07-30 | 2013-11-13 | 百度在线网络技术（北京）有限公司 | Image tagging method and device |
| CN105184283A (en) * | 2015-10-16 | 2015-12-23 | 天津中科智能识别产业技术研究院有限公司 | Method and system for marking key points in human face images |
| EP3107032A1 (en) * | 2015-06-17 | 2016-12-21 | Thomson Licensing | Method and device for facial landmark annotation and corresponding computer program product and computer-readable medium |
| US20180088794A1 (en) * | 2016-09-23 | 2018-03-29 | Apple Inc. | Devices, Methods, and Graphical User Interfaces for a Unified Annotation Layer for Annotating Content Displayed on a Device |
-
2018
- 2018-06-27 CN CN201810678758.9A patent/CN110147701A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0607926A2 (en) * | 1993-01-18 | 1994-07-27 | Sharp Kabushiki Kaisha | Information processing apparatus with a gesture editing function |
| US20100290699A1 (en) * | 2009-05-15 | 2010-11-18 | Google Inc. | Landmarks from Digital Photo Collections |
| CN103390282A (en) * | 2013-07-30 | 2013-11-13 | 百度在线网络技术（北京）有限公司 | Image tagging method and device |
| EP3107032A1 (en) * | 2015-06-17 | 2016-12-21 | Thomson Licensing | Method and device for facial landmark annotation and corresponding computer program product and computer-readable medium |
| CN105184283A (en) * | 2015-10-16 | 2015-12-23 | 天津中科智能识别产业技术研究院有限公司 | Method and system for marking key points in human face images |
| US20180088794A1 (en) * | 2016-09-23 | 2018-03-29 | Apple Inc. | Devices, Methods, and Graphical User Interfaces for a Unified Annotation Layer for Annotating Content Displayed on a Device |
Non-Patent Citations (1)
| Title |
|---|
| 胡涛;吴涛;李焱;: "一种基于场景识别的快速语义标注方法", 华中科技大学学报(自然科学版), no. 1, pages 103 - 107 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111259822A (en) * | 2020-01-19 | 2020-06-09 | 杭州微洱网络科技有限公司 | Method for detecting key point of special neck in E-commerce image |
| CN113539446A (en) * | 2020-04-21 | 2021-10-22 | 杭州普健医疗科技有限公司 | CT image labeling method and system, storage medium and terminal |
| CN111626233A (en) * | 2020-05-29 | 2020-09-04 | 江苏云从曦和人工智能有限公司 | Key point marking method, system, machine readable medium and equipment |
| CN111626233B (en) * | 2020-05-29 | 2021-07-13 | 江苏云从曦和人工智能有限公司 | Key point marking method, system, machine readable medium and equipment |
| US11361490B2 (en) | 2020-08-18 | 2022-06-14 | Here Global B.V. | Attention guidance for ground control labeling in street view imagery |
| CN112508127A (en) * | 2020-12-22 | 2021-03-16 | 北京百度网讯科技有限公司 | Data labeling method and device, electronic equipment, medium and product |
| CN112528929A (en) * | 2020-12-22 | 2021-03-19 | 北京百度网讯科技有限公司 | Data labeling method and device, electronic equipment, medium and product |
| WO2022142783A1 (en) * | 2020-12-29 | 2022-07-07 | 华为云计算技术有限公司 | Image processing method and related device |
| CN113010069A (en) * | 2021-03-12 | 2021-06-22 | 浙江大华技术股份有限公司 | Switching method and device for picture labels, electronic device and storage medium |
| CN113011403A (en) * | 2021-04-30 | 2021-06-22 | 恒睿(重庆)人工智能技术研究院有限公司 | Gesture recognition method, system, medium, and device |
| CN113011403B (en) * | 2021-04-30 | 2023-11-24 | 恒睿(重庆)人工智能技术研究院有限公司 | Gesture recognition method, system, medium and device |
| CN113407869A (en) * | 2021-06-02 | 2021-09-17 | 北京爱笔科技有限公司 | Beacon labeling method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147701A (en) | Key point mask method, device, computer equipment and storage medium | |
| CN102402369B (en) | Electronic equipment and its operation indicating mark moving method | |
| CN104335226B (en) | Use the fingerprint register of partial fingerprints | |
| CN107024981A (en) | Exchange method and device based on virtual reality | |
| CN104850332B (en) | Intelligent terminal control method and intelligent terminal | |
| CN101339453B (en) | Simulated mouse input method based on interactive input apparatus | |
| CN107648848A (en) | Information processing method and device, storage medium, electronic equipment | |
| CN107132979A (en) | Exchange method, device and the computer-readable recording medium of accurate selection target in mobile device game | |
| CN104991640B (en) | Programing system in kind and method on interactive interface | |
| CN109086095A (en) | The quick open method of application program, device, terminal and storage medium | |
| CN107194213A (en) | A kind of personal identification method and device | |
| CN109215441A (en) | A kind of Piano Teaching method, apparatus and computer storage medium | |
| CN104866097B (en) | The method of hand-held signal output apparatus and hand-held device output signal | |
| CN106911963A (en) | Method for information display and device in a kind of terminal device | |
| Roig-Maimó et al. | Head-tracking interfaces on mobile devices: Evaluation using Fitts’ law and a new multi-directional corner task for small displays | |
| CN107272881A (en) | A kind of data inputting method and device, entering method keyboard and electronic equipment | |
| CN105242776A (en) | Control method for intelligent glasses and intelligent glasses | |
| CN114327064A (en) | Plotting method, system, equipment and storage medium based on gesture control | |
| CN107688399A (en) | A kind of input method and device, a kind of device for being used to input | |
| CN109828672A (en) | It is a kind of for determining the method and apparatus of the human-machine interactive information of smart machine | |
| CN106201070A (en) | False-touch prevention method and device | |
| CN105022571A (en) | Fingerprint information processing method and system | |
| CN107765980A (en) | Input method and device, terminal device and computer-readable recording medium | |
| CN109954272A (en) | A kind of handle key value mapping method, relevant apparatus and handle based on user's habit | |
| CN116360589A (en) | Method and medium for inputting information by virtual keyboard and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |