CN110189333A - A kind of picture semantic divides semi-automatic mask method and device - Google Patents
A kind of picture semantic divides semi-automatic mask method and device Download PDFInfo
- Publication number
- CN110189333A CN110189333A CN201910430851.2A CN201910430851A CN110189333A CN 110189333 A CN110189333 A CN 110189333A CN 201910430851 A CN201910430851 A CN 201910430851A CN 110189333 A CN110189333 A CN 110189333A
- Authority
- CN
- China
- Prior art keywords
- picture
- tracked
- image
- classification
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000002372 labelling Methods 0.000 claims abstract description 48
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 42
- 238000012986 modification Methods 0.000 claims description 32
- 230000004048 modification Effects 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 description 21
- 230000008901 benefit Effects 0.000 description 5
- 239000000243 solution Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 238000000638 solvent extraction Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000001174 ascending effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000001746 injection moulding Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Abstract
The present invention provides a kind of picture semantics to divide semi-automatic mask method and device, and this method includes obtaining one group of image comprising multiframe picture, and first frame picture is selected from the image got；First frame picture semantic is divided into multiple images block, receives mark personnel to the classification labeling operation of each image block in first frame picture；Using first frame picture as initial picture, using next frame picture adjacent with initial picture in multiframe picture as tracked picture, image-region corresponding with the image block of classification is marked is tracked in being tracked picture using default track algorithm, the image-region traced into is labeled as respective classes；Using the tracked picture after currently marking as new initial picture, continues the tracked picture adjacent to new initial picture using default track algorithm and carry out tracking mark, until multiframe picture marks completion.The present invention can effectively promote annotating efficiency, reduce artificial mark cost.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to a kind of picture semantic divide semi-automatic mask method and
Device.
Background technique
Have become the main view in automatic Pilot field currently based on target detection and the Scene Semantics segmentation of deep learning
Feel cognitive method.Wherein, semantic segmentation can be partitioned into road surface in picture, automobile, lane line etc. for the picture of input
Different classes of content, its essence is the classification for each pixel in picture.
Target detection or Scene Semantics segmentation either based on deep learning, require to be related to a large amount of picture and figure
The segmentation of piece labeling operation, especially Scene Semantics, partitioning scheme popular at present is polygon partitioning scheme, but this
Partitioning scheme takes time and effort, and cost of labor is huge, and the effect is unsatisfactory for mark, is easy to appear the object edge ratio of segmentation
The problems such as rougher.
Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind
The picture semantic for stating problem divides semi-automatic mask method and device.
According to the present invention on the one hand, a kind of semi-automatic mask method of picture semantic segmentation is provided, comprising:
One group of image comprising multiframe picture is obtained, first frame picture is selected from the image got；
The first frame picture semantic is divided into multiple images block, receives mark personnel in the first frame picture
The classification labeling operation of each image block；
Using the first frame picture as initial picture, by next frame figure adjacent with initial picture in the multiframe picture
Piece is tracked in the tracked picture and the image block with mark classification as picture is tracked using default track algorithm
The image-region traced into is labeled as respective classes by corresponding image-region；
Using the tracked picture after currently marking as new initial picture, continue it is adjacent to new initial picture by with
Track picture carries out tracking mark using default track algorithm, until the multiframe picture marks completion.
Optionally, the method also includes:
It is in office once to the tracked picture mark after, judge whether deposit in the annotation results of the tracked picture
In marking error；
If so, showing a variety of modification mode options for being labeled modification to the tracked picture；
Receive it is described mark personnel selection any option, according to by the corresponding modification mode of selection option to it is described by with
Track picture is labeled modification.
Optionally, a variety of modification mode options of modification are labeled to the tracked picture, comprising:
The option of artificial mark modification is carried out to the tracked picture；
Tracked the option of mark again to the tracked picture.
Optionally, the first frame picture semantic is divided into multiple images block, receives mark personnel to the first frame
The classification labeling operation of each image block in picture, comprising:
The first frame picture semantic is divided into multiple polygon mats as described image block；
Mark personnel are received to the classification labeling operation of arbitrary polygon block in the first frame picture.
Optionally, the first frame picture semantic is divided into multiple images block, receives mark personnel to the first frame
The classification labeling operation of each image block in picture, comprising:
The first frame picture semantic is divided into multiple super-pixel block as described image block；
Mark personnel are received to the classification labeling operation of super-pixel block any in the first frame picture.
Optionally, the method also includes: the identical super-pixel block of classification will be marked in the first frame picture and merged into
Combination block；
Figure corresponding with the image block of classification is marked is tracked in the tracked picture using default track algorithm
As region, the image-region traced into, which is labeled as respective classes, includes:
Figure corresponding with the combination block of classification is marked is tracked in the tracked picture using default track algorithm
As region, the image-region traced into is labeled as respective classes.
Optionally, the method also includes: selected in the super-pixel block of each mark classification in the first frame picture
Typical super-pixel block is selected, the typical case super-pixel block periphery includes multiple super-pixel block for belonging to same mark classification；
Figure corresponding with the image block of classification is marked is tracked in the tracked picture using default track algorithm
As region, the image-region traced into, which is labeled as respective classes, includes:
Image district corresponding with typical case's super-pixel block is tracked in the tracked picture using default track algorithm
The image-region traced into is labeled as respective classes by domain；
Continuous selection operation on the basis of receiving typical super-pixel block of the mark personnel after marking classification, will continuously choose
It operates corresponding super-pixel block and is labeled as classification identical with typical super-pixel block classification.
Optionally, the image-region traced into is labeled as after respective classes, further includes:
Continuous selection operation on the basis of receiving any super-pixel block of the mark personnel after marking classification, will continuously choose
It operates corresponding super-pixel block and is labeled as classification identical with benchmark super-pixel block classification.
Optionally, the method also includes:
If there is also the image-regions not traced into the tracked picture, will not traced into according to default clustering algorithm
Image-region in the tracked picture mark classification after image block clustered；
According to the classification for the image block for belonging to same cluster with the image-region not traced into, do not traced into described
Image-region carry out classification mark.
According to the present invention on the other hand, a kind of semi-automatic annotation equipment of picture semantic segmentation is additionally provided, comprising:
Module is chosen, suitable for successively choosing multiframe picture according to designated pictures frame number interval from video gathered in advance,
Obtain the first frame picture in the multiframe picture；
Divide labeling module, suitable for the first frame picture semantic is divided into multiple images block, receives mark personnel couple
The classification labeling operation of each image block in the first frame picture；
Labeling module is tracked, is suitable for using the first frame picture as initial picture, it will be in the multiframe picture and initially
The adjacent next frame picture of picture tracks in the tracked picture as picture is tracked, using default track algorithm and band
There is the corresponding image-region of image block of mark classification, the image-region traced into is labeled as respective classes；
The tracking labeling module is further adapted for continuing using the tracked picture after currently marking as new initial picture
The tracked picture adjacent to new initial picture carries out tracking mark using default track algorithm, until the multiframe picture is equal
Mark is completed.
According to the present invention on the other hand, a kind of computer storage medium, the computer storage medium storage are additionally provided
There is computer program code, when the computer program code is run on the computing device, the calculating equipment is caused to execute
Picture semantic described in any embodiment above divides semi-automatic mask method.
In embodiments of the present invention, by selecting first frame picture from one group of image of acquisition, and by marking personnel couple
After each image block in first frame picture carries out classification mark, using first frame picture as initial picture, and by multiframe picture
In the next frame picture adjacent with initial picture as picture is tracked, thus using default track algorithm in being tracked picture
Tracking image-region corresponding with the image block of classification is marked, is labeled as respective classes for the image-region traced into.Into
And again using the tracked picture after currently marking as new initial picture, continue it is adjacent to new initial picture by with
Track picture carries out tracking mark using default track algorithm, until multiframe picture marks completion.The embodiment of the present invention pair as a result,
Multiframe picture in continuous videos, can be automanual using default track algorithm realization based on the picture for having marked classification
Picture semantic segmentation mark, not only substantially increases the annotating efficiency to picture, it is ensured that the accuracy of picture mark.Especially
It is front and back two adjacent in the picture to be marked that successively samples in video to be opened or that there are scenes for front and back several is similar
Picture, the present invention program dramatically save repetition or similar manpower mark cost.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
According to the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will be brighter
The above and other objects, advantages and features of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows the process signal that picture semantic according to an embodiment of the invention divides semi-automatic mask method
Figure；
Fig. 2 shows the structural representations that picture semantic according to an embodiment of the invention divides semi-automatic annotation equipment
Figure；
Fig. 3 shows the structural representation that picture semantic in accordance with another embodiment of the present invention divides semi-automatic annotation equipment
Figure；
The picture semantic that Fig. 4 shows further embodiment according to the present invention divides the structural representation of semi-automatic annotation equipment
Figure；
The picture semantic that Fig. 5 shows another embodiment according to the present invention divides the structural representation of semi-automatic annotation equipment
Figure.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
Usually when being labeled to actual acquisition picture, continuous video is first acquired, is then spaced from continuous videos
Certain frame number selection has the picture of different to be labeled, although having done certain interval sampling to video, usually images
Head frame frequency is very fast, and the image that the same camera of ordinary circumstance acquires in the certain time under the same scene does not have very
Big difference, therefore the picture in same section of video after interval sampling is cleared up still has very big similitude, this can undoubtedly give
Picture marks the huge repetitive operation of work bring.In order to effectively improve picture annotating efficiency, the embodiment of the present invention is mentioned
A kind of picture semantic has been supplied to divide semi-automatic mask method.Fig. 1 shows picture semantic according to an embodiment of the invention point
Cut the flow diagram of semi-automatic mask method.Referring to Fig. 1, this method includes at least step S102 to step S108.
Step S102 obtains one group of image comprising multiframe picture, and first frame picture is selected from the image got.
First frame picture semantic is divided into multiple images block by step S104, receives mark personnel in first frame picture
Each image block classification labeling operation.
Step S106, using first frame picture as initial picture, by next frame adjacent with initial picture in multiframe picture
Picture is tracked and the image block pair with mark classification as picture is tracked, using default track algorithm in being tracked picture
The image-region traced into is labeled as respective classes by the image-region answered.
Step S108 continues using the tracked picture after currently marking as new initial picture to new initial picture
Adjacent tracked picture carries out tracking mark using default track algorithm, until multiframe picture marks completion.
The embodiment of the present invention is for the multiframe picture in continuous videos, first using first frame picture therein as initial graph
Piece, the next frame picture adjacent with initial picture is as picture is tracked, by the tracking in being tracked picture and with mark
The corresponding image-region of the image block of classification, to carry out classification mark to the image-region traced into.Then it will just mark again
At tracked picture as new initial picture, the next frame picture adjacent with new initial picture is as new tracked figure
Piece, by tracking image district corresponding with the mark image block of classification is had in new initial picture in new tracked picture
Domain, and classification mark is carried out to the image-region traced into, it so continues to execute until multiframe picture marks completion, thus real
Show and automanual picture semantic segmentation mark is carried out using default track algorithm based on the picture for having marked classification, not only greatly
Annotating efficiency to picture is improved greatly, it is ensured that the accuracy of picture mark.Especially for successively sampling in video
Picture in it is adjacent and there are the similar picture of scene, the present invention program can greatly save repetition or similar manpower mark
Form this.
Step S102 is seen above, in an embodiment of the present invention, one group of image comprising multiframe picture of acquisition can be with
The combination for the multiframe picture successively chosen from one section gathered in advance continuous video according to designated pictures frame number interval.Its
In, designated pictures frame number can be three frames, five frames etc. any frame number, it is therefore an objective to select the figure in video with different
Piece, and marked accordingly, the embodiment of the present invention does not do specific restriction to designated pictures frame number.Additionally need explanation
It is that first frame picture here is not necessarily the first frame picture of complete video, but according to designated pictures frame number interval sampling
In the multiframe picture of selection, first picture is arranged on video time.
In an embodiment of the present invention, it is in office once to tracked picture mark after, can also judge currently to be marked
It whether there is marking error in the annotation results of the picture of completion.Marking error if it exists can then be shown to tracked picture
It is labeled a variety of modification mode options of modification.Then by receiving any option of mark personnel selection, foundation is selected
The corresponding modification mode of option is labeled modification to tracked picture.
In this embodiment, a variety of modification mode options for modification being labeled to tracked picture may include to by with
Track picture carries out the option of artificial mark modification, is tracked the option of mark again to tracked picture.Mark personnel can be with
It is selected accordingly according to demand, if marking error present in annotation results is less, can choose artificial mark modification,
If marking error present in annotation results is more, it can choose and tracking mark again is carried out to tracked picture, to reduce
Mark personnel mark the workload of modification.
Certainly, if learning that there is no marking errors in annotation results by judgement, if that in the one group of image obtained
There is also the pictures for not carrying out tracking mark, then continuing to carry out tracking mark to next frame picture, if the group picture obtained
Whole pictures as in marked, then can be shown final annotation results.
After marking completion to the multiframe picture in video gathered in advance, the result of mark can also be showed
Mark personnel, mark personnel can check annotation results, if correct through detection mark, completion be marked, if deposited
In a small amount of marking error, mark personnel can correct a mistake part, be labeled without accent to picture.
Specifically, showing the mark of tracking picture after marking completion to the multiframe picture in video gathered in advance
Note as a result, and issue the prompt information whether modified to annotation results, annotation results are repaired if receiving mark personnel
Change instruction, corresponding modification operation is carried out to annotation results according to modification instruction.
Step S104 is seen above, in an embodiment of the present invention, first frame picture semantic can be divided into multiple more
Picture semantic is divided into multiple polygonal regions as image block by side shape block, each polygonal region is defined as more than one
Side shape block, and a corresponding image block.In turn, mark personnel be can receive to the classification of arbitrary polygon block in first frame picture
Labeling operation.
In an alternative embodiment of the invention, first frame picture semantic can be divided into multiple super-pixel block as image
Picture semantic is divided into multiple super-pixel block by block, each multiple super-pixel block are as an image block.In turn, Ke Yijie
Mark personnel are received to the classification labeling operation of super-pixel block any in first frame picture.
Certainly, the embodiment of the present invention can also use other partitioning schemes by first frame picture segmentation be multiple subunits with
As image block, the embodiment of the present invention does not do specific restriction to this.
For below using super-pixel block as image block, introduce to the classification mark of each image block in first frame picture
Process.
In order to improve annotating efficiency, mark personnel, can be only to figure during the super-pixel block to picture is labeled
Fraction super-pixel block in piece is labeled, and the unification to other super-pixel block is realized by the way of continuous selection operation
Mark.Specifically, firstly, receive mark personnel to the classification labeling operation of any super-pixel block, and be any super-pixel block point
With corresponding mark classification.Then, continuous selection operation of the mark personnel on the basis of the super-pixel block for having marked classification, will even
The continuous corresponding super-pixel block of selection operation is labeled as classification identical with benchmark super-pixel block classification.Also, it is continuous when monitoring
When the corresponding super-pixel block of selection operation forms closed loop, the super-pixel block within closed loop can also be labeled as and selection operation pair
Answer the identical classification of super-pixel block classification.
Furthermore it is also possible to the operation that mark personnel adjust super-pixel threshold size in super-pixel block be received, so that super-pixel
Threshold value meets the mark demand of mark personnel.For example, if mouse roller rotates from the bottom to top, the descending tune of super-pixel threshold value
It is whole, if mouse roller from top to bottom rotates, the ascending adjustment of super-pixel threshold value.In another example if mouse roller from top to bottom
Rotation, then the descending adjustment of super-pixel threshold value, if mouse roller rotates from the bottom to top, the ascending tune of super-pixel threshold value
It is whole.
In this embodiment, when adjust super-pixel threshold size when, if region shared by a super-pixel block with marked
The overlapping region in region shared by the super-pixel block of classification is greater than 1/2 of region shared by the super-pixel block, which is marked
For classification identical with the super-pixel block classification of overlapping region.
See above step S106 and combine foregoing embodiments, the notation methods based on super-pixel block, if choose by with
All super-pixel block are tracked in track picture, then the super-pixel block quantity for needing to track is more, so as to cause algorithm calculating
Therefore the problems such as amount is big, and the computer speed of service is slow in order to reduce tracking calculation amount, saves tracking and calculates the time, in the present invention
Embodiment can also save tracking calculation amount using the following two kinds mode.
Mode one, before carrying out tracking mark to tracked picture, first will in first frame picture mark classification it is identical
Super-pixel block merges into combination block, that is, the super-pixel block for having marked classification is merged into several according to same category and is not advised
Big combination block then.In turn, subsequent to be tracked in being tracked picture using default track algorithm and with mark classification
The corresponding image-region of combination block, the image-region traced into is labeled as respective classes.
Mode two, before carrying out tracking mark to tracked picture, first each mark classification in first frame picture
Super-pixel block in filter out typical super-pixel block, the object as tracking.In turn, it is subsequent using default track algorithm by with
Tracking image-region corresponding with typical super-pixel block, is labeled as respective classes for the image-region traced into track picture.
When screening typical super-pixel block, can be chosen using cluster mode than more typical same category of pixel
Block, for example, can choose if most of or whole super-pixel block around some super-pixel block belong to generic
This super-pixel block is as typical super-pixel block.In addition, the embodiment is when filtering out typical super-pixel block, it can be from generic phase
A typical super-pixel block is selected in the super-pixel block region of connection, naturally it is also possible to multiple typical super-pixel are selected, this
Embodiment does not do specific restriction to the quantity for the typical super-pixel block selected.
In this approach, due to only being tracked to typical super-pixel block, in the image-region mark that will be traced into
It infuses as classification mark can also be carried out to remaining super-pixel block by mark personnel after respective classes.For example, receiving mark people
Continuous selection operation on the basis of typical super-pixel block of the member after marking classification, by the corresponding super-pixel block of continuous selection operation
It is labeled as classification identical with typical super-pixel block classification.Certainly, the corresponding super-pixel block shape of continuous selection operation is being monitored
When at closed loop, the super-pixel block within closed loop can also be labeled as to classification identical with typical super-pixel block classification.
In embodiments of the present invention, default track algorithm can use existing track algorithm, and the embodiment of the present invention is to this
Specific restriction is not done.For example, the image block that segmentation obtains is super-pixel block, it can be by constructing absorbing state Markov Chain figure
The tracking of (AMC graph) Lai Shixian super-pixel block.In another example the image block divided is super-pixel block, it can also be by building
Vertical judgement property appearance model (the appearance model that can indicate and distinguish target and background) obtains region confidence map (i.e. basis
The value of the confidence of all super-pixel obtains in one frame picture confidence map), establish object tracker observation model and etc. realize
Super-pixel block tracking.
In an embodiment of the present invention, if since image scene, object space or the article size in picture change greatly
Etc. reasons, it is current be tracked in picture tracking less than with the corresponding image-region of the mark image block of classification, or with
Track failure then can be tracked picture and clustered for this frame again, will be immediate with the image-region that can not track
The classification for having marked image block (tracking successful image block), is applied directly in the image-region that can not be tracked.
Specifically, if being tracked in picture, there is also the image-regions not traced into, can be according to default clustering algorithm
By the image-region not traced into and in tracked picture mark classification after image block cluster, in turn, according to not with
Track to image-region belong to same cluster image block classification, classification mark is carried out to the image-region that does not trace into.?
In the embodiment, default clustering algorithm may include k-means method, it is of course also possible to use other clustering algorithms, the present invention
Embodiment does not do specific restriction to this.
Based on the same inventive concept, the embodiment of the invention also provides a kind of picture semantics to divide semi-automatic annotation equipment,
Fig. 2 shows the structural schematic diagrams that picture semantic according to an embodiment of the invention divides semi-automatic annotation equipment.Referring to figure
2, it includes choosing module 210, segmentation labeling module 220, tracking labeling module that picture semantic, which divides semi-automatic annotation equipment 200,
230。
The picture semantic for now introducing the embodiment of the present invention divides each composition of semi-automatic annotation equipment 200 or the function of device
And the connection relationship between each section:
Module 210 is chosen, suitable for obtaining one group of image comprising multiframe picture, selects first frame from the image got
Picture；
Divide labeling module 220, is coupled with module 210 is chosen, suitable for first frame picture semantic is divided into multiple images
Block receives mark personnel to the classification labeling operation of each image block in first frame picture；
Labeling module 230 is tracked, is coupled with segmentation labeling module 220, is suitable for using first frame picture as initial picture, it will
The next frame picture adjacent with initial picture is being tracked figure using default track algorithm as picture is tracked in multiframe picture
Tracking image-region corresponding with the image block of classification is marked, is labeled as respective class for the image-region traced into piece
Not；
Labeling module 230 is tracked, is further adapted for using the tracked picture after currently marking as new initial picture, is continued pair
The adjacent tracked picture of new initial picture carries out tracking mark using default track algorithm, until multiframe picture has marked
At.
In an embodiment of the present invention, segmentation labeling module 220 is further adapted for for first frame picture semantic being divided into multiple more
Side shape block receives mark personnel to the classification labeling operation of arbitrary polygon block in first frame picture as image block.
In an alternative embodiment of the invention, segmentation labeling module 220 is further adapted for for first frame picture semantic being divided into multiple
Super-pixel block receives mark personnel to the classification labeling operation of super-pixel block any in first frame picture as image block.
First frame picture semantic is divided into multiple super-pixel block as image block, at one if dividing labeling module 220
In embodiment, referring to Fig. 3, it can also include composite module 240 that picture semantic, which divides semi-automatic annotation equipment 200, mark with segmentation
Injection molding block 220 and tracking labeling module 230 couple respectively, and composite module 240 is suitable for that classification will be marked in first frame picture identical
Super-pixel block merge into combination block.Tracking labeling module 230 be further adapted for using default track algorithm in being tracked picture with
Track image-region corresponding with the combination block of classification is marked, is labeled as respective classes for the image-region traced into.
First frame picture semantic is divided into multiple super-pixel block as image block, another if dividing labeling module 220
In a embodiment, referring to fig. 4, picture semantic, which divides semi-automatic annotation equipment 200, to include selecting module 250, with segmentation
Labeling module 220 and tracking labeling module 230 couple respectively, and selecting module 250 is suitable for each mark in first frame picture
Typical super-pixel block is selected in the super-pixel block of classification, typical super-pixel block periphery belongs to the super of same mark classification comprising multiple
Block of pixels.Tracking labeling module 230 is further adapted for tracking in being tracked picture and typical super-pixel block using default track algorithm
The image-region traced into is labeled as respective classes by corresponding image-region.
With continued reference to Fig. 4, in this embodiment, it can also include continuous that picture semantic, which divides semi-automatic annotation equipment 200,
Module 260 is chosen, suitable for receiving mark after the image-region traced into is labeled as respective classes by tracking labeling module 230
Continuous selection operation on the basis of typical super-pixel block of the note personnel after marking classification, by the corresponding super picture of continuous selection operation
Plain block is labeled as classification identical with typical super-pixel block classification.
The embodiment of the invention also provides another picture semantics to divide semi-automatic annotation equipment, and Fig. 5 is shown according to this
The picture semantic of invention one embodiment divides the structural schematic diagram of semi-automatic annotation equipment.Referring to Fig. 5, picture semantic segmentation half
Automatic marking device 200 includes choosing module 210, segmentation labeling module 220, tracking labeling module 230, judgment module 271, exhibition
Show module 270, modified module 280, cluster module 290.Wherein, about selection module 210, segmentation labeling module 220, tracking mark
The introduction of injection molding block 230 refers to foregoing embodiments.
Judgment module 271, with tracking labeling module 230 couple, be suitable for it is in office once to tracked picture mark after, sentence
It whether there is marking error in the annotation results of disconnected tracked picture.
Display module 270 is coupled with judgment module 271, if it is wrong to there is mark in the annotation results suitable for tracked picture
Accidentally, then a variety of modification mode options that modification is labeled to tracked picture are shown.
Wherein, it is labeled a variety of modification mode options of modification to tracked picture, including tracked picture is carried out
The option of artificial mark modification, the option for being tracked mark again to tracked picture.
Modified module 280 is coupled with display module 270, suitable for receiving any option of mark personnel selection, and according to quilt
The corresponding modification mode of selection option is labeled modification to tracked picture.
Cluster module 290 is coupled with tracking labeling module 230, if being tracked in picture, there is also the images not traced into
Region carries out the image block after marking classification in the image-region not traced into and tracked picture according to default clustering algorithm
Cluster.According to the classification for the image block for belonging to same cluster with the image-region not traced into, to the image-region not traced into
Carry out classification mark.
The present invention also provides a kind of computer storage medium, computer storage medium is stored with computer program code,
When computer program code is run on the computing device, cause to calculate the picture semantic point that equipment executes any embodiment above
Cut semi-automatic mask method.
According to the combination of any one above-mentioned preferred embodiment or multiple preferred embodiments, the embodiment of the present invention can reach
It is following the utility model has the advantages that
In embodiments of the present invention, more by successively being chosen from video gathered in advance according to designated pictures frame number interval
Frame picture, and by mark personnel in first frame picture each image block carry out classification mark after, using first frame picture as
Initial picture, and using next frame picture adjacent with initial picture in multiframe picture as tracked picture, thus using default
Track algorithm tracking in being tracked picture image-region corresponding with the image block of classification is marked, the image that will be traced into
Area marking is respective classes.In turn, continue again using the tracked picture after currently marking as new initial picture to new
Initial picture adjacent tracked picture tracking mark is carried out using default track algorithm, until multiframe picture has marked
At.The embodiment of the present invention can use pre- multiframe picture in continuous videos based on the picture for having marked classification as a result,
If track algorithm realizes automanual picture semantic segmentation mark, the annotating efficiency to picture is not only substantially increased, it is ensured that
The accuracy of picture mark.Opened especially for front and back two adjacent in the picture to be marked successively sampled in video or
There are the similar picture of scene, the present invention program dramatically saves repetition or similar manpower mark cost for front and back several.
It is apparent to those skilled in the art that the specific work of the system of foregoing description, device and unit
Make process, can refer to corresponding processes in the foregoing method embodiment, for brevity, does not repeat separately herein.
In addition, each functional unit in each embodiment of the present invention can be physically independent, can also two or
More than two functional units integrate, and can be all integrated in a processing unit with all functional units.It is above-mentioned integrated
Functional unit both can take the form of hardware realization, can also be realized in the form of software or firmware.
Those of ordinary skill in the art will appreciate that: if integrated functional unit is realized in the form of software and as only
Vertical product when selling or using, can store in a computer readable storage medium.Based on this understanding, this hair
Bright technical solution is substantially or all or part of the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium comprising some instructions, with (such as personal so that calculating equipment
Computer, server or network equipment etc.) all or part of step of execution various embodiments of the present invention method in operating instruction
Suddenly.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk
Or the various media that can store program code such as CD.
Alternatively, realizing that all or part of the steps of preceding method embodiment can be (all by the relevant hardware of program instruction
Such as personal computer, the calculating equipment of server or network equipment etc.) it completes, program instruction can store to be calculated in one
In machine read/write memory medium, when program instruction is executed by the processor of calculating equipment, calculates equipment and execute each reality of the present invention
Apply all or part of the steps of a method.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Present invention has been described in detail with reference to the aforementioned embodiments for pipe, those skilled in the art should understand that: at this
Within the spirit and principle of invention, it is still possible to modify the technical solutions described in the foregoing embodiments or right
Some or all of the technical features are equivalently replaced；And these are modified or replaceed, and do not make corresponding technical solution de-
From protection scope of the present invention.
Claims (10)
1. a kind of picture semantic divides semi-automatic mask method, comprising:
One group of image comprising multiframe picture is obtained, first frame picture is selected from the image got；
The first frame picture semantic is divided into multiple images block, receives mark personnel to each figure in the first frame picture
As the classification labeling operation of block；
Using the first frame picture as initial picture, next frame picture adjacent with initial picture in the multiframe picture is made
To be tracked picture, tracked in the tracked picture using default track algorithm corresponding with the mark image block of classification
Image-region, the image-region traced into is labeled as respective classes；
Using the tracked picture after currently marking as new initial picture, continue the tracked figure adjacent to new initial picture
Piece carries out tracking mark using default track algorithm, until the multiframe picture marks completion.
2. according to the method described in claim 1, wherein, further includes:
It is in office once to the tracked picture mark after, judge in the annotation results of the tracked picture with the presence or absence of marking
Infuse mistake；
If so, showing a variety of modification mode options for being labeled modification to the tracked picture；
Any option of the mark personnel selection is received, foundation is by the corresponding modification mode of selection option to the tracked figure
Piece is labeled modification.
3. according to the method described in claim 2, wherein, a variety of modification modes of modification are labeled to the tracked picture
Option, comprising:
The option of artificial mark modification is carried out to the tracked picture；
Tracked the option of mark again to the tracked picture.
4. method according to claim 1-3, wherein the first frame picture semantic is divided into multiple images
Block receives mark personnel to the classification labeling operation of each image block in the first frame picture, comprising:
The first frame picture semantic is divided into multiple polygon mats as described image block；
Mark personnel are received to the classification labeling operation of arbitrary polygon block in the first frame picture.
5. method according to claim 1-3, wherein the first frame picture semantic is divided into multiple images
Block receives mark personnel to the classification labeling operation of each image block in the first frame picture, comprising:
The first frame picture semantic is divided into multiple super-pixel block as described image block；
Mark personnel are received to the classification labeling operation of super-pixel block any in the first frame picture.
6. according to the method described in claim 5, wherein, further includes: identical super by classification is marked in the first frame picture
Block of pixels merges into combination block；
Image district corresponding with the image block of classification is marked is tracked in the tracked picture using default track algorithm
Domain, the image-region traced into, which is labeled as respective classes, includes:
Image district corresponding with the combination block of classification is marked is tracked in the tracked picture using default track algorithm
The image-region traced into is labeled as respective classes by domain.
7. according to the method described in claim 5, wherein, further includes: each mark classification in the first frame picture
Typical super-pixel block is selected in super-pixel block, the typical case super-pixel block periphery includes multiple super pictures for belonging to same mark classification
Plain block；
Image district corresponding with the image block of classification is marked is tracked in the tracked picture using default track algorithm
Domain, the image-region traced into, which is labeled as respective classes, includes:
Image-region corresponding with typical case's super-pixel block is tracked in the tracked picture using default track algorithm, it will
The image-region traced into is labeled as respective classes；
Continuous selection operation on the basis of receiving typical super-pixel block of the mark personnel after marking classification, by continuous selection operation
Corresponding super-pixel block is labeled as classification identical with typical super-pixel block classification.
8. method according to claim 1-3, wherein further include:
If there is also the image-region not traced into the tracked picture, the figure that will do not traced into according to default clustering algorithm
As region and the image block after mark classification in the tracked picture are clustered；
According to the classification for the image block for belonging to same cluster with the image-region not traced into, to the figure not traced into
As region carries out classification mark.
9. a kind of picture semantic divides semi-automatic annotation equipment, comprising:
Module is chosen, suitable for obtaining one group of image comprising multiframe picture, first frame picture is selected from the image got；
Divide labeling module, suitable for the first frame picture semantic is divided into multiple images block, receives mark personnel to described
The classification labeling operation of each image block in first frame picture；
Track labeling module, be suitable for using the first frame picture as initial picture, by the multiframe picture with initial picture
Adjacent next frame picture is tracked in the tracked picture using default track algorithm as picture is tracked and has mark
The corresponding image-region of image block for infusing classification, is labeled as respective classes for the image-region traced into；
The tracking labeling module is further adapted for continuing using the tracked picture after currently marking as new initial picture to new
Initial picture adjacent tracked picture tracking mark is carried out using default track algorithm, until the multiframe picture marks
It completes.
10. a kind of computer storage medium, the computer storage medium is stored with computer program code, when the computer
When program code is run on the computing device, the calculating equipment perform claim is caused to require the described in any item picture languages of 1-8
Justice divides semi-automatic mask method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910430851.2A CN110189333B (en) | 2019-05-22 | 2019-05-22 | Semi-automatic marking method and device for semantic segmentation of picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910430851.2A CN110189333B (en) | 2019-05-22 | 2019-05-22 | Semi-automatic marking method and device for semantic segmentation of picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110189333A true CN110189333A (en) | 2019-08-30 |
| CN110189333B CN110189333B (en) | 2022-03-15 |
Family
ID=67717297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910430851.2A Active CN110189333B (en) | 2019-05-22 | 2019-05-22 | Semi-automatic marking method and device for semantic segmentation of picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110189333B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112037255A (en) * | 2020-08-12 | 2020-12-04 | 深圳市道通智能航空技术有限公司 | Target tracking method and device |
| CN112418335A (en) * | 2020-11-27 | 2021-02-26 | 北京云聚智慧科技有限公司 | Model training method based on continuous image frame tracking and labeling and electronic equipment |
| CN113178079A (en) * | 2021-04-06 | 2021-07-27 | 青岛以萨数据技术有限公司 | Marking system, method and storage medium for signal lamp and lane line |
| CN113343857A (en) * | 2021-06-09 | 2021-09-03 | 浙江大华技术股份有限公司 | Labeling method, labeling device, storage medium and electronic device |
| WO2022160736A1 (en) * | 2021-01-28 | 2022-08-04 | 上海商汤智能科技有限公司 | Image annotation method and apparatus, electronic device, storage medium and program |
| WO2023024959A1 (en) * | 2021-08-27 | 2023-03-02 | 广州市网星信息技术有限公司 | Image labeling method and system, and device and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559237A (en) * | 2013-10-25 | 2014-02-05 | 南京大学 | Semi-automatic image annotation sample generating method based on target tracking |
| CN106023145A (en) * | 2016-05-06 | 2016-10-12 | 哈尔滨工程大学 | Remote sensing image segmentation and identification method based on superpixel marking |
| CN108986134A (en) * | 2018-08-17 | 2018-12-11 | 浙江捷尚视觉科技股份有限公司 | A kind of semi-automatic mask method of video object based on correlation filtering tracking |
| CN109409248A (en) * | 2018-09-30 | 2019-03-01 | 上海交通大学 | Semanteme marking method, apparatus and system based on deep semantic network |
| CN109766830A (en) * | 2019-01-09 | 2019-05-17 | 深圳市芯鹏智能信息有限公司 | A kind of ship seakeeping system and method based on artificial intelligence image procossing |
-
2019
- 2019-05-22 CN CN201910430851.2A patent/CN110189333B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103559237A (en) * | 2013-10-25 | 2014-02-05 | 南京大学 | Semi-automatic image annotation sample generating method based on target tracking |
| CN106023145A (en) * | 2016-05-06 | 2016-10-12 | 哈尔滨工程大学 | Remote sensing image segmentation and identification method based on superpixel marking |
| CN108986134A (en) * | 2018-08-17 | 2018-12-11 | 浙江捷尚视觉科技股份有限公司 | A kind of semi-automatic mask method of video object based on correlation filtering tracking |
| CN109409248A (en) * | 2018-09-30 | 2019-03-01 | 上海交通大学 | Semanteme marking method, apparatus and system based on deep semantic network |
| CN109766830A (en) * | 2019-01-09 | 2019-05-17 | 深圳市芯鹏智能信息有限公司 | A kind of ship seakeeping system and method based on artificial intelligence image procossing |
Non-Patent Citations (2)
| Title |
|---|
| SZIL´ARD VAJDA 等: "Label the many with a few: Semi-automatic medical image modality discovery in a large image collection", 《2014 IEEE SYMPOSIUM ON COMPUTATIONAL INTELLIGENCE IN HEALTHCARE AND E-HEALTH (CICARE)》 * |
| 曹奇敏: "网络信息文本挖掘若干问题研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112037255A (en) * | 2020-08-12 | 2020-12-04 | 深圳市道通智能航空技术有限公司 | Target tracking method and device |
| CN112418335A (en) * | 2020-11-27 | 2021-02-26 | 北京云聚智慧科技有限公司 | Model training method based on continuous image frame tracking and labeling and electronic equipment |
| CN112418335B (en) * | 2020-11-27 | 2024-04-05 | 北京云聚智慧科技有限公司 | Model training method based on continuous image frame tracking annotation and electronic equipment |
| WO2022160736A1 (en) * | 2021-01-28 | 2022-08-04 | 上海商汤智能科技有限公司 | Image annotation method and apparatus, electronic device, storage medium and program |
| CN113178079A (en) * | 2021-04-06 | 2021-07-27 | 青岛以萨数据技术有限公司 | Marking system, method and storage medium for signal lamp and lane line |
| CN113178079B (en) * | 2021-04-06 | 2022-08-23 | 青岛以萨数据技术有限公司 | Marking system, method and storage medium for signal lamp and lane line |
| CN113343857A (en) * | 2021-06-09 | 2021-09-03 | 浙江大华技术股份有限公司 | Labeling method, labeling device, storage medium and electronic device |
| CN113343857B (en) * | 2021-06-09 | 2023-04-18 | 浙江大华技术股份有限公司 | Labeling method, labeling device, storage medium and electronic device |
| WO2023024959A1 (en) * | 2021-08-27 | 2023-03-02 | 广州市网星信息技术有限公司 | Image labeling method and system, and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110189333B (en) | 2022-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110189333A (en) | A kind of picture semantic divides semi-automatic mask method and device | |
| EP2523165B1 (en) | Image processing method and image processing device | |
| CN108846365B (en) | Detection method and device for fighting behavior in video, storage medium and processor | |
| RU2637989C2 (en) | Method and device for identifying target object in image | |
| CN102694966B (en) | Construction method of full-automatic video cataloging system | |
| US9373040B2 (en) | Image matching using motion manifolds | |
| CN105493078B (en) | Colored sketches picture search | |
| CN110443210A (en) | A kind of pedestrian tracting method, device and terminal | |
| CN103347167A (en) | Surveillance video content description method based on fragments | |
| CN110619284B (en) | Video scene division method, device, equipment and medium | |
| EP3172681A1 (en) | Identifying presentation styles of educational videos | |
| CN107886512A (en) | A kind of method for determining training sample | |
| Madan et al. | Synthetically trained icon proposals for parsing and summarizing infographics | |
| Zhao et al. | Automatic blur region segmentation approach using image matting | |
| CN110008900A (en) | A kind of visible remote sensing image candidate target extracting method by region to target | |
| CN110136166A (en) | A kind of automatic tracking method of multichannel picture | |
| CN110458115A (en) | A kind of integrated algorithm of target detection of the multiframe based on timing | |
| CN111429512B (en) | Image processing method and device, storage medium and processor | |
| Yan | RETRACTED ARTICLE: Researches on hybrid algorithm for moving target detection and tracking in sports video | |
| CN110196917A (en) | Personalized LOGO format method for customizing, system and storage medium | |
| Höferlin et al. | Interactive schematic summaries for exploration of surveillance video | |
| Hu et al. | Instance segmentation based semantic matting for compositing applications | |
| CN114913321A (en) | Object attention mining method and system based on local-to-global knowledge migration | |
| CN106503740A (en) | Picture classification method and device | |
| CN108682021A (en) | Rapid hand tracking, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220325 Address after: 430051 No. b1336, chuanggu startup area, taizihu cultural Digital Creative Industry Park, No. 18, Shenlong Avenue, Wuhan Economic and Technological Development Zone, Wuhan, Hubei Province Patentee after: Yikatong (Hubei) Technology Co.,Ltd. Address before: No.c101, chuanggu start up area, taizihu cultural Digital Industrial Park, No.18 Shenlong Avenue, Wuhan Economic Development Zone, Hubei Province Patentee before: HUBEI ECARX TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |