CN112508020A - Labeling method and device, electronic equipment and storage medium - Google Patents

Labeling method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN112508020A
CN112508020A CN202011529117.0A CN202011529117A CN112508020A CN 112508020 A CN112508020 A CN 112508020A CN 202011529117 A CN202011529117 A CN 202011529117A CN 112508020 A CN112508020 A CN 112508020A
Authority
CN
China
Prior art keywords
graph
drawing graph
target image
labeling
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011529117.0A
Other languages
Chinese (zh)
Inventor
罗泽丰
钟华平
何聪辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Sensetime Technology Co Ltd
Original Assignee
Shenzhen Sensetime Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Sensetime Technology Co Ltd filed Critical Shenzhen Sensetime Technology Co Ltd
Priority to CN202011529117.0A priority Critical patent/CN112508020A/en
Publication of CN112508020A publication Critical patent/CN112508020A/en
Priority to JP2022520125A priority patent/JP2023510443A/en
Priority to PCT/CN2021/087285 priority patent/WO2022134390A1/en
Priority to KR1020227009804A priority patent/KR20220093091A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)

Abstract

The present disclosure relates to a labeling method and apparatus, an electronic device, and a storage medium, the method comprising: receiving a first annotation operation on a target image, wherein the first annotation operation is used for drawing a first drawing graph indicating a first image area in the target image; and under the condition that at least part of a second drawing graph in the target image is overlapped with the first drawing graph, adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation.

Description

Labeling method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a labeling method and apparatus, an electronic device, and a storage medium.
Background
In the field of artificial intelligence, information contained in an image can be automatically recognized and understood by training a neural network, and the training is often performed through a labeled image in the process of training the neural network.
In the process of labeling an image, an image is often drawn to determine a labeled area, and in the related art, the efficiency of labeling the image is low.
Disclosure of Invention
The present disclosure presents a labeling scheme.
According to an aspect of the present disclosure, there is provided an annotation method, including:
receiving a first annotation operation on a target image, wherein the first annotation operation is used for drawing a first drawing graph indicating a first image area in the target image;
and under the condition that at least part of a second drawing graph in the target image is overlapped with the first drawing graph, adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation.
In one possible implementation, before the adjusting the first drawing graph, the method further includes:
and detecting whether a drawing graph with an overlapped part with the first drawing graph exists in at least one previous drawing graph in the target image.
In one possible implementation, before the adjusting the first drawing graph, the method further includes:
and receiving a first user instruction, wherein the first user instruction is used for indicating that the first drawing graph is adjusted.
In a possible implementation manner, the adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation includes:
and removing the part of the first drawing graph, which is overlapped with the second drawing graph, from the first drawing graph to obtain a drawing graph, wherein the drawing graph is used as a labeling result corresponding to the first labeling operation.
In a possible implementation manner, the adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation includes:
under the condition that the first drawing graph and the second drawing graph are frame graphs, determining a first line segment element in the first drawing graph in the second drawing graph and a second line segment element outside the second drawing graph in the first drawing graph according to the positions of the first drawing graph and the second drawing graph;
and taking a drawing graph formed by the first line segment element and the second line segment element as a labeling result corresponding to the first labeling operation.
In one possible implementation manner, after the receiving the first annotation operation on the target image, the method further includes:
displaying the target image and a plurality of drawing graphs including the first drawing graph of the target image on a display interface;
receiving a second user instruction for selecting the first drawing figure from the plurality of drawing figures;
and determining the selected first drawing graph according to the position indicated by the second user instruction in the display interface and the positions of the plurality of drawing graphs in the display interface.
In one possible implementation, the target image is displayed on a web interface.
In one possible implementation manner, the first annotation operation is used for performing semantic segmentation annotation on the target image.
According to an aspect of the present disclosure, there is provided an annotation apparatus including:
the system comprises a receiving module, a judging module and a display module, wherein the receiving module is used for receiving a first annotation operation on a target image, and the first annotation operation is used for drawing a first drawing graph indicating a first image area in the target image;
and the adjusting module is used for adjusting the first drawing graph under the condition that at least part of the second drawing graph in the target image is overlapped with the first drawing graph to obtain a labeling result corresponding to the first labeling operation.
In one possible implementation, the apparatus further includes:
and the detection module is used for detecting whether a drawing graph which has an overlapped part with the first drawing graph exists in at least one previous drawing graph in the target image.
In one possible implementation, the apparatus further includes:
and the first user instruction receiving module is used for receiving a first user instruction, and the first user instruction is used for indicating to adjust the first drawing graph.
In a possible implementation manner, the adjusting module is configured to remove, from the first drawing graph, a part of the first drawing graph that overlaps with the second drawing graph, to obtain a drawing graph, where the part of the first drawing graph is used as a labeling result corresponding to the first labeling operation.
In a possible implementation manner, the adjusting module is configured to determine, according to positions of the first drawing graph and the second drawing graph, a first line segment element in the second drawing graph and a second line segment element in the first drawing graph, which is located outside the second drawing graph, in the second drawing graph when the first drawing graph and the second drawing graph are border graphs; and taking a drawing graph formed by the first line segment element and the second line segment element as a labeling result corresponding to the first labeling operation.
In one possible implementation, the apparatus further includes:
the display module is used for displaying the target image and a plurality of drawing graphs including the first drawing graph of the target image on a display interface;
a second user instruction receiving module, configured to receive a second user instruction, where the second user instruction is used to select the first drawing graph from the plurality of drawing graphs;
and the determining module is used for determining the selected first drawing graph according to the position indicated by the second user instruction in the display interface and the positions of the plurality of drawing graphs in the display interface.
In one possible implementation, the target image is displayed on a web interface.
In one possible implementation manner, the first annotation operation is used for performing semantic segmentation annotation on the target image.
According to an aspect of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.
According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.
In the embodiment of the disclosure, in the process of labeling the target image, for a first drawing graph drawn by a first labeling operation of a user on the target image, the first drawing graph can be automatically adjusted under the condition that a second drawing graph in the target image is at least partially overlapped with the first drawing graph, so as to obtain a labeling result corresponding to the first labeling operation. Compared with the manual adjustment of the first drawing graph, the efficiency of labeling the target image is improved, and the labeling time is saved. Especially for the situation that the external outline of the second drawing graph is relatively complex, the user does not need to perform additional manual adjustment on the same outline part, and resource consumption caused by the fact that the user repeats a part labeling process is reduced.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.
Fig. 1 shows a flow chart of an annotation method according to an embodiment of the present disclosure.
FIG. 2 shows a schematic diagram of drawing a graph according to an embodiment of the present disclosure.
FIG. 3 shows a schematic diagram of annotation results according to an embodiment of the disclosure.
FIG. 4 shows a schematic diagram of drawing a graph according to an embodiment of the present disclosure.
FIG. 5 shows a schematic diagram of annotation results according to an embodiment of the disclosure.
FIG. 6 shows a flow chart of an annotation method according to an embodiment of the disclosure.
FIG. 7 shows a block diagram of an annotation device in accordance with an embodiment of the disclosure.
FIG. 8 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.
Fig. 9 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.
Fig. 1 shows a flowchart of an annotation method according to an embodiment of the present disclosure, and as shown in fig. 1, the annotation method includes:
in step S11, a first annotation operation on a target image is received, wherein the first annotation operation is used for drawing a first drawing figure indicating a first image area in the target image.
The target image may be an image to be annotated, and the image may be an image of any format, which is not limited by the embodiment of the present disclosure. Alternatively, the target image may be a sample image for machine learning, and the sample image may be used for supervised learning in machine learning after labeling.
A user, such as an annotator, can annotate a first image region in a target image by a first annotation operation. In some embodiments, the user may determine the first image region to be labeled by drawing a graphic, and optionally, the graphic drawn by the user may be a polygon or a circle, and the drawn graphic may be a regular graphic or an irregular graphic, which is not limited in this disclosure. The user can give a semantic meaning to the image information in the determined first image region by adding annotation information to the first image region.
For example, the content in the target image is that a person stands on a lawn, and in the process of labeling the lawn, the user may draw a drawing graph matching the shape of the image area where the lawn is located, and then add the labeling information "lawn" to the drawn drawing graph to give the semantic meaning of "lawn" to the selected image area.
In step S12, when a second drawing graph in the target image at least partially overlaps the first drawing graph, the first drawing graph is adjusted to obtain a labeling result corresponding to the first labeling operation.
The second drawing graph may also be a graph drawn by a user, or may also be a graph obtained by other means, for example, a graph drawn after image recognition of the target image by a neural network.
In a possible implementation manner, the second drawing graph may be a drawing graph designated by a user, that is, the first drawing graph is automatically adjusted when the first drawing graph and the second drawing graph designated by the user at least partially overlap. The user may specify the second drawing graph by performing a selection operation on the drawing graph in the target image, or the user may also specify the second drawing graph by selecting information such as an identifier of the drawing graph, and the specific implementation manner of specifying the second drawing graph is not limited in the embodiment of the present disclosure.
In a possible implementation manner, the second drawing graph may also be one or more drawing graphs in the drawing graphs included in the target image, for example, the second drawing graph may be a drawing graph that at least partially overlaps with the first drawing graph in the drawing graphs included in the target image, so that when a drawing graph that at least partially overlaps with the first drawing graph exists in the target image, the first drawing graph is automatically adjusted without user specification, and convenience in use of a user is improved.
The first drawing figure and the second drawing figure at least partially overlap, and the image area indicated by the second drawing figure and the first drawing figure at least partially has the same area. For example, referring to fig. 2, the screen content of the target image is that a person stands on a lawn, the area framed by the lower polygon (first drawing figure) is the lawn area, the area framed by the leftmost polygon (second drawing figure) is the person, the legs of the person block part of the lawn, and the person other than the legs do not block the lawn. The first drawing graph comprises an image area where the lawn is located and an image area where the leg part of the lawn is shielded, the second drawing graph comprises an image area where the portrait is located, and obviously, the first drawing graph and the second drawing graph are partially overlapped.
In the case that the first drawing graph and the second drawing graph have partial overlap, the first drawing graph may be automatically adjusted in various ways, for example, the overlapped portion may be removed from the first drawing graph to obtain the first drawing graph with the overlapped portion removed, or the overlapped portion in the first drawing graph may be reduced. For example, referring to fig. 3, after the overlapping portion in the first drawing graph is removed, the adjusted first drawing graph is shown in fig. 3, and the first drawing graph no longer includes the leg portion of the second drawing graph that blocks the lawn.
After the first drawing graph is adjusted, the image area marked by the adjusted first drawing graph can be used as the image area marked by the first marking operation, and a marking result corresponding to the first marking operation is obtained. In the embodiment of the present disclosure, by removing or reducing the overlapping portion in the first drawing graph, the interference information in the first drawing graph is removed or reduced, and the accuracy of the neural network obtained by machine learning can be improved.
In the embodiment of the disclosure, in the process of labeling the target image, for a first drawing graph drawn by a first labeling operation of a user on the target image, the first drawing graph can be automatically adjusted under the condition that a second drawing graph in the target image is at least partially overlapped with the first drawing graph, so as to obtain a labeling result corresponding to the first labeling operation. Compared with the manual adjustment of the first drawing graph, the efficiency of labeling the target image is improved, and the labeling time is saved.
There may be multiple video processing methods provided by the present disclosure, and in one possible implementation, before the adjusting the first drawing graph, the method further includes: and detecting whether a drawing graph with an overlapped part with the first drawing graph exists in at least one previous drawing graph in the target image.
The previously drawn figure may be a figure drawn before the first drawn figure, the previously drawn figure may be a figure drawn by a user, or may be a figure obtained by other means, for example, a figure drawn after image recognition of a target image by a neural network. Taking the previously drawn graph as the graph drawn by the user as an example, after the user finishes drawing the previously drawn graph and then draws the first drawn graph, whether the previously drawn graph and the first drawn graph have an overlapping part can be detected.
In some embodiments, the detection step in this implementation may be performed after it is determined that the first drawing graph is drawn completely, that is, after the first drawing graph is drawn completely, execution of the detection step may be triggered without receiving a user instruction or other operations of a user, so that the user operations are fewer and the convenience is higher.
In this implementation, there may be a plurality of specific detection processes, and optionally, whether there is a drawing pattern that has an overlapping portion with the first drawing pattern in at least one previous drawing pattern in the target image may be detected based on the position coordinates of the at least one previous drawing pattern and the first drawing pattern in the target image.
The graph drawn in the embodiment of the present disclosure may be a frame graph or a filling graph. The border pattern may be a pattern including only lines of the border, and the filling pattern may be a pattern including the border and contents filled in the border.
In the case where the previous drawing pattern and the first drawing pattern are both frame patterns, it may be determined whether or not at least a part of a frame of the previous drawing pattern exists within the frame of the first drawing pattern based on the coordinates of the previous drawing pattern and the coordinates of the first drawing pattern, and in the case where it is determined that at least a part of a frame of the previous drawing pattern exists within the frame of the first drawing pattern, it may be determined that a drawing pattern having an overlapping portion with the first drawing pattern exists within the previous drawing pattern. In addition, it may be determined whether or not at least a part of the point set in the point set constituting the previous drawing figure exists in the frame of the first drawing figure, and when it is determined that at least a part of the point set in the point set constituting the previous drawing figure exists in the frame of the first drawing figure, it may be determined that a drawing figure having an overlapping portion with the first drawing figure exists in the previous drawing figure.
Under the condition that the prior drawing graph and the first drawing graph are both filled graphs, whether pixel points with the same coordinates exist in the prior drawing graph and the first drawing graph or not can be judged, and under the condition that the pixel points with the same coordinates exist in the prior drawing graph and the first drawing graph, the drawing graph with the overlapped part with the first drawing graph exists in the prior drawing graph is determined.
In one possible implementation, before the adjusting the first drawing graph, the method further includes: and receiving a first user instruction, wherein the first user instruction is used for indicating that the first drawing graph is adjusted.
The user instructs the terminal device or the server to adjust the first drawing graph by sending a first user instruction. Optionally, the user may issue the first user instruction by operating a specific control in an application program of the terminal device. For example, after the first drawing graph is drawn, the user may perform a specified operation on a specific control of the application program operation interface to issue a first user instruction, the control may be displayed on the interface in the form of a touch button, the specified operation may be a click operation or other operation, and after the specified operation is detected, the application program may be triggered to automatically adjust the first drawing graph. In some embodiments, the user may also issue the first user instruction through a network connection (e.g., a connection to the server via a specific URL link), or may issue the first user instruction in other ways, which is not limited by the embodiments of the present disclosure.
In some embodiments, after receiving the first user instruction, it may be detected whether a drawing pattern having an overlapping portion with the first drawing pattern exists in at least one previous drawing pattern in the target image, and in a case where it is detected that a drawing pattern having a partial overlapping portion with the first drawing pattern exists in the target image, the first drawing pattern may be automatically adjusted.
Or, in some embodiments, it may also be detected whether a drawing pattern that has a portion overlapping with the first drawing pattern exists in at least one previous drawing pattern in the target image, and when it is detected that a drawing pattern that has a portion overlapping with the first drawing pattern exists in the target image, a prompt is issued to the user in a user interface of the terminal device to prompt the user that a second drawing pattern overlapping with the first drawing pattern exists, and the user selects whether to automatically adjust the first drawing pattern, at this time, the user may issue a first user instruction to instruct to automatically adjust the first drawing pattern.
When it is detected that a drawing figure partially overlapping with the first drawing figure exists in the target image, the first drawing figure and the second drawing figure with the overlapping part can be displayed in a display style different from other drawing figures, so that the user can conveniently view the drawing figures. For example, the first drawing figure and the second drawing figure in which the overlapping portion exists may be displayed with a red border, and the other drawing figures may be displayed with a gray border.
In a possible implementation manner, the adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation includes: and removing the part of the first drawing graph, which is overlapped with the second drawing graph, from the first drawing graph to obtain a drawing graph, wherein the drawing graph is used as a labeling result corresponding to the first labeling operation.
In the embodiment of the present disclosure, the semantics of the image region indicated by the overlapping portion of the first drawing figure and the second drawing figure may be the same as the semantics of the image region indicated by the second drawing figure but different from the semantics of the image region other than the overlapping portion in the first image region. In this case, a portion of the first drawing graphic that overlaps the second drawing graphic may be removed from the first drawing graphic as a labeling result corresponding to the first labeling operation. The image area with the same semantic meaning as the second image area in the first image area is removed from the obtained labeling result, the semantic meaning of the first image area indicated by the obtained first drawing graph is more accurate, and the accuracy of the neural network obtained by machine learning can be improved. Meanwhile, the first image area does not need to be adjusted manually by a user, the user operation is less, the convenience is higher, the efficiency of labeling the target image is improved, and the labeling time is saved.
In a possible implementation manner, the adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation includes: under the condition that the first drawing graph and the second drawing graph are frame graphs, determining a first line segment element in the first drawing graph in the second drawing graph and a second line segment element outside the second drawing graph in the first drawing graph according to the positions of the first drawing graph and the second drawing graph; and taking a drawing graph formed by the first line segment element and the second line segment element as a labeling result corresponding to the first labeling operation.
The position of the drawing figure may be represented by the coordinates of line segment elements constituting the drawing figure, or may be represented by the coordinates of points constituting the drawing figure.
In some embodiments, the drawing graph may be a polygon, which may be considered a graph composed of line segment elements, the positions of which may be represented by the coordinates of the line segment endpoints, e.g., the line segment elements may be represented as (x)1,y1,x2,y2)，(x1,y1) And (x)2,y2) Respectively representing the coordinates of the two vertices of the line segment element. The coordinates of the plurality of line segment elements constituting the drawing figure may indicate the position of the drawing figure.
In some embodiments, the frame of the drawing graph is formed by a series of point sets, for example, the drawing graph can be regarded as a set formed by pixel points, and the coordinates of each pixel point forming the drawing graph can represent the position of the drawing graph.
In the process of drawing the graph, a user can detect the coordinates of a drawing tool used for drawing the graph in the drawing process in the target image, and the coordinates of pixel points and/or the coordinates of line segment element endpoints forming the drawn graph can be obtained. The drawing tool may be a tool such as a brush, etc., whose coordinates in the target image may be changed by moving a mouse, or may be changed by touching a display screen/touch panel, etc.
The first line segment element in the first drawing graph and the second line segment element in the first drawing graph, which is located outside the second drawing graph, in the second drawing graph can be determined according to the coordinates of the pixel points of the second drawing graph and the first drawing graph. Or, according to the coordinates of the end points of the line segment elements of the second drawing graph and the first drawing graph, determining a first line segment element in the second drawing graph, which is located in the first drawing graph, and a second line segment element in the first drawing graph, which is located outside the second drawing graph.
Specifically, the coordinate ranges of all the first pixels constituting the first drawing pattern may be recorded as (x)min～xmax，ymin～ymax) If the x value and the y value of the coordinate of a certain pixel point of the second drawing graph fall into the coordinate range, the pixel point can be determined to be positioned in the first drawing graph, and otherwise, the pixel point can be determined to be positioned outside the first drawing graph. Thus, a first line segment element in the second drawing graph, which is located inside the first drawing graph, and a second line segment element in the first drawing graph, which is located outside the second drawing graph, can be determined.
Or, if the x value and the y value of the coordinate of the end point of the line segment element of the second drawing figure both fall within the above coordinate range, it may be determined that the end point is located within the first drawing figure, otherwise, it may be determined that the end point is located outside the first drawing figure, and in addition, it may also be determined that the intersection point of the line segment elements of the first drawing figure and the second drawing figure is located outside the first drawing figure, and according to the intersection point, the end point of the line segment element of the second drawing figure located within the first drawing figure and the end point of the line segment element of the second drawing figure located outside the first drawing figure, it may be determined that the first line segment element located within the first drawing figure in the second drawing figure and the second line segment element located outside the second drawing figure in the first drawing figure are located.
After the first line segment element and the second line segment element are determined, a drawing graph formed by the first line segment element and the second line segment element can be used as a labeling result corresponding to the first labeling operation.
For example, as shown in FIG. 4 for the first drawing figure P1 and the second drawing figure P2, the set of line segment elements of P1 may be represented as { a }1a2,a2b3,b3a4,a4a3,a3b2,b2a1The set of line segment elements of P2 can be represented as b1b2,b2b3,b3b4,b4b5,b5b6,b6b7,b7b1In which a is1-a4And b1-b7Are the end points of the line segments. Wherein, the line segment element b of P13a4,a4a3,a3b2Located in P2, { b1b2,b3b4,b4b5,b5b6,b6b7,b7b1Located outside P2. As shown in FIG. 5, the line segment elements { b }3a4,a4a3,a3b2And { b }and1b2,b3b4,b4b5,b5b6,b6b7,b7b1The constructed drawing figure is used as the labeling result corresponding to the first labeling operation, and the set of line segment elements of the second drawing figure can be kept unchanged.
In the embodiment of the disclosure, under the condition that the first drawing graph and the second drawing graph are frame graphs, the first line segment element in the first drawing graph in the second drawing graph and the second line segment element outside the second drawing graph in the first drawing graph are determined according to the positions of the first drawing graph and the second drawing graph, so that the drawing graph formed by the first line segment element and the second line segment element can be used as a labeling result corresponding to the first labeling operation, the labeling result of the first labeling operation can be obtained quickly, the efficiency of the labeling operation is improved, and the user experience is better.
In one possible implementation manner, after the receiving the first annotation operation on the target image, the method further includes: displaying the target image and a plurality of drawing graphs including the first drawing graph of the target image on a display interface; receiving a second user instruction for selecting the first drawing figure from the plurality of drawing figures; and determining the selected first drawing graph according to the position indicated by the second user instruction in the display interface and the positions of the plurality of drawing graphs in the display interface.
And the user selects the first drawing graph from the plurality of drawing graphs by sending out a second user instruction. Optionally, the user may select the first drawing graph by performing a selection operation on the drawing graphs in the target image. Specifically, when the user triggers the second instruction, the position indicated by the mouse or the touch point in the display interface when the second instruction is triggered may be determined, and the positions of the plurality of drawing graphs in the display interface may also be determined after the graphs are drawn, and when it is determined that the position indicated by the second user instruction is located inside the first drawing graph, the selected first drawing graph may be determined. Thus, the first drawing figure selected by the second user instruction can be determined.
Alternatively, the user may select the first drawing pattern by selecting information such as an identifier of the drawing pattern, which is not described in detail in this disclosure.
In the embodiment of the disclosure, the user can realize the designated operation of the first drawing graph by sending the second user instruction, the user can independently designate the first drawing graph to be adjusted, the user can adjust the drawing graph required to be adjusted conveniently, the accuracy of the neural network obtained by machine learning can be improved, diversified user requirements can be met, and the user experience is good.
In the field of artificial intelligence, semantic segmentation is an important research content in the field of computer vision, and image semantic segmentation can be applied to numerous application scenarios, such as the fields of target recognition, target detection and the like. By semantic segmentation, an image can be segmented into regions with different semantic information, for example, semantic labels (such as sky, lawn, people, trees, various small animals, etc.) can be added to objects in the image after semantic segmentation is performed on one image.
In the process of labeling an image, a graphic is drawn on the image to indicate a labeled region, and a polygon is generally used to frame the regions with the same semantic meaning. In an image that needs to be semantically segmented, there are cases where a plurality of objects are mutually occluded, for example, the content of a target image is that a person stands on a lawn, and the legs of the person in the image occlude part of the lawn. The edge of the shielding part is the boundary of the semantic segmentation, and the semantics at two sides of the boundary are different. In the process of labeling semantic units with different semantics in an image, if regions on two sides of a boundary line are labeled respectively, the boundary line needs to be labeled twice, and since the shape of the boundary line is often irregular, the efficiency of labeling the image twice is low.
In an application scenario of semantic segmentation, the present disclosure provides a possible implementation manner, in which a first annotation operation is used to perform semantic segmentation annotation on the target image. The semantics of the image region indicated by the overlapping portion of the first drawing figure and the second drawing figure may be the same as those of the image region indicated by the second drawing figure but different from those of the image region other than the overlapping portion in the first image region. For example, referring to the above example, the first image area is an area where a lawn is located, but includes a leg portion that blocks the lawn, and the second image area is an area where a person in the image is located. In this case, the overlapping portion is removed from the first drawing pattern, the semantics of the first image region indicated by the obtained first drawing pattern are more accurate, and the accuracy of the neural network obtained by machine learning can be improved. Meanwhile, after the user draws the second drawing graph, under the condition that the user marks the boundary line between the two semantic units by drawing the second drawing graph, in the process of drawing the first drawing graph, the user does not need to draw the boundary line between the two semantic units again, and does not need to manually adjust the first drawing graph, so that the user operation is less, the convenience is higher, the efficiency of marking the target image is improved, and the marking time is saved.
In one possible implementation, the target image is displayed on a web interface, which may be, for example, a web interface encoded in the HTML5 language. Then, in the process of drawing the graphics, the graphics can be drawn through a Canvas (Canvas) in HTML5, thereby realizing annotation of the target image.
In the following, taking an example that a target image is displayed in a web interface, and taking a first annotation operation for performing semantic segmentation annotation on the target image as a specific application scenario of the present disclosure, an annotation method provided by the present disclosure is exemplarily described, and what is not described in detail in this section may refer to the foregoing related description, and similarly, what is in this section may also be used to exemplarily describe the foregoing content.
Referring to fig. 6, in a possible application scenario provided by the present disclosure, a labeling method provided by the present disclosure includes:
step 201, displaying the target image through the webpage.
At step 202, drawing graphics on the target image through a Canvas (Canvas) in HTML5, obtaining a plurality of drawing graphics.
The drawing graph can be a polygon, and the process of drawing the graph is as follows:
firstly, the creation of the polygon point is realized through a mouse button release event (mouseup) in the JavaScript language, specifically, after a user clicks a left button (presses the left button and then releases the left button), the canvas may create a starting point of a new graph or add a new polygon vertex to the newly created graph in response to a user operation that the left button is released, and after a user clicks a right button (presses the right button and then releases the right button), the canvas may connect the starting point of the graph and the terminal in response to the user operation that the right button is released, so as to realize the closing of the created polygon.
Capturing the position of the mouse on the browser through an e event attribute in a callback event of a mouse movement event (mousemove) in a JavaScript language to determine the position of each operation of the user on the target image, thereby determining the position of the drawing graph drawn by the user on the target image.
The drawing graphs comprise a plurality of graphs, and the drawing graphs are graphs drawn by the user on the areas in the target images to be labeled.
Step 203, receiving a selection operation instruction, and determining a first drawing graph according to the selection operation instruction.
The selection operation instruction is used to select a graphic to be adjusted from the plurality of drawn graphics, and the selection operation instruction is the second user instruction described above, where the graphic to be adjusted is referred to as a first drawn graphic.
According to the position indicated by the selection operation instruction in the webpage and the positions of the plurality of drawing graphs in the webpage, when the position indicated by the selection operation instruction in the webpage is determined to be located inside the first drawing graph, the selected first drawing graph can be determined.
Step 204, receiving an adjustment operation instruction.
The adjustment operation instruction is used to instruct to perform adjustment processing on the first drawing graph in the webpage, and the division operation instruction is the first user instruction described above.
Step 205, responding to the adjustment operation instruction, detecting whether a second drawing graph with an overlapped part with the first drawing graph exists in the plurality of drawing graphs in the target image.
For example, the content of the image is that a person stands in a lawn, the leg of the person in the image shields part of the lawn, the first drawing figure drawn by the user can be the area where the lawn is located, and because the edge of the lawn is generally straight and not irregular, the first drawing figure includes the area where the leg of the person in the lawn is located, and the second drawing figure can be the area where the person is located, in order to facilitate the user to draw the first drawing figure quickly.
And step 206, removing the overlapped part of the first drawing graph and the second drawing graph from the first drawing graph to obtain an adjustment result.
Under the condition that the first drawing graph is a frame graph, according to the positions of the first drawing graph and the second drawing graph, a first line segment element in the second drawing graph, which is positioned in the first drawing graph, and a line segment element in the first drawing graph, which is positioned outside the second drawing graph, can be determined; and then, taking a graph formed by the first line segment element and the line segment element positioned outside the second drawing graph in the first drawing graph as an adjustment result of the first drawing graph, wherein the adjusted first drawing graph is used for marking semantic information of the first image area in the target image.
In addition, according to the positions of the first drawing graph and the second drawing graph, a second line segment element in the second drawing graph in the first drawing graph is determined, and then the second line segment element is removed, so that no other line segment element exists in the second drawing graph.
Compared with the first drawing graph, the area in the second drawing graph is removed from the adjustment result of the first drawing graph and is used for labeling the semantic units in the target image, and when a user labels the boundary line between two semantic units by drawing the second drawing graph, the user does not need to draw the boundary line between the two semantic units in the process of drawing the other semantic unit, so that the labeling efficiency is improved, and the labeling time is saved.
In a possible implementation manner, the annotation method may be performed by an electronic device such as a terminal device or a server, the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, and the method may be implemented by a processor calling a computer readable instruction stored in a memory. Alternatively, the method may be performed by a server.
It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above methods of the specific embodiments, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.
In addition, the present disclosure also provides an annotation device, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any one of the annotation methods provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions in the method sections are referred to and are not described again.
Fig. 7 shows a block diagram of an annotation apparatus according to an embodiment of the present disclosure, which, as shown in fig. 7, includes:
a receiving module 301, configured to receive a first annotation operation on a target image, where the first annotation operation is used to draw a first drawing graph indicating a first image region in the target image;
an adjusting module 302, configured to adjust the first drawing graph when a second drawing graph in the target image is at least partially overlapped with the first drawing graph, so as to obtain a labeling result corresponding to the first labeling operation.
In one possible implementation, the apparatus further includes:
and the detection module is used for detecting whether a drawing graph which has an overlapped part with the first drawing graph exists in at least one previous drawing graph in the target image.
In one possible implementation, the apparatus further includes:
and the first user instruction receiving module is used for receiving a first user instruction, and the first user instruction is used for indicating to adjust the first drawing graph.
In a possible implementation manner, the adjusting module 302 is configured to remove, from the first drawing graph, a portion of the first drawing graph that overlaps with the second drawing graph, to obtain a drawing graph, as a labeling result corresponding to the first labeling operation.
In a possible implementation manner, the adjusting module 302 is configured to determine, according to positions of the first drawing graph and the second drawing graph, a first line segment element in the second drawing graph located inside the first drawing graph and a second line segment element in the first drawing graph located outside the second drawing graph in the first drawing graph, when the first drawing graph and the second drawing graph are border graphs; and taking a drawing graph formed by the first line segment element and the second line segment element as a labeling result corresponding to the first labeling operation.
In one possible implementation, the apparatus further includes:
the display module is used for displaying the target image and a plurality of drawing graphs including the first drawing graph of the target image on a display interface;
a second user instruction receiving module, configured to receive a second user instruction, where the second user instruction is used to select the first drawing graph from the plurality of drawing graphs;
and the determining module is used for determining the selected first drawing graph according to the position indicated by the second user instruction in the display interface and the positions of the plurality of drawing graphs in the display interface.
In one possible implementation, the target image is displayed on a web interface.
In one possible implementation manner, the first annotation operation is used for performing semantic segmentation annotation on the target image.
In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.
Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a non-volatile computer readable storage medium.
An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.
The embodiments of the present disclosure also provide a computer program product, which includes computer readable code, and when the computer readable code runs on a device, a processor in the device executes instructions for implementing the labeling method provided in any of the above embodiments.
The embodiments of the present disclosure also provide another computer program product for storing computer readable instructions, which when executed cause a computer to perform the operations of the annotation method provided in any of the above embodiments.
The electronic device may be provided as a terminal, server, or other form of device.
Fig. 8 illustrates a block diagram of an electronic device 800 in accordance with an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal.
Referring to fig. 8, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.
The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.
The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.
The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.
In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.
Fig. 9 illustrates a block diagram of an electronic device 1900 in accordance with an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 9, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932TM) Graph-based drawing from apple IncGraphic user interface operating system (Mac OS X)TM) Multi-user, multi-process computer operating system (Unix)TM) Free and open native code Unix-like operating System (Linux)TM) Open native code Unix-like operating System (FreeBSD)TM) Or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.
The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.
The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.
The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.
The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (11)

1. A method of labeling, comprising:
receiving a first annotation operation on a target image, wherein the first annotation operation is used for drawing a first drawing graph indicating a first image area in the target image;
and under the condition that at least part of a second drawing graph in the target image is overlapped with the first drawing graph, adjusting the first drawing graph to obtain a labeling result corresponding to the first labeling operation.
2. The method of claim 1, wherein prior to adjusting the first rendered graphic, the method further comprises:
and detecting whether a drawing graph with an overlapped part with the first drawing graph exists in at least one previous drawing graph in the target image.
3. The method according to any one of claims 1 or 2, wherein prior to the adjusting the first rendered graphic, the method further comprises:
and receiving a first user instruction, wherein the first user instruction is used for indicating that the first drawing graph is adjusted.
4. The method according to any one of claims 1 to 3, wherein the adjusting the first drawing graph to obtain the labeling result corresponding to the first labeling operation includes:
and removing the part of the first drawing graph, which is overlapped with the second drawing graph, from the first drawing graph to obtain a drawing graph, wherein the drawing graph is used as a labeling result corresponding to the first labeling operation.
5. The method according to any one of claims 1 to 4, wherein the adjusting the first drawing graph to obtain the labeling result corresponding to the first labeling operation includes:
under the condition that the first drawing graph and the second drawing graph are frame graphs, determining a first line segment element in the first drawing graph in the second drawing graph and a second line segment element outside the second drawing graph in the first drawing graph according to the positions of the first drawing graph and the second drawing graph;
and taking a drawing graph formed by the first line segment element and the second line segment element as a labeling result corresponding to the first labeling operation.
6. The method according to any one of claims 1-5, wherein after said receiving a first annotation operation on a target image, the method further comprises:
displaying the target image and a plurality of drawing graphs including the first drawing graph of the target image on a display interface;
receiving a second user instruction for selecting the first drawing figure from the plurality of drawing figures;
and determining the selected first drawing graph according to the position indicated by the second user instruction in the display interface and the positions of the plurality of drawing graphs in the display interface.
7. The method of any of claims 1-6, wherein the target image is displayed on a web interface.
8. The method of any of claims 1-7, wherein the first annotation operation is used to semantically segment and annotate the target image.
9. A marking device, comprising:
the system comprises a receiving module, a judging module and a display module, wherein the receiving module is used for receiving a first annotation operation on a target image, and the first annotation operation is used for drawing a first drawing graph indicating a first image area in the target image;
and the adjusting module is used for adjusting the first drawing graph under the condition that at least part of the second drawing graph in the target image is overlapped with the first drawing graph to obtain a labeling result corresponding to the first labeling operation.
10. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to invoke the memory-stored instructions to perform the method of any one of claims 1 to 8.
11. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 8.
CN202011529117.0A 2020-12-22 2020-12-22 Labeling method and device, electronic equipment and storage medium Pending CN112508020A (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
CN202011529117.0A CN112508020A (en) 2020-12-22 2020-12-22 Labeling method and device, electronic equipment and storage medium
JP2022520125A JP2023510443A (en) 2020-12-22 2021-04-14 Labeling method and device, electronic device and storage medium
PCT/CN2021/087285 WO2022134390A1 (en) 2020-12-22 2021-04-14 Labeling method and apparatus, electronic device, and storage medium
KR1020227009804A KR20220093091A (en) 2020-12-22 2021-04-14 Labeling method and apparatus, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011529117.0A CN112508020A (en) 2020-12-22 2020-12-22 Labeling method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112508020A true CN112508020A (en) 2021-03-16

Family

ID=74923349

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011529117.0A Pending CN112508020A (en) 2020-12-22 2020-12-22 Labeling method and device, electronic equipment and storage medium

Country Status (4)

Country Link
JP (1) JP2023510443A (en)
KR (1) KR20220093091A (en)
CN (1) CN112508020A (en)
WO (1) WO2022134390A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022134390A1 (en) * 2020-12-22 2022-06-30 深圳市商汤科技有限公司 Labeling method and apparatus, electronic device, and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115344937B (en) * 2022-10-20 2023-03-17 三一筑工科技股份有限公司 Reinforcing steel bar marking method and device, electronic equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110286992A (en) * 2019-07-02 2019-09-27 中国工商银行股份有限公司 The method and device that interface figure is redrawn
CN110660313A (en) * 2018-06-29 2020-01-07 百度在线网络技术（北京）有限公司 Information presentation method and device
CN110796201A (en) * 2019-10-31 2020-02-14 深圳前海达闼云端智能科技有限公司 Method for correcting label frame, electronic equipment and storage medium
US20200098116A1 (en) * 2018-09-26 2020-03-26 International Business Machines Corporation Image labeling
CN111027543A (en) * 2019-11-25 2020-04-17 北京云测信息技术有限公司 Image annotation method and device
CN111666936A (en) * 2019-03-08 2020-09-15 北京市商汤科技开发有限公司 Labeling method, labeling device, labeling system, electronic equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111814885B (en) * 2020-07-10 2021-06-22 云从科技集团股份有限公司 Method, system, device and medium for managing image frames
CN112508020A (en) * 2020-12-22 2021-03-16 深圳市商汤科技有限公司 Labeling method and device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110660313A (en) * 2018-06-29 2020-01-07 百度在线网络技术（北京）有限公司 Information presentation method and device
US20200098116A1 (en) * 2018-09-26 2020-03-26 International Business Machines Corporation Image labeling
CN111666936A (en) * 2019-03-08 2020-09-15 北京市商汤科技开发有限公司 Labeling method, labeling device, labeling system, electronic equipment and storage medium
CN110286992A (en) * 2019-07-02 2019-09-27 中国工商银行股份有限公司 The method and device that interface figure is redrawn
CN110796201A (en) * 2019-10-31 2020-02-14 深圳前海达闼云端智能科技有限公司 Method for correcting label frame, electronic equipment and storage medium
CN111027543A (en) * 2019-11-25 2020-04-17 北京云测信息技术有限公司 Image annotation method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022134390A1 (en) * 2020-12-22 2022-06-30 深圳市商汤科技有限公司 Labeling method and apparatus, electronic device, and storage medium

Also Published As

Publication number Publication date
WO2022134390A1 (en) 2022-06-30
JP2023510443A (en) 2023-03-14
KR20220093091A (en) 2022-07-05

Similar Documents

Publication Publication Date Title
US10942616B2 (en) Multimedia resource management method and apparatus, and storage medium
EP4246447A2 (en) Method and device for dynamically displaying icon according to background image
CN112465843A (en) Image segmentation method and device, electronic equipment and storage medium
US20200007944A1 (en) Method and apparatus for displaying interactive attributes during multimedia playback
WO2016192325A1 (en) Method and device for processing logo on video file
CN107122113B (en) Method and device for generating picture
EP3147802B1 (en) Method and apparatus for processing information
EP2921969A1 (en) Method and apparatus for centering and zooming webpage and electronic device
CN112991553B (en) Information display method and device, electronic equipment and storage medium
CN110968364B (en) Method and device for adding shortcut plugins and intelligent device
CN109285126B (en) Image processing method and device, electronic equipment and storage medium
CN110989905A (en) Information processing method and device, electronic equipment and storage medium
CN112907760A (en) Three-dimensional object labeling method and device, tool, electronic equipment and storage medium
CN113065591B (en) Target detection method and device, electronic equipment and storage medium
CN113806054A (en) Task processing method and device, electronic equipment and storage medium
CN112508020A (en) Labeling method and device, electronic equipment and storage medium
CN112860061A (en) Scene image display method and device, electronic equipment and storage medium
CN110209445B (en) Information reminding method, device, terminal and storage medium
CN106020694B (en) Electronic equipment, and method and device for dynamically adjusting selected area
CN112269620A (en) Display method and device, electronic equipment and storage medium
CN108829473B (en) Event response method, device and storage medium
US20220301220A1 (en) Method and device for displaying target object, electronic device, and storage medium
CN113869295A (en) Object detection method and device, electronic equipment and storage medium
CN114266305A (en) Object identification method and device, electronic equipment and storage medium
CN114116106A (en) Chart display method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40039778

Country of ref document: HK