WO2021082507A1

WO2021082507A1 - Semi-automated image segmentation and data annotation method, electronic device and storage medium

Info

Publication number: WO2021082507A1
Application number: PCT/CN2020/100347
Authority: WO
Inventors: 邓辅秦; 黄永深; 彭健烽; 冯华; 陈颖颖; 李伟科
Original assignee: 五邑大学
Priority date: 2019-10-31
Filing date: 2020-07-06
Publication date: 2021-05-06
Also published as: CN110910401A

Abstract

A semi-automated image segmentation and data annotation method, an electronic device and a storage medium. The method comprises: on the basis multiple first pixel coordinates located in a real target area and multiple second pixel coordinates located in a real background area selected by a user on an image to be annotated, and on the basis of an energy function, determining whether each pixel in the image belongs to the real target area or the real background area; outputting the outermost coordinates of a predicted target area; and the user then inputs category information corresponding to the predicted target area, and the image annotation task can then be completed. The method simplifies the number of mouse clicks during image annotation, reduces the cost of manual annotation, and improves the efficiency of manual annotation.

Description

Semi-automatic image segmentation data labeling method, electronic device and storage medium

Technical field

The present invention relates to the technical field of image processing, in particular to a semi-automatic image segmentation data labeling method, electronic device and storage medium.

Background technique

Deep learning-based image segmentation algorithms are widely used in a series of applications that require precise identification of the category and location of objects, such as garbage classification systems, autonomous driving, and processing defect detection systems. Then, image segmentation algorithms based on deep learning require a large amount of manually labeled data. Come to train. At present, the main method of image segmentation data is based on the edge of the target object, which is observed by the naked eye and manually judged, and the mouse clicks and labels point by point.

Therefore, in the prior art, in order to meet the needs of image annotation data for the training of image segmentation networks based on deep learning, a technical crowdsourcing platform came into being. Some companies recruit tens of thousands of data annotators, but due to such data annotation methods The method of data labeling by visual observation and manual judgment requires manual mouse clicks for dozens or even hundreds of times when labeling an image, which is inefficient.

Summary of the invention

The present invention aims to solve at least one of the technical problems existing in the prior art. For this reason, the present invention provides a semi-automatic image segmentation data labeling method, electronic device and storage medium, which can reduce the number of manual mouse clicks during image labeling and improve labeling efficiency.

In the first aspect, an embodiment of the present invention provides a semi-automatic image segmentation data labeling method, including the following steps:

Step S1: Display an image to be annotated, where the image to be annotated includes a real target area and a real background area;

Step S2: Obtain a plurality of first pixel coordinates located in the real target area, and generate a target area seed point set from the plurality of first pixel coordinates;

Step S3: Obtain a plurality of second pixel coordinates located in the real background area, and generate a background area seed point set from the plurality of second pixel coordinates;

Step S4: Establish a target seed gray value histogram based on the target area seed point set;

Step S5: Establish a background seed gray value histogram based on the background area seed point set;

Step S6: establishing an undirected graph representing the image to be labeled, constructing an energy function, performing image segmentation on the image to be labeled based on the minimum cut algorithm, and obtaining a binarized image after the image to be labeled is segmented, The binarized picture includes a predicted target area and a predicted background area;

Step S7: Obtain the outermost coordinate points of the predicted target area on the binarized image based on an edge tracking algorithm, and generate an edge coordinate set from a plurality of the outermost coordinate points;

Step S8: Generate a contour of the prediction target area based on the edge coordinate set, and highlight the contour on the image to be annotated;

Step S9: Determine whether a complete selection instruction is received, if yes, obtain the category information of the target area, and save the edge coordinate set and the category information as a json file; if not, return to step S1.

A semi-automatic image segmentation data labeling method according to an embodiment of the present invention has at least the following beneficial effects: the method is based on the multiple first pixel coordinates selected by the user and located in the real target area on the image to be labelled. The multiple second pixel coordinates in the real background area are judged based on the energy function whether each pixel in the image to be labeled belongs to the real target area or the real background area, and the outermost coordinates of the predicted target area are output, and then input by the user The category information corresponding to the predicted target area can complete the image labeling task. Therefore, the semi-automatic image segmentation data labeling method provided in this embodiment greatly simplifies the number of mouse clicks during image labeling, reduces the cost of manual labeling, and speeds up the efficiency of manual labeling.

In another specific embodiment of the present invention, "generating a plurality of the outermost coordinate points into an edge coordinate set" in the step S7 further includes the following steps:

Step S7.1: Create a set A of all the outermost coordinate points, establish a set A', add any one of the outermost coordinate points p ₀ in the set A to the set A', and add all the outermost coordinate points in the set A except The outermost coordinate points other than p ₀ _{establish a set A 2} , and the first marked coordinate point p is set to p ₀ ;

Step S7.2: Determine whether the _{number of elements in the set A 2} is zero, if not, execute step S7.2a, if yes, execute step S7.2b;

Step S7.2a: Calculate _{the distance d between all the outermost coordinate points in the set A 2} and the first mark coordinate point p, and set the first mark coordinate point p to the minimum value of the distance d in the _{set A 2} The outermost coordinate point, the outermost coordinate point corresponding to the minimum distance d in the _{set A 2} _{is added to the set A'and deleted from the set A 2} , and the step S7.2 is returned;

Step S7.2b: Sort the outermost coordinate points according to the order in which the outermost coordinate points are added to the set A';

Step S7.3: Establish an edge coordinate set, add p ₀ to the edge coordinate set and delete it from the set A', and set the second mark coordinate point _{p'to p 0} ;

Step S7.4: Determine whether the number of elements in the set A'is one, if not, execute step S7.4a, if yes, execute step S7.4b;

Step S7.4a: Determine whether the second mark coordinate point p'and the two outermost coordinate points ranked in the first two positions in the set A'are three-point collinear, if yes, go to step S7.4a1, if not, Step S7.4a2 is executed;

Step S7.4a1: delete the outermost coordinate point that is ranked first in the set A'from the set A', and return to step S7.4;

Step S7.4a2: Set the second mark coordinate point p'as the outermost coordinate point ranked at the top in the set A', and add the outermost coordinate point ranked at the top in the set A' To the edge coordinate set and delete from the set A', return to step S7.4;

Step S7.4b: Add the outermost coordinate points in the set A'to the edge coordinate set, and output the edge coordinate set.

In another specific embodiment of the present invention, the step S8 further includes:

Step S8.4: Perform shadow processing on the prediction target area on the image to be labeled.

In a second aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and capable of running on the processor. When the processor executes the program, the implementation is as follows: The semi-automatic image segmentation data labeling method described in any one of the first aspect of the present invention.

Since an electronic device of an embodiment of the present invention executes the semi-automatic image segmentation data labeling method according to any one of the first aspect of the present invention, it has all the beneficial effects of the first aspect of the present invention.

In a third aspect, an embodiment of the present invention provides a computer-readable storage medium that stores computer-executable instructions, and is characterized in that: the computer-executable instructions are used to execute the semiconductor device according to any one of the first aspects of the present invention. Automatic image segmentation data labeling method.

Since the computer-readable storage medium of the embodiment of the present invention stores computer-executable instructions for executing the semi-automated image segmentation data labeling method according to any one of the first aspect of the present invention, it has the advantages of the first aspect of the present invention. All beneficial effects.

The additional aspects and advantages of the present invention will be partly given in the following description, and partly will become obvious from the following description, or be understood through the practice of the present invention.

Description of the drawings

The above and/or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, in which:

Fig. 1 is a flowchart of a method according to a second embodiment of the present invention;

2 is an effect diagram of a semi-automatic image segmentation data labeling method according to an embodiment of the present invention;

3 is a schematic diagram of the principle of the graph segmentation algorithm according to the second embodiment of the present invention;

4 is a schematic diagram of the structure of the electronic device according to the first embodiment of the present invention;

Reference signs:

The electronic device 100, the processor 101, and the memory 102.

Detailed ways

The embodiments of the present invention are described in detail below. The embodiments described below with reference to the accompanying drawings are exemplary, and are only used for explaining the present invention, and should not be construed as limiting the present invention.

In the description of the present invention, several means one or more, multiple means two or more, greater than, less than, exceeding, etc. are understood to not include the number, and above, below, and within are understood to include the number. If it is described that the first and second are only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying the relative importance or implicitly specifying the number of the indicated technical features or implicitly specifying the order of the indicated technical features relationship.

The existing data labeling method is to use open source image segmentation data labeling tools (such as Labelme, Labelimg, etc.) to mark and plot the edges of the target image (by clicking the mouse to draw the edge of the target object in the image, if you want to Dotting and plotting of complex target objects may require hundreds of clicks of the mouse), and then separate the target image from the background, that is, segment the image to obtain the target image, and then perform data annotation on the obtained target image. If the shape of the target image is complex, it may be necessary to plot hundreds of points, that is, the number of clicks of the mouse may be as high as hundreds of times, which will make the eyes of the data annotator fatigue. If the shape of the target image is complex, the hand of the data annotator will also experience fatigue and soreness due to long-time mouse clicks. Such a data labeling method is inefficient. When there are many target images that need to be labelled or the shape of the target image is complex, the image segmentation time will be very long, which will seriously affect the speed of data labeling, resulting in low data labeling efficiency.

Based on this, the present invention provides a semi-automatic image segmentation data labeling method, electronic device and storage medium. By displaying the image to be labelled on the electronic screen, the user uses the mouse to obtain the image to be labelled twice on the image to be labelled. The multiple first pixel coordinates in the real target area and multiple second pixel coordinates in the real background area in, the energy function is constructed, and the image to be labeled is segmented based on the minimum cut algorithm to obtain a prediction corresponding to the real target area The outermost coordinate points of the target area, and finally the labeler judges the difference between the predicted target area and the real target area, and judges whether to enter the "full selection instruction", so that the computer finally saves the edge coordinate set and the category information as a json file , Complete the final annotation. Therefore, the present invention can reduce the number of clicks of the mouse when annotator mark data, and allows annotator to choose whether to accept the result of this automatic segmentation by inputting a "full selection instruction", which not only improves the efficiency of annotation, but also ensures the annotation The precision.

Many different embodiments or examples are provided below to implement different solutions of the present invention. It should be understood that the following descriptions are only exemplary descriptions, rather than specific limitations on the invention.

Referring to FIG. 4, an electronic device 100 provided by the first embodiment of the present invention includes a memory 102 and a processor 101. In FIG. 4, a processor 101 and a memory 102 are taken as an example.

The processor and the memory may be connected through a bus or in other ways. In FIG. 4, the connection through a bus is taken as an example.

As a non-transitory computer-readable storage medium, the memory 102 can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 102 may optionally include a memory 102 remotely provided with respect to the processor, and these remote memories may be connected to the electronic device 100 via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

Those skilled in the art can understand that the device structure shown in FIG. 4 does not constitute a limitation on the electronic device 100, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different component arrangement.

1 and 4 show the electronic device according to the first embodiment of the present invention. In this embodiment, the processor 101 in the electronic device 100 can be used to call the semi-automatic image segmentation data labeling method stored in the memory 102, And perform the following steps:

Based on the hardware structure of the above electronic device, various embodiments of a semi-automatic image segmentation data labeling method of the present invention are proposed.

Referring to FIG. 1, it is a semi-automatic image segmentation data labeling method according to a second embodiment of the present invention, which includes the following steps:

In this embodiment, the image to be annotated can be displayed to the annotator through a PC computer screen or a tablet computer screen, and the image to be annotated includes a real target area and a real background area. For example, if there are lawns and footballs in the image to be labeled, the tagger judges that the footballs belong to the real target area and the lawns belong to the real background area. The method of obtaining the first pixel coordinates is to use the labeler to press and hold down the left mouse button on the PC and drag in the real target area to obtain the pixels of the real target area that the mouse drags through as the first pixel coordinates. In order to facilitate feedback to the annotator, the first pixel can be displayed in the real target area. After the first pixel coordinate is obtained, the second pixel coordinate is obtained in the real background area in the same manner.

In this embodiment, first, an undirected graph G=<V, E> is used to represent the image to be divided (ie, the image to be labeled), and V and E are sets of vertices and edges, respectively. The picture here is slightly different from the normal picture. An ordinary graph is composed of vertices and edges. If the edges are directional, such a graph is called a directed graph, otherwise it is an undirected graph, and the edges are weighted. Different edges can have different weights. , Respectively represent different physical meanings. However, the graph in this embodiment has two more vertices on the basis of the ordinary graph, and these two vertices are represented by the symbols "S" and "T", and are collectively referred to as terminal vertices. All other vertices must be connected to these 2 vertices to form part of the edge set. Therefore, there are two types of vertices and two types of edges in the graph used to represent the image to be annotated in this embodiment.

The first type of vertices and edges are: the first type of ordinary vertices corresponds to each pixel in the image. The connection of every two neighboring vertices (corresponding to every two neighboring pixels in the image) is an edge. This kind of edge is also called n-links.

The second type of vertices and edges are: In addition to image pixels, there are two other terminal vertices, called S and T respectively. There is a connection between each ordinary vertex and these 2 terminal vertices, forming the second type of edge. Such edges are also called t-links.

Figure 3 shows the s-t graph corresponding to the image, each pixel corresponds to a corresponding vertex in the graph, and there are two vertices s and t. The graph in Figure 3 has two kinds of edges, one of which represents the n-links that connect every two ordinary vertices in the neighborhood, and the other represents the t-links that connects every ordinary vertex to s and t. In the foreground and background segmentation, s generally represents the foreground target (ie, the real target area), and t generally represents the background (ie, the real background area).

Each edge of the graph in Figure 3 has a non-negative weight we, which can also be understood as a cost. A cut is a subset C of the edge set E in the graph, and the cost of this cut (represented as |C|) is the sum of the weights of all the edges of the edge subset C.

This embodiment needs to require such a set of edges. This set of edges includes the above two types of edges. The disconnection of all edges in the set will result in the separation of the remaining "S" and "T" graphs, so it is called this The set of edges is "cut". If a cut has the smallest sum of the ownership values of its edges, then this is called the minimum cut, which is the result of the graph cut. The Ford-Folkson theorem shows that the maximum flow max flow of the network is equal to the minimum cut min cut. Therefore, the max-flow/min-cut algorithm invented by Boykov and Kolmogorov can be used to obtain the minimum cut of the s-t graph. This minimum cut divides the vertices of the graph into two disjoint subsets S and T, where s∈S, t∈T and S∪T=V. These two subsets correspond to the foreground pixel set (ie, the real target area) and the background pixel set (ie, the real background area) of the image, which is equivalent to completing the image segmentation.

The weight of the edge in the graph determines the final segmentation result, and the weight of the edge can be determined by the energy function. Image segmentation can be regarded as a pixel labeling problem. The label of the target (s-node) is set to 1, and the label of the background (t-node) is set to 0. This process can be obtained by minimizing the graph cut to minimize the energy function. It is obvious that the cut that occurs at the boundary between the target and the background is what we want (equivalent to cutting off the connection between the background and the target in the image, which is equivalent to dividing it). At the same time, the energy should also be minimal at this time. Suppose the label of each pixel of the entire image is L={l ₁ ,l ₂ ,,,l _p }, where l _i is 0 (indicating that the pixel is located in the real background area) or 1 (indicating that the pixel is located in the real background area) target area). Assuming that the segmentation of the image is L, the energy of the image can be expressed as:

E(L)=aR(L)+B(L)

Among them, R(L) is the regional term, B(L) is the boundary term, and a is an important factor between the regional term and the boundary term, which determines their influence on energy. If a is 0, then only boundary factors are considered, and regional factors are not considered. E(L) represents the weight, that is, the loss function, also called the energy function. The goal of the graph cut is to optimize the energy function to minimize its value.

For area items,

Among them, R _p (l _p ) represents the penalty for assigning label l _p _{to pixel p, and the weight of the energy term of R p} (l _p ) can be obtained by comparing the gray level of pixel p with the gray histogram of the given target and foreground In other words, it is the probability that the pixel p belongs to the label l _p . I want the pixel p to be assigned to its label l _p with the highest probability. At this time, we want the energy to be the smallest, so we generally take the negative logarithm of the probability, so the weight of t-link The values are as follows:

R _p (1)=-ln Pr(L _p |'obj'); R _p (0)=-ln Pr(L _p |'bkg')

It can be seen from the above two formulas that when the probability that the gray value of the pixel p belongs to the target Pr(Ip|'obj') is greater than the background Pr(L _p |'bkg'), then R _p (1) is less than R _p (0) _{, which} means that when the pixel p is more likely to belong to the target _, classifying p as the target will make the energy R(L) small. Then, if all pixels are correctly divided into targets or backgrounds, then the energy is the smallest at this time.

For the boundary term,

Among them, p and q are neighborhood pixels, and the boundary term mainly reflects the boundary attributes of segmenting L. B<p,q> can be parsed as a penalty for discontinuity between pixels p and q. Generally speaking, if p and q are more similar ( For example, their gray scale), then the _{larger B <p,q>} , if they are very different, then B _<p,q> is close to 0. In other words, if the difference between the two neighboring pixels is very small, then it is likely to belong to the same target or the same background. If they are very different, it means that the two pixels are likely to be between the target and the background. The edge part is more likely to be divided, so when the difference between the two neighboring pixels is larger, the _{smaller B<p,q>} , that is, the smaller the energy.

In this embodiment, an image is divided into two disjoint parts of the target and the background, and the image segmentation technology is used to achieve this. First, the graph is composed of vertices and edges, and edges have weights. Then we need to build a graph, this graph has two types of vertices, two types of edges and two types of weights. An ordinary vertex is composed of each pixel of the image, and then there is an edge between every two neighboring pixels, and its weight is determined by the "boundary term" mentioned above. There are also two terminal vertices s (target) and t (background). Every common vertex and s have a connection, that is, an edge. The weight of the edge is determined by the "regional energy term" R _p (1), each The weight of the edge connecting the common vertex and t is determined by the "regional energy term" R _p (0). In this way, the weights of all edges can be determined, that is, the graph is determined. At this time, the min cut (minimum cut) algorithm can be used to find the smallest cut. This min cut is the set of weights and the smallest edges. The disconnection of these edges can just cause the target and the background to be separated, that is, min. Cut corresponds to the minimization of energy. The min cut and the max flow (minimum cut) of the graph are equivalent, so the min cut of the st graph can be found through the max flow algorithm.

In this embodiment, a graph structure is used to represent the image to be labeled, an energy function is constructed, and the image to be labeled is segmented based on the minimum cut or maximum flow algorithm. And after the image is segmented, the image will be divided into two parts, namely the prediction target area and the prediction background area, and then all the pixels in the prediction target area are set to black, and all the pixels in the predicted background area are set to white, so all pixels of the image to be labeled It is divided into two values, black and white, which is the binary image. According to these data, an edge tracking algorithm is used on the binarized picture to obtain the outermost coordinate points of the prediction target area on the binarized picture.

Since the graph segmentation algorithm is used in this embodiment to automatically obtain the outermost coordinate points of the predicted target area, in order to facilitate the annotator to determine the accuracy of the outermost coordinate points of the predicted target area obtained by this graph segmentation, it is based on the edge The coordinate set generates the contour of the prediction target area, and the contour is highlighted on the to-be-labeled image, which is convenient for the labeler to compare.

When the annotator thinks that the predicted target area obtained this time is ideal, he can send a "full selection instruction" to the electronic device in the first embodiment. For example, the instruction can be sent by hitting the enter key on the keyboard. Next, the display screen will display an interface for inputting category information. The labeler inputs category information corresponding to the predicted target area through the keyboard, such as football, and then the edge coordinate set and the category information are saved as a json file to complete the semi-automatic labeling. In another case, when the annotator thinks that the predicted target area obtained this time is not ideal, he can send an "incomplete selection instruction" to the electronic device in the first embodiment. For example, the annotator can press the space bar on the keyboard to The electronic device sends the instruction, and after receiving the instruction, the electronic device executes step S1 again. Therefore, the semi-automatic image segmentation data labeling method provided in this embodiment improves labeling efficiency while also allowing labelers to control the accuracy of labeling.

In the semi-automatic image segmentation data labeling method of the third embodiment of the present invention, on the basis of the second embodiment, the step S7 of "generating an edge coordinate set from a plurality of the outermost coordinate points" further includes the following steps :

Step S7.1: Create a set A of all the outermost coordinate points, establish a set A', add any one of the outermost coordinate points p ₀ in the set A to the set A', except for the set A The outermost coordinate points other than p ₀ _{establish a set A 2} , and the first marked coordinate point p is set to p ₀ ;

Step S7.4a: Determine whether the second mark coordinate point p'and the two outermost coordinate points in the top two positions in the set A'are three-point collinear, if yes, perform step S7.4a1, if not, Step S7.4a2 is executed;

Step S7.4a2: Set the second marking coordinate point p'as the outermost coordinate point ranked at the top in the set A', and add the outermost coordinate point ranked at the top in the set A' To the edge coordinate set and delete from the set A', return to step S7.4;

In purely manual labeling, for the case where part of the outline of the real target area is a straight line, for example, for the case where the real target area is a square, the annotator generally only uses the mouse to click on the four vertices of the square, and one of the two adjacent vertices Draw a straight line between. Therefore, only four pixels are needed to represent the coordinates of the square, which greatly reduces the amount of data. When the semi-automatic labeling method is used, since the edge coordinates of the predicted target area are obtained by the edge tracking algorithm, they are composed of a series of pixels that are neighbors, resulting in a large amount of data.

Based on this, this embodiment provides a simplified algorithm for obtaining the outermost coordinate points of the predicted target area. The algorithm includes two parts. The first part is steps S7.1 to S7.2. These steps are performed in the order in which the outermost coordinate points of the obtained prediction target area are added to the set A'. Sort. If the outermost coordinate points are passed through the outermost coordinate points in the order in which the outermost coordinate points are added to the set A′, it will just form the contour of the prediction target area. Therefore, the second part consisting of step S7.3 to step S7.4 is to check whether the three adjacent points on the contour are collinear in the order in which the outermost coordinate points are added to the set A'. , If collinear, remove the middle point, only keep the first and last two points, realize the effect of manual labeling, and reduce the amount of data generated by semi-automatic labeling.

The semi-automatic image segmentation data labeling method of the fourth embodiment of the present invention is based on the second embodiment and the third embodiment. In the step S8, "generate the contour of the prediction target area based on the edge coordinate set, And highlighting the outline on the image to be annotated," also includes the following steps:

Step S8.1: On the image to be annotated, the two outermost coordinate points that are added to the edge coordinate set in an adjacent order are connected by a straight line;

Step S8.2: On the image to be annotated, the outermost coordinate point added to the edge coordinate set in the last order is connected with p ₀ by a straight line;

Step S8.3: Generate the contour from the pixels passing by the straight line on the image to be annotated, and highlight the pixels corresponding to the straight line.

Based on the third embodiment, it can be known that passing through these outermost coordinate points in the order in which the outermost coordinate points are added to the set A′ just encloses the contour of the prediction target area. Therefore, while using the third embodiment to reduce the amount of semi-automatic labeling data, it also facilitates the generation of the contour of the predicted target area, reduces the calculation time when generating the contour, and improves the efficiency of the algorithm. At the same time, the pixels of the outline are increased in brightness and white in color, which is convenient for the user to identify the edge of the currently selected area.

The semi-automatic image segmentation data labeling method of the fifth embodiment of the present invention is based on the fourth embodiment, and the step S8 further includes:

In this embodiment, shadow processing is performed on the predicted target area, and the output is a darkened image of the local area. The function is to facilitate the user to identify the selected local area in the target object.

The computer-readable storage medium of the fifth embodiment of the present invention stores computer-executable instructions for executing the semi-automated image segmentation data labeling according to any one of the second to fifth embodiments above method.

Referring to FIG. 2, it is an effect diagram of image processing to be labeled using the semi-automatic image segmentation data labeling method of an embodiment of the present invention.

In the first step, the image to be labeled is displayed on the computer screen. The real target area on the image to be labeled is a football, and the real background area is a lawn.

In the second step, the marker moves the mouse on the football, clicks the left button of the mouse, and then drags the mouse to draw a stroke on the football;

In the third step, the marker moves the mouse to the lawn, clicks the left button of the mouse, and then drags the mouse to draw a stroke on the lawn;

The fourth step is to automatically obtain the contour coordinates of the football through image segmentation and process the shadow of the football;

In the fifth step, by executing the simplified algorithm of the third embodiment of the present invention, the points where the football contour coordinates automatically obtained by the graph segmentation are located on the same execution are merged, thereby reducing the amount of data.

The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above-mentioned embodiments, and within the scope of knowledge possessed by a person of ordinary skill in the technical field, it is possible to make various changes without departing from the purpose of the present invention. Kind of change.

Claims

A semi-automatic image segmentation data labeling method is characterized in that it comprises the following steps:

Step S1: Display an image to be annotated, where the image to be annotated includes a real target area and a real background area;

Step S2: Obtain a plurality of first pixel coordinates located in the real target area, and generate a target area seed point set from the plurality of first pixel coordinates;

Step S3: Obtain a plurality of second pixel coordinates located in the real background area, and generate a background area seed point set from the plurality of second pixel coordinates;

Step S4: Establish a target seed gray value histogram based on the target area seed point set;

Step S5: Establish a background seed gray value histogram based on the background area seed point set;

Step S6: establishing an undirected graph representing the image to be labeled, constructing an energy function, performing image segmentation on the image to be labeled based on the minimum cut algorithm, and obtaining a binarized image after the image to be labeled is segmented, The binarized picture includes a predicted target area and a predicted background area;

Step S7: Obtain the outermost coordinate points of the predicted target area on the binarized image based on an edge tracking algorithm, and generate an edge coordinate set from a plurality of the outermost coordinate points;

Step S8: Generate a contour of the prediction target area based on the edge coordinate set, and highlight the contour on the image to be annotated;

Step S9: Determine whether a complete selection instruction is received, if yes, obtain the category information of the target area, and save the edge coordinate set and the category information as a json file; if not, return to step S1.
A semi-automatic image segmentation data labeling method according to claim 1, wherein the step S7 of "generating a plurality of outermost coordinate points into an edge coordinate set" further comprises the following steps:

Step S7.1: Create a set A of all the outermost coordinate points, establish a set A', add any one of the outermost coordinate points p 0 in the set A to the set A', except for the set A The outermost coordinate points other than p 0 establish a set A 2 , and the first marked coordinate point p is set to p 0 ;

Step S7.2: Determine whether the number of elements in the set A 2 is zero, if not, execute step S7.2a, if yes, execute step S7.2b;

Step S7.2a: Calculate the distance d between all the outermost coordinate points in the set A 2 and the first mark coordinate point p, and set the first mark coordinate point p to the minimum value of the distance d in the set A 2 The outermost coordinate point, the outermost coordinate point corresponding to the minimum distance d in the set A 2 is added to the set A'and deleted from the set A 2 , and the step S7.2 is returned;

Step S7.2b: Sort the outermost coordinate points according to the order in which the outermost coordinate points are added to the set A';

Step S7.3: Establish an edge coordinate set, add p 0 to the edge coordinate set and delete it from the set A', and set the second mark coordinate point p'to p 0 ;

Step S7.4: Determine whether the number of elements in the set A'is one, if not, execute step S7.4a, if yes, execute step S7.4b;

Step S7.4a: Determine whether the second mark coordinate point p'and the two outermost coordinate points in the top two positions in the set A'are three-point collinear, if yes, perform step S7.4a1, if not, Step S7.4a2 is executed;

Step S7.4a1: delete the outermost coordinate point that is ranked first in the set A'from the set A', and return to step S7.4;

Step S7.4a2: Set the second marking coordinate point p'as the outermost coordinate point ranked at the top in the set A', and add the outermost coordinate point ranked at the top in the set A' To the edge coordinate set and delete from the set A', return to step S7.4;

Step S7.4b: Add the outermost coordinate points in the set A'to the edge coordinate set, and output the edge coordinate set.
A semi-automatic image segmentation data labeling method according to claim 3, wherein said step S8 further comprises:

Step S8.4: Perform shadow processing on the prediction target area on the image to be labeled.
An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and capable of running on the processor, characterized in that: the processor executes the program as described in claims 1 to 3 Any one of the semi-automatic image segmentation data labeling methods.
A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the semi-automatic image segmentation data labeling method according to any one of claims 1 to 3.