CN117292129A - Image segmentation method, device, equipment and medium - Google Patents

Image segmentation method, device, equipment and medium Download PDF

Info

Publication number
CN117292129A
CN117292129A CN202311224285.2A CN202311224285A CN117292129A CN 117292129 A CN117292129 A CN 117292129A CN 202311224285 A CN202311224285 A CN 202311224285A CN 117292129 A CN117292129 A CN 117292129A
Authority
CN
China
Prior art keywords
image
prompter
target
determining
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311224285.2A
Other languages
Chinese (zh)
Inventor
张松林
严雪飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Fuya Intelligent Technology Development Co ltd
Original Assignee
Shanghai Fuya Intelligent Technology Development Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Fuya Intelligent Technology Development Co ltd filed Critical Shanghai Fuya Intelligent Technology Development Co ltd
Priority to CN202311224285.2A priority Critical patent/CN117292129A/en
Publication of CN117292129A publication Critical patent/CN117292129A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/56Extraction of image or video features relating to colour

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an image segmentation method, device, equipment and medium. The method comprises the following steps: acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented; screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented; determining a target object meeting screening color conditions according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold; and dividing the image to be divided according to the target object. Screening the image to be segmented through a preset characteristic color screening interval to obtain a prompter candidate point, further obtaining a target segmentation mask through a preset prompter segmentation model, and filtering through a preset pixel duty ratio threshold value to obtain a target object. The method and the device realize automatic determination of the target object to be segmented, ensure the accuracy of the determination of the target object and improve the efficiency of image segmentation.

Description

Image segmentation method, device, equipment and medium
Technical Field
The present invention relates to the field of computer vision, and in particular, to a method, apparatus, device, and medium for image segmentation.
Background
In the field of computer vision-related technology, segmentation of objects or backgrounds in images (pixel-level classification) has been an important direction of research in this field, mainly depending on its wide application in multiple scenes; for example, in the field of unmanned aerial vehicle inspection, an alarm region can be accurately located according to image segmentation of an abnormal target; in the augmented reality scene, the more accurate relative pose can be calculated and acquired according to the image segmentation of the actual target, so that better and real visual experience is obtained; etc.
The main data source of the image segmentation task through the deep learning model is to use a labeling tool to manually label the polygons.
However, this labeling method has the following significant drawbacks: firstly, for targets with smaller pixel areas or targets with irregular shapes, the traditional manual labeling needs to take longer time to label one by one, the labeling efficiency is lower, and the labeling cost is higher; the demarcation of the target boundary pixels can only be carried out by adopting a plurality of straight lines, so that the marking errors of part of non-straight line boundary pixels can be caused, and the cutting accuracy of the labels is limited by objective conditions; secondly, under the development trend of the current deep learning model, the model structure is increasingly complex, the number of model parameters is also increasing explosively, so that the data volume required by model training is also becoming very huge (the number of masks in the current part of large model training data is hundreds of millions), and the traditional manual labeling method cannot meet the requirement of rapidly increasing model training data scale.
Disclosure of Invention
The invention provides an image segmentation method, an image segmentation device, image segmentation equipment and a medium, which are used for realizing automatic determination of an image segmentation target object.
According to a first aspect of the present invention, there is provided an image segmentation method comprising:
acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented;
screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented;
determining a target object meeting the color screening condition according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold;
and dividing the image to be divided according to the target object.
According to a second aspect of the present invention, there is provided an image segmentation apparatus comprising:
the information acquisition module is used for acquiring an image to be segmented and screening color conditions and determining digital array information corresponding to each pixel coordinate point in the image to be segmented;
the first determining module is used for screening the digital array information according to a preset characteristic color screening interval and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented;
The second determining module is used for determining a target object meeting the color screening condition according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold;
and the image segmentation module is used for segmenting the image to be segmented according to the target object.
According to a third aspect of the present invention, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the image segmentation method according to any one of the embodiments of the present invention.
According to a fourth aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute the image segmentation method according to any one of the embodiments of the present invention.
According to the technical scheme, the digital array information corresponding to each pixel coordinate point in the image to be segmented is determined by acquiring the image to be segmented and screening color conditions; screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented; determining a target object meeting screening color conditions according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold; and dividing the image to be divided according to the target object. Screening the image to be segmented through a preset characteristic color screening interval to obtain a prompter candidate point, further obtaining a target segmentation mask through a preset prompter segmentation model, and filtering through a preset pixel duty ratio threshold value to obtain a target object. The method and the device realize automatic determination of the target object to be segmented, ensure the accuracy of the determination of the target object and improve the efficiency of image segmentation.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of an image segmentation method according to a first embodiment of the present invention;
fig. 2 is a flowchart of an image segmentation method according to a second embodiment of the present invention;
fig. 3 is a flowchart of an image segmentation method according to a second embodiment of the present invention;
fig. 4 is a schematic diagram of a candidate point set of a prompter in an image segmentation method according to a second embodiment of the present invention;
fig. 5 is a diagram illustrating an example of a target segmentation mask in an image segmentation method according to a second embodiment of the present invention;
Fig. 6 is a diagram illustrating an example of a target segmentation mask set in an image segmentation method according to a second embodiment of the present invention;
fig. 7 is a diagram illustrating an object in an image segmentation method according to a second embodiment of the present invention;
fig. 8 is a schematic structural view of an image segmentation apparatus according to a third embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device implementing an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of an image segmentation method according to an embodiment of the present invention, where the method may be applied to automatic labeling of an image segmentation object, and the method may be performed by an image segmentation apparatus, and the image segmentation apparatus may be implemented in a form of hardware and/or software, and the image segmentation apparatus may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented.
In this embodiment, the image to be segmented may be understood as an image in which the object of interest needs to be segmented. The color screening condition can be understood as a color condition corresponding to the object of interest to be separated, if the object of interest is a green curtain scaffold on the outer layer of the building, the color screening condition is green. The pixel coordinate point can be understood as the pixel coordinate corresponding to each point of the picture in the pixel coordinate system. Digital array information can be understood as representing images in a three-way digital array.
Specifically, the processor may acquire the image to be segmented through uploading or the like, the user may select the object of interest, for example, select through clicking a mouse or the like, the processor may acquire a screening color condition corresponding to the object of interest, the processor may directly determine digital array information corresponding to each pixel coordinate point under the RGB format of the image to be segmented, and may also convert the image to be segmented into a required format, and obtain digital array information under a required test.
For example, the processor may read data from an image to be segmented, where the read data is in an h×w×c digital array (H is the number of pixels in the height direction of the image; W is the number of pixels in the width direction of the image; C is R, G, B,3 channels.); other formats of conversion may be performed on the image data, for example, conversion of HSV (Hue; saturation; value) gamut is performed on the digital array, where the converted data format is an h×w×c digital array (H is the number of pixels in the height direction of the image; W is the number of pixels in the width direction of the image; C is H, S, V,3 channels), and the conversion is performed by the following method:
HSV color gamut conversion formula
(R,G,B)=(R,G,B)/255.0
Max=maX(R,G,B)
Min=min(R,G,B)
S=((Max-Min)*2255
V=Max*255
Wherein R is the value of the red color channel in the image data to be segmented (the value range is [0,255 ]), G is the value of the green color channel in the image data to be segmented (the value range is [0,255 ]), B is the value of the blue color channel in the image data to be segmented (the value range is [0,255 ]), max is the maximum value in R, G, B, min is the minimum value in R, G, B, H is the value range of [0,180], S is the value range of [0,255], and V is the value range of [0,255].
And S120, screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting the color screening condition in the image to be segmented.
In this embodiment, the preset feature color filtering interval may be understood as a color filtering interval corresponding to a color filtering condition, and is used to filter out a color corresponding to the color filtering condition. A set of candidate points of a word-of-a-sentence may be understood to include a set of all the screened points that match the color screening criteria.
Specifically, the processor may screen the numerical value of each channel in the digital array information according to a preset characteristic color screening interval, determine whether the numerical value is in the preset characteristic color screening interval, and use each point in the preset characteristic color screening interval in the image to be segmented as a candidate point set of the prompter meeting the color screening condition.
S130, determining a target object meeting the screening color condition according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold.
In this embodiment, the preset prompter segmentation model may be understood as a model, such as Segment Anything Model, SAM, for extracting an object corresponding to a prompter candidate point. The preset pixel duty threshold may be understood as a duty threshold for screening out whether or not the set duty threshold is for a target object satisfying the screening color condition. The target object may be understood as a target object in the image to be segmented that satisfies the screening color condition.
Specifically, the processor may input the set of candidate points of the prosecution into a preset prosecution segmentation model, determine the overall object to which the candidate points of the prosecution belong, and determine whether each overall object is a target object meeting the screening color condition by comparing with a preset pixel duty threshold value because the overall object determined by the preset prosecution segmentation model may have deviation, but only a small portion of colors of the overall object do not belong to the target object meeting the screening color condition, so as to find out the partial error object by setting a preset pixel duty threshold value, and determine whether each overall object is a target object meeting the screening color condition in the image to be segmented.
S140, dividing the image to be divided according to the target object.
Specifically, the processor may segment the image to be segmented according to the target object in a preset manner.
According to the technical scheme, the digital array information corresponding to each pixel coordinate point in the image to be segmented is determined by acquiring the image to be segmented and screening color conditions; screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented; determining a target object meeting screening color conditions according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold; and dividing the image to be divided according to the target object. Screening the image to be segmented through a preset characteristic color screening interval to obtain a prompter candidate point, further obtaining a target segmentation mask through a preset prompter segmentation model, and filtering through a preset pixel duty ratio threshold value to obtain a target object. The method and the device realize automatic determination of the target object to be segmented, ensure the accuracy of the determination of the target object and improve the efficiency of image segmentation.
Example two
Fig. 2 is a flowchart of an image segmentation method according to a second embodiment of the present invention, where the method is further refined based on the foregoing embodiment, as shown in fig. 2, and includes:
s210, acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented.
S220, determining a corresponding digital array in the digital array information according to each pixel coordinate point in the image to be segmented.
Specifically, the processor may determine, for each pixel coordinate point in the image to be segmented, a digital array corresponding to each pixel coordinate point in the digital array information.
S230, comparing the channel value of each channel in the digital array with a preset characteristic color screening interval.
In this embodiment, each channel is understood to be a channel in a digital array representing different data.
Specifically, the processor may compare the channel values of each channel in the digital array sequentially with the channel values of each channel in the digital array according to the preset feature color filtering interval, so as to determine whether each channel value of the digital array corresponding to each pixel coordinate point in the image to be segmented is in the preset feature color filtering interval.
For example, taking a green curtain as an example, in the RGB data format, each channel value of the green preset feature color filtering interval may be set to R < =0.5×max (R), G > =0.5×max (G), and B < =0.5×max (B). Or under HSV format image data, each channel value of the preset feature color filtering interval of green may be set to 30< =h < =90, 50< =s < =255, and 50< =v < =255.
And S240, when the numerical value of each channel belongs to a preset characteristic color screening interval, taking the pixel coordinate point corresponding to the digital array as a prompter candidate point.
Specifically, when the values of the channels all belong to the preset characteristic color screening interval, the processor can use the pixel coordinate point corresponding to the digital array as a candidate point of the prompter.
S250, determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented according to each candidate point of the prompter.
Specifically, the processor may use all the candidate points of the prompter in the image to be segmented as a set to form a candidate set of the prompter meeting the color screening condition.
And S260, determining a target segmentation mask set of the image to be segmented according to the candidate point set of the prompter and a preset prompter segmentation model.
In this embodiment, the target segmentation mask set may be understood as a set comprising a plurality of target segmentation masks. A target partition mask may be understood as a mask representing all the points belonging to one object.
Specifically, the processor may input a preset extracting word segmentation model according to any several extracting word candidate points in the extracting word candidate point set, determine a target segmentation mask of a complete object to which the extracting word candidate point belongs, delete the selected extracting word candidate points in the extracting word candidate point set, continue to select extracting word candidate points from the remaining extracting word candidate points, determine the target segmentation mask until all extracting word candidate points are selected or reach a set minimum threshold, and use the obtained target segmentation mask as the target segmentation mask set of the image to be segmented.
Further, on the basis of the above embodiment, the step of determining the target segmentation mask set of the image to be segmented according to the candidate point set of the prompter and the preset prompter segmentation model may be optimized as follows:
a1, randomly selecting a set number of candidate points of the prompter from the candidate points of the prompter as target candidate points of the prompter.
In the present embodiment, the target candidate points may be understood as the candidate points for the word to be input into the model.
Specifically, the processor may randomly select a set number of candidate points of the prompter from the candidate point set of the prompter, as the target candidate points of the prompter.
b1, determining a target segmentation mask corresponding to an intermediate target of the image to be segmented according to each target candidate point and a preset prompter segmentation model.
In this embodiment, the intermediate target may be understood as an object segmented by a preset prompter segmentation model.
Specifically, the processor may perform preset word-segmentation model word-segmentation on each target word-segmentation candidate point, for example, using an open-source word-segmentation model: segment Anything Model the SAM performs the prompter segmentation to obtain the corresponding target segmentation mask.
The yellow hanging tower is arranged in front of the green cloth curtain, the green cloth curtain is arranged at the gap due to the fact that the structure of the yellow hanging tower is provided with the gap, the green cloth curtain in the gap of the yellow hanging tower is included in the candidate point A of the word, and after the candidate point A of the word is segmented by the preset word segmentation model, the middle target to which the candidate point A of the word belongs is the yellow hanging tower.
The step of determining the target segmentation mask corresponding to the intermediate target of the image to be segmented according to each target candidate point and the preset prompter segmentation model may be optimized as follows:
and b11, performing position coding on the pixel coordinate points corresponding to the candidate points of each target prompter to obtain position coding information.
In the present embodiment, the position-coding information can be understood as coding information for conversion into a reflection position.
Specifically, the processor may perform position coding on the pixel coordinate points corresponding to each target candidate point of the extracted word through a preset extracted word segmentation model, input the extracted word (token) type of the model as the pixel point coordinates, perform position coding on N coordinate points (w, h), and encode the N coordinate points into an n×256 array (promt token) to obtain position coding information.
And b12, inputting the image to be segmented into a preset visual model to obtain image coding information.
In this embodiment, the preset visual model may be understood as a model for obtaining image coding information, such as ViT model. Image coding information can be understood as coding information for representing an image with a smaller number of bits.
Specifically, the processor may input the image to be segmented into a preset visual model to obtain image coding information (256×64×64).
And b13, determining classification coding information corresponding to each pixel coordinate point.
In this embodiment, classification coding information may be understood as coding information that classifies coordinates of each pixel point into foreground or background.
Specifically, the processor may learn, according to the preset prompter segmentation model, that each pixel coordinate is a code belonging to a foreground or a code of a background (an array of n×256), to obtain the classified code information.
Illustratively, taking the yellow tower crane as an example, the yellow tower crane body is a foreground, and the green cloth curtain of the tower crane gap is a background.
And b14, inputting the position coding information, the image coding information and the classification coding information into a preset prompter segmentation model, and determining a target segmentation mask corresponding to an intermediate target of the image to be segmented.
Specifically, the processor may send the classification coding information, the position coding information and the image coding information to the model decoding module (including the multi-layer self-attention module, the cross-attention module and the fully-connected network module) at the same time, and finally obtain the target segmentation mask with the highest output score of the model.
And c1, filtering out the candidate points of the extracted words to be filtered corresponding to the target segmentation mask in the candidate points of the extracted words to obtain an updated candidate points set of the extracted words.
The object segmentation mask comprises foreground object pixel coordinate points belonging to foreground objects and background object pixel coordinate points belonging to background objects.
In this embodiment, the foreground object may be understood as an object as a subject, and the foreground object pixel coordinate points may be understood as all pixel coordinate points belonging to the foreground object in the object segmentation mask. The background object may be understood as an object that serves as a background in the object segmentation mask, and the background object pixel coordinate points may be understood as all pixel coordinate points belonging to the background object.
In this embodiment, the candidate points of the extracted words to be filtered may be understood as the candidate points of the extracted words corresponding to the foreground object in the object segmentation mask.
Specifically, the processor may filter the candidate set of the extracted words, remove the candidate points (coordinates of the pixel points) of the extracted words corresponding to the foreground target pixel coordinate points in the target segmentation mask (the set of coordinates of the pixel points) from the candidate set of the extracted words according to the target segmentation mask obtained in the above step, and only retain the candidate set of the extracted words except the foreground target mask to obtain the updated candidate set of the extracted words.
d1, returning to the step of determining the target segmentation mask based on the updated candidate points of the prompter, and determining the target segmentation mask set according to each target segmentation mask until the number of the candidate points of the prompter in the updated candidate points of the prompter is smaller than the set ending threshold.
In the present embodiment, setting the end threshold may be understood as a threshold for ending the target segmentation mask determination step.
Specifically, the processor may return to the step of determining the target segmentation mask based on the updated set of candidate points for the prompter, that is, the steps a1 to c1 described above, until the number of candidate points for the prompter in the updated set of candidate points for the prompter is less than the set end threshold, and use each target segmentation mask as the target segmentation mask set for the image to be segmented.
S270, determining the duty ratio of the characteristic color pixel points corresponding to each intermediate target according to the target segmentation mask set and the candidate point set of the prompter.
In this embodiment, the characteristic color pixel point duty ratio may be understood as the duty ratio of the characteristic color in the intermediate target to all colors in the intermediate target.
Further, on the basis of the above-mentioned strength, the step of determining the characteristic color pixel point duty ratio corresponding to each intermediate target according to the target segmentation mask set and the candidate point set of the prompter may be further optimized as follows:
a2, determining the number of foreground coordinate points of the foreground target pixel coordinate points in the target segmentation mask.
In this embodiment, the number of foreground coordinate points may be understood as the number of all coordinate points belonging to the foreground object.
Specifically, the processor may count the number of all foreground target pixel coordinate points belonging to the foreground target in the target segmentation mask, to obtain the number of foreground coordinate points.
b2, determining a color coordinate point set belonging to the foreground object in the candidate point set of the prompter, and determining the number of color coordinate points in the color coordinate point set.
It is to be noted that the set of candidate points for a prompter includes only candidate points for a prompter that satisfy the screening color condition, and the target segmentation mask includes all points belonging to the intermediate target, possibly including some points that do not satisfy the screening color condition.
In the present embodiment, the number of color coordinate points can be understood as the total number of pixel coordinate points satisfying the screening color condition.
Specifically, the processor may search the candidate set of candidate points for a set of color coordinates belonging to the foreground object, and count the number of color coordinates.
And c2, determining the characteristic color pixel point duty ratio of the intermediate target corresponding to the target segmentation mask according to the number of the color coordinate points and the number of the foreground coordinate points.
Specifically, the processor may divide the number of color coordinate points by the number of foreground coordinate points to obtain a characteristic color pixel point duty ratio of the intermediate target corresponding to the target segmentation mask.
And S280, when the duty ratio of the pixel points of the characteristic colors is larger than a preset pixel duty ratio threshold, taking the corresponding intermediate target as a target object meeting the color screening condition.
Specifically, the processor may compare the characteristic color pixel point duty ratio with a preset pixel duty ratio threshold, and when the characteristic color pixel point duty ratio is greater than the preset pixel duty ratio threshold, use the corresponding intermediate target as a target object that satisfies the color screening condition. Discarding intermediate targets less than or equal to.
For example, taking the yellow tower crane in the above embodiment as an example, since the yellow tower crane body is yellow, that is, the foreground object in the object segmentation mask is the yellow tower crane, the background object is the green cloth curtain, all the candidate points of the prompter corresponding to the green cloth curtain are counted as the number of color coordinate points, the yellow tower crane corresponds to the number of foreground coordinate points, and since the main body is the yellow tower crane, the ratio of the number of the color coordinate points to the number of the foreground coordinate points is very low and is smaller than the preset pixel ratio threshold, the yellow tower crane is abandoned.
S290, dividing the image to be divided according to the target object.
According to the technical scheme, the images to be segmented are screened through the preset characteristic color screening interval to obtain the candidate points of the words, then the preset word segmentation model is adopted to carry out word segmentation on the single candidate points of the words to obtain the high-quality target segmentation mask, further the candidate point sets of the words in the range of the target segmentation mask are filtered, the word segmentation and the candidate point filtering are continuously subjected to iterative processing, repeated determination of one candidate point of the words is avoided, the word segmentation speed is improved, the image segmentation speed is further improved, the target segmentation mask set is obtained until the number of the remaining candidate points of the words is smaller than the set end threshold, and then the target objects in the images to be segmented are obtained through filtering through the preset pixel occupation ratio threshold. The method and the device realize automatic determination of the target object to be segmented, ensure the accuracy of the determination of the target object, improve the image segmentation efficiency and avoid the problems of low efficiency of manual annotation and annotation errors.
In order to facilitate understanding of the present solution, an example is described, and fig. 3 is a flowchart illustrating an image segmentation method according to a second embodiment of the present invention. As shown in fig. 3, the method comprises the following steps:
S310, acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented;
s320, screening the digital array information through a preset characteristic color screening interval to obtain a candidate point set of the prompter meeting the screening color condition;
s330, selecting one candidate point of the extracting words from the candidate point set of extracting words, inputting the candidate point of the extracting words into a preset extracting word segmentation model, and obtaining a target segmentation mask;
s340, filtering the candidate point set of the prompter according to the target segmentation mask to obtain the candidate point set of the prompter after filtering;
s350, judging whether the number of the candidate points of the concentrated extracting words of the candidate points of the extracting words after filtering is smaller than a set ending threshold value; if yes; then jump to step S360; if not, jumping to step S330;
s360, obtaining a target segmentation mask set corresponding to the image to be segmented;
s370, traversing a target segmentation mask in a target segmentation mask set, and determining the duty ratio of the number of color coordinate points of the target segmentation mask in a candidate point set of the prompter to the number of foreground coordinate points of a foreground target pixel coordinate point in the target segmentation mask to obtain the duty ratio of a characteristic color pixel point corresponding to the target segmentation mask;
s380, taking a target segmentation mask with the characteristic color pixel point duty ratio larger than a preset pixel duty ratio threshold value in the image to be segmented as a target object, and discarding the target segmentation mask with the characteristic color pixel point duty ratio smaller than or equal to the preset pixel duty ratio threshold value;
S390, obtaining all target objects meeting the screening color condition in the image to be segmented, and segmenting the image to be segmented according to the target objects.
For the sake of understanding the present solution, a specific example is shown, where the condition of screening color is green, and fig. 4 is a schematic diagram of a candidate set of points for a prompter in an image segmentation method provided by the second embodiment of the present invention, where a preset feature color screening interval is green, and digital array information is screened by the green screening interval, so that the candidate set of points for a prompter meeting the condition of screening color is shown as a white portion in fig. 4, and due to the conditions of changing the lighting condition and the local area of the curtain being non-green, the identified white portion in the figure may appear in mottled condition. Fig. 5 is a diagram illustrating an example of a target segmentation mask in an image segmentation method according to a second embodiment of the present invention, where, as shown in fig. 5, a star-shaped sign is a candidate point of a preset prompter segmentation model, and a white part is a target segmentation mask generated by the preset prompter segmentation model according to the candidate point of the prompter, so that it can be seen that the target segmentation mask processed by the preset prompter segmentation model includes an overall intermediate object to which a green curtain belongs. Fig. 6 is an exemplary diagram of a target segmentation mask set in an image segmentation method according to a second embodiment of the present invention, as shown in fig. 6, it can be seen that all intermediate objects including a green curtain further include a non-green tower crane, and since a green curtain is disposed in a gap of the non-green tower crane, the non-green tower crane is also determined as the intermediate object when determining the target segmentation mask. Fig. 7 is a diagram of an example of a target object in an image segmentation method according to a second embodiment of the present invention, as shown in fig. 7, since the whole non-green hanging tower is non-green, and only a part of the non-green hanging tower is provided with a green background curtain, the determined characteristic color pixel point duty ratio of the non-green hanging tower is smaller than a preset pixel duty ratio threshold, and the intermediate object not belonging to the green curtain is filtered through the characteristic color pixel point duty ratio and the preset pixel duty ratio threshold, so as to obtain the final target object shown in fig. 7.
Example III
Fig. 8 is a schematic structural diagram of an image segmentation apparatus according to a third embodiment of the present invention. As shown in fig. 8, the apparatus includes: an information acquisition module 41, a first determination module 42, a second determination module 43, and an image segmentation module 44. Wherein,
the information acquisition module 41 is configured to acquire an image to be segmented and filter color conditions, and determine digital array information corresponding to each pixel coordinate point in the image to be segmented;
the first determining module 42 is configured to screen the digital array information according to a preset feature color screening interval, and determine a candidate point set of a prompter meeting a color screening condition in the image to be segmented;
a second determining module 43, configured to determine a target object that meets the color screening condition according to the candidate set of points for the word-extracting, a preset word-extracting segmentation model, and a preset pixel duty threshold;
the image segmentation module 44 is configured to segment the image to be segmented according to the target object.
According to the technical scheme, the digital array information corresponding to each pixel coordinate point in the image to be segmented is determined by acquiring the image to be segmented and screening color conditions; screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented; determining a target object meeting screening color conditions according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold; and dividing the image to be divided according to the target object. Screening the image to be segmented through a preset characteristic color screening interval to obtain a prompter candidate point, further obtaining a target segmentation mask through a preset prompter segmentation model, and filtering through a preset pixel duty ratio threshold value to obtain a target object. The method and the device realize automatic determination of the target object to be segmented, ensure the accuracy of the determination of the target object and improve the efficiency of image segmentation.
Further, the first determining module 42 is specifically configured to:
determining a corresponding digital array in the digital array information aiming at each pixel coordinate point in the image to be segmented;
comparing the channel value of each channel in the digital array with the preset characteristic color screening interval;
when each channel value belongs to the preset characteristic color screening interval, taking a pixel coordinate point corresponding to the digital array as a prompter candidate point;
and determining a candidate point set of the prompter meeting the color screening condition in the image to be segmented according to each candidate point of the prompter.
Further, the second determining module 43 includes:
the first determining unit is used for determining a target segmentation mask set of the image to be segmented according to the candidate point set of the prompter and the preset prompter segmentation model;
the second determining unit is used for determining the characteristic color pixel point duty ratio corresponding to each intermediate target according to the target segmentation mask set and the candidate point set;
and the third determining unit is used for taking the corresponding intermediate target as a target object meeting the color screening condition when the characteristic color pixel point duty ratio is larger than the preset pixel duty ratio threshold value.
Further, the first determination unit includes:
a first determining subunit, configured to randomly select a set number of candidate points of the prompter from the candidate point set of the prompter, as target candidate points of the prompter;
the second determining subunit is used for determining a target segmentation mask corresponding to the intermediate target of the image to be segmented according to each target prompter candidate point and the preset prompter segmentation model;
a third determining subunit, configured to filter, in the candidate point set for the extracted words, candidate points to be filtered corresponding to the foreground target pixel coordinate points, obtaining the updated candidate point set of the prompter;
and a fourth determining subunit, configured to return to the step of determining a target segmentation mask based on the updated candidate set of extracted terms until the number of candidate extracted terms in the updated candidate set of extracted terms is less than a set ending threshold, and determine the target segmentation mask set according to each of the target segmentation masks.
Wherein the second determining subunit is specifically configured to:
position coding is carried out on the pixel coordinate points corresponding to the target prompter candidate points, so that position coding information is obtained;
inputting the image to be segmented into a preset visual model to obtain image coding information;
Determining classification coding information corresponding to each pixel coordinate point;
and inputting the position coding information, the image coding information and the classification coding information into the preset prompter segmentation model, and determining a target segmentation mask corresponding to an intermediate target of the image to be segmented.
The target segmentation mask comprises foreground target pixel coordinate points belonging to foreground targets and background target pixel coordinate points belonging to background targets.
Further, the second unit is specifically configured to:
determining the number of foreground coordinate points of foreground target pixel coordinate points in the target segmentation mask;
determining a set of color coordinate points belonging to the foreground object in the set of candidate points of the prompter, and determining the number of color coordinate points in the set of color coordinate points;
and determining the characteristic color pixel point duty ratio of the intermediate target corresponding to the target segmentation mask according to the number of the color coordinate points and the number of the foreground coordinate points.
The image segmentation device provided by the embodiment of the invention can execute the image segmentation method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 9 shows a schematic diagram of an electronic device 50 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 9, the electronic device 50 includes at least one processor 51, and a memory, such as a Read Only Memory (ROM) 52, a Random Access Memory (RAM) 53, etc., communicatively connected to the at least one processor 51, in which the memory stores a computer program executable by the at least one processor, and the processor 51 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 52 or the computer program loaded from the storage unit 58 into the Random Access Memory (RAM) 53. In the RAM 53, various programs and data required for the operation of the electronic device 50 can also be stored. The processor 51, the ROM 52 and the RAM 53 are connected to each other via a bus 54. An input/output (I/O) interface 55 is also connected to bus 54.
Various components in the electronic device 50 are connected to the I/O interface 55, including: an input unit 56 such as a keyboard, a mouse, etc.; an output unit 57 such as various types of displays, speakers, and the like; a storage unit 58 such as a magnetic disk, an optical disk, or the like; and a communication unit 59 such as a network card, modem, wireless communication transceiver, etc. The communication unit 59 allows the electronic device 50 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks.
The processor 51 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 51 performs the respective methods and processes described above, such as an image segmentation method.
In some embodiments, the image segmentation method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 58. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 50 via the ROM 52 and/or the communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the image segmentation method described above may be performed. Alternatively, in other embodiments, the processor 51 may be configured to perform the image segmentation method in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. An image segmentation method, comprising:
acquiring an image to be segmented and screening color conditions, and determining digital array information corresponding to each pixel coordinate point in the image to be segmented;
screening the digital array information according to a preset characteristic color screening interval, and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented;
determining a target object meeting the color screening condition according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold;
And dividing the image to be divided according to the target object.
2. The method according to claim 1, wherein the filtering the digital array information according to a preset feature color filtering interval, and determining a candidate set of prosecution points in the image to be segmented that satisfies a color filtering condition, includes:
determining a corresponding digital array in the digital array information aiming at each pixel coordinate point in the image to be segmented;
comparing the channel value of each channel in the digital array with the preset characteristic color screening interval;
when each channel value belongs to the preset characteristic color screening interval, taking a pixel coordinate point corresponding to the digital array as a prompter candidate point;
and determining a candidate point set of the prompter meeting the color screening condition in the image to be segmented according to each candidate point of the prompter.
3. The method according to claim 1, wherein the determining the target object in the image to be segmented that satisfies the color screening condition according to the candidate set of extracted words, a preset extracted word segmentation model, and a preset pixel duty ratio threshold value includes:
determining a target segmentation mask set of the image to be segmented according to the candidate point set of the prompter and the preset prompter segmentation model;
Determining the duty ratio of the characteristic color pixel points corresponding to each intermediate target according to the target segmentation mask set and the candidate point set;
and when the characteristic color pixel point duty ratio is larger than the preset pixel duty ratio threshold, taking the corresponding intermediate target as a target object meeting the color screening condition.
4. A method according to claim 3, wherein the object segmentation mask includes foreground object pixel coordinate points belonging to foreground objects and background object pixel coordinate points belonging to background objects.
5. The method of claim 4, wherein the determining the target segmentation mask set for the image to be segmented according to the set of candidate points for the prompter and the preset prompter segmentation model comprises:
randomly selecting a set number of candidate points of the prompter from the candidate point set of the prompter as target candidate points of the prompter;
determining a target segmentation mask corresponding to an intermediate target of the image to be segmented according to each target prompter candidate point and the preset prompter segmentation model;
filtering out candidate points of the extracted words to be filtered corresponding to the foreground target pixel coordinate points in the candidate points of the extracted words to obtain an updated candidate points of the extracted words;
And returning to the step of determining the target segmentation mask based on the updated candidate points of the prompter until the number of candidate points of the prompter in the updated candidate points of the prompter is smaller than a set ending threshold, and determining the target segmentation mask set according to each target segmentation mask.
6. The method according to claim 5, wherein the determining a target segmentation mask corresponding to an intermediate target of the image to be segmented according to each of the target candidate points and the preset prompter segmentation model includes:
position coding is carried out on the pixel coordinate points corresponding to the target prompter candidate points, so that position coding information is obtained;
inputting the image to be segmented into a preset visual model to obtain image coding information;
determining classification coding information corresponding to each pixel coordinate point;
and inputting the position coding information, the image coding information and the classification coding information into the preset prompter segmentation model, and determining a target segmentation mask corresponding to an intermediate target of the image to be segmented.
7. The method of claim 4, wherein determining the feature color pixel point duty ratio corresponding to the intermediate target according to the target segmentation mask set and the candidate set of prompter points comprises:
Determining the number of foreground coordinate points of foreground target pixel coordinate points in the target segmentation mask;
determining a set of color coordinate points belonging to the foreground object in the set of candidate points of the prompter, and determining the number of color coordinate points in the set of color coordinate points;
and determining the characteristic color pixel point duty ratio of the intermediate target corresponding to the target segmentation mask according to the number of the color coordinate points and the number of the foreground coordinate points.
8. An image dividing apparatus, comprising:
the information acquisition module is used for acquiring an image to be segmented and screening color conditions and determining digital array information corresponding to each pixel coordinate point in the image to be segmented;
the first determining module is used for screening the digital array information according to a preset characteristic color screening interval and determining a candidate point set of the prompter meeting color screening conditions in the image to be segmented;
the second determining module is used for determining a target object meeting the color screening condition according to the candidate point set of the prompter, a preset prompter segmentation model and a preset pixel duty ratio threshold;
and the image segmentation module is used for segmenting the image to be segmented according to the target object.
9. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the image segmentation method of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the image segmentation method according to any one of claims 1-7.
CN202311224285.2A 2023-09-21 2023-09-21 Image segmentation method, device, equipment and medium Pending CN117292129A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311224285.2A CN117292129A (en) 2023-09-21 2023-09-21 Image segmentation method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311224285.2A CN117292129A (en) 2023-09-21 2023-09-21 Image segmentation method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN117292129A true CN117292129A (en) 2023-12-26

Family

ID=89238369

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311224285.2A Pending CN117292129A (en) 2023-09-21 2023-09-21 Image segmentation method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN117292129A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117995317A (en) * 2024-04-03 2024-05-07 北京云庐科技有限公司 Method, device and medium for estimating heavy atom position based on electron density map

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117995317A (en) * 2024-04-03 2024-05-07 北京云庐科技有限公司 Method, device and medium for estimating heavy atom position based on electron density map

Similar Documents

Publication Publication Date Title
CN111784685A (en) Power transmission line defect image identification method based on cloud edge cooperative detection
CN106803257B (en) Method for segmenting disease spots in crop disease leaf image
CN108318773B (en) Transmission conductor strand breakage detection method and system
CN110399842B (en) Video processing method and device, electronic equipment and computer readable storage medium
CN113887418A (en) Method and device for detecting illegal driving of vehicle, electronic equipment and storage medium
JP7429756B2 (en) Image processing method, device, electronic device, storage medium and computer program
CN110599453A (en) Panel defect detection method and device based on image fusion and equipment terminal
CN112001317A (en) Lead defect identification method and system based on semantic information and terminal equipment
CN113221878A (en) Detection frame adjusting method and device applied to signal lamp detection and road side equipment
CN112184837A (en) Image detection method and device, electronic equipment and storage medium
CN117557784B (en) Target detection method, target detection device, electronic equipment and storage medium
CN108900895B (en) Method and device for shielding target area of video stream
CN109118494B (en) Overlapping region segmentation method and device based on concave point matching
CN117333443A (en) Defect detection method and device, electronic equipment and storage medium
CN117292129A (en) Image segmentation method, device, equipment and medium
CN112991308B (en) Image quality determining method and device, electronic equipment and medium
CN114399617B (en) Method, device, equipment and medium for identifying shielding pattern
CN108961357B (en) Method and device for strengthening over-explosion image of traffic signal lamp
CN115311321A (en) Background replacing method, device, electronic equipment and storage medium
CN111932470A (en) Image restoration method, device, equipment and medium based on visual selection fusion
CN109146893B (en) Oil light area segmentation method and device and mobile terminal
CN113099202B (en) Automatic white balance optimization method, equipment and computer readable storage medium
CN113362297B (en) Image processing method, apparatus and storage medium for printed circuit board
CN112507956B (en) Signal lamp identification method and device, electronic equipment, road side equipment and cloud control platform
CN117557777A (en) Sample image determining method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination