CN115830046A - Interactive image segmentation method, device, equipment and storage medium - Google Patents
Interactive image segmentation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115830046A CN115830046A CN202211203622.5A CN202211203622A CN115830046A CN 115830046 A CN115830046 A CN 115830046A CN 202211203622 A CN202211203622 A CN 202211203622A CN 115830046 A CN115830046 A CN 115830046A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- interactive
- information
- result
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 204
- 238000003709 image segmentation Methods 0.000 title claims abstract description 50
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000011218 segmentation Effects 0.000 claims abstract description 324
- 230000003993 interaction Effects 0.000 claims abstract description 175
- 238000012937 correction Methods 0.000 claims abstract description 89
- 238000013507 mapping Methods 0.000 claims abstract description 50
- 238000001514 detection method Methods 0.000 claims abstract description 36
- 238000004088 simulation Methods 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000000694 effects Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 4
- 230000004048 modification Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 3
- 230000002829 reductive effect Effects 0.000 description 3
- 230000036961 partial effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000001965 increasing effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, and discloses an interactive image segmentation method, device, equipment and storage medium. The method comprises the following steps: acquiring an interactive mode and segmentation information adopted when a target image is interactively segmented; according to the interaction mode, selecting a preset region mapping strategy, and coding the segmentation information to obtain coding information; according to the coding information, a preset interactive segmentation model is utilized to segment a target object of the target image to obtain a local segmentation result; and carrying out satisfaction detection on the local segmentation result, and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image. The invention improves the flexibility of image target interactive segmentation.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an interactive image segmentation method, apparatus, device, and storage medium.
Background
When a conventional labeling tool (e.g., lableMe, lableImage, etc.) is used to label a segmented data set, it is usually necessary to label the segmented data set along the edge of a target object in a dotting manner, and in general, tens of clicks or even hundreds of clicks along an outline are needed to obtain a finer result, which is costly. In contrast, with the interactive segmentation method for labeling, the segmentation of the target may be completed with only a few interactions. Interactive segmentation has many forms, mainly including: 1) Clicking: the method based on clicking mainly comprises two types of clicking, wherein positive clicking is used for generating a target area, and negative clicking is used for eliminating a wrong non-target area; 2) Scribing: the scribing-based method mainly obtains a segmentation result by drawing any straight line, curve or broken line on a target or background area; 3) Picture frame: the frame-based method determines a bounding box containing the object through a set of diagonal vertices, and usually needs to provide additional mutual information to obtain the segmentation result of the object inside the bounding box.
The existing interactive segmentation method generally only supports one interactive mode, and most of the interactive mode is mainly in the form of clicking, because the interaction of clicking is simplest. However, the single interaction approach lacks flexibility, such as the segmentation result in scenes with longer target objects is not fine enough. The existing interactive segmentation method has the problem that the edge segmentation effect of a modified target object is difficult to segment accurately, especially when the edge contour of the target is not clear enough and the difference between an edge area and a background is small. Namely, the existing interactive segmentation method is not accurate enough for edge segmentation of the modified target object.
Disclosure of Invention
The invention mainly aims to solve the technical problem that the edge segmentation of a corrected target object is not accurate enough in the conventional interactive segmentation method.
The invention provides an interactive image segmentation method in a first aspect, which comprises the following steps: acquiring an interactive mode and segmentation information adopted when a target image is interactively segmented; selecting a preset region mapping strategy according to the interaction mode, and coding the segmentation information to obtain coding information; according to the coding information, a preset interactive segmentation model is utilized to segment the target object of the target image to obtain a local segmentation result; and carrying out satisfaction detection on the local segmentation result, and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
Optionally, in a first implementation manner of the first aspect of the present invention, the preset area mapping policy includes a first area mapping policy, a second area mapping policy, and a third area mapping policy, and the selecting the preset area mapping policy according to the interaction manner, and performing coding processing on the segmentation information to obtain coding information includes: if the interactive mode is a first interactive mode, extracting click information in the segmentation information, and performing distance transformation coding in the target image by using the first region mapping strategy according to the click information to obtain first coding information; if the interactive mode is a second interactive mode, extracting scribing information in the segmentation information, and performing adaptive scaling coding in the target image by using the second region mapping strategy according to the scribing information to obtain second coding information; if the interactive mode is a third interactive mode, extracting picture frame information in the segmentation information, and performing frame coding on the target image by using the third region mapping strategy according to the picture frame information to obtain third coding information, wherein the coding information comprises first coding information, second coding information and third coding information.
Optionally, in a second implementation manner of the first aspect of the present invention, the obtaining the interactive segmentation result of the target image by performing at least one correction on the local segmentation result according to the detected result includes: selecting a new interaction mode adopted for each local segmentation result to be corrected according to the detection result and the preset interaction probability; based on each local segmentation result to be corrected, adopting a selected new interaction mode to execute interactive segmentation simulation operation in the target image to obtain new segmentation information; and according to the new segmentation information, performing edge correction on a local segmentation result to be corrected to obtain an interactive segmentation result of the target image.
Optionally, in a third implementation manner of the first aspect of the present invention, the selecting, according to the detection result and according to the preset interaction probability, a new interaction manner that is adopted for the local segmentation result to be corrected each time includes: if the detected result is passed, selecting the second interaction mode as a new interaction mode adopted for the local segmentation result to be corrected; if the detected result is failed, counting the number of times of correction of the partial segmentation result and the correction random number of the target image each time; and comparing the correction random number with the interaction probability according to the correction times, and selecting a new interaction mode adopted by each local segmentation result to be corrected according to the comparison result.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the interaction probability includes a frame interaction probability and a scribe line interaction probability, and the comparing, according to the correction times, the correction random number and the interaction probability, and according to a comparison result, selecting a new interaction manner used for each local segmentation result to be corrected includes: if the correction times are preset first times, comparing the corresponding correction random number with the scribing interaction probability, and if the corresponding correction random number is smaller than the scribing interaction probability, selecting the second interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; if the correction times are preset second times, comparing the magnitude between the corresponding correction random number and the picture frame interaction probability, and if the corresponding correction random number is smaller than the picture frame interaction probability, selecting the third interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; and if the correction times are not the first times or the second times, selecting the first interaction mode as a new interaction mode adopted by the local segmentation result to be corrected at this time, wherein the first times are greater than the second times.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the performing an interactive segmentation simulation operation in the target image by using a selected new interactive manner based on each to-be-corrected local segmentation result to obtain new segmentation information includes: if the selected new interactive mode is the first interactive mode or the second interactive mode, selecting a maximum edge area with segmentation errors in a local segmentation result to be corrected of the target image each time, and simulating scribing operation on a local contour with a preset size proportion in the segmentation error area to be used as new segmentation information; and if the selected new interaction mode is the third interaction mode, generating an external matrix frame in the local segmentation result to be corrected of the target image each time, and adding a disturbance effect in the external matrix frame as new segmentation information.
A second aspect of the present invention provides an interactive image segmentation apparatus, comprising: the acquisition module is used for acquiring an interactive mode and segmentation information adopted when the target image is interactively segmented; the coding module is used for selecting a preset region mapping strategy according to the interaction mode and coding the segmentation information to obtain coding information; the segmentation module is used for segmenting a target object of the target image by utilizing a preset interactive segmentation model according to the coding information to obtain a local segmentation result; and the correction module is used for carrying out satisfaction detection on the local segmentation result and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
Optionally, in a first implementation manner of the second aspect of the present invention, the preset region mapping policy includes a first region mapping policy, a second region mapping policy, and a third region mapping policy, and the encoding module includes: the click coding unit is used for extracting click information in the segmentation information if the interaction mode is a first interaction mode, and performing distance transformation coding in the target image by using the first region mapping strategy according to the click information to obtain first coding information; the line marking coding unit is used for extracting line marking information in the segmentation information if the interactive mode is a second interactive mode, and carrying out self-adaptive scaling coding on the target image by utilizing the second region mapping strategy according to the line marking information to obtain second coding information; and the frame coding unit is used for extracting the frame information in the segmentation information if the interactive mode is a third interactive mode, and carrying out frame coding in the target image by utilizing the third region mapping strategy according to the frame information to obtain third coding information, wherein the coding information comprises first coding information, second coding information and third coding information.
Optionally, in a second implementation manner of the second aspect of the present invention, the modifying module includes: the interaction selection unit is used for selecting a new interaction mode adopted for each local segmentation result to be corrected according to the detection result and the preset interaction probability; the simulation segmentation unit is used for executing interactive segmentation simulation operation in the target image by adopting a selected new interactive mode based on each local segmentation result to be corrected to obtain new segmentation information; and the edge correction unit is used for performing edge correction on the local segmentation result to be corrected according to the new segmentation information to obtain an interactive segmentation result of the target image.
Optionally, in a third implementation manner of the second aspect of the present invention, the interaction selecting unit is further configured to: if the detected result is passed, selecting the second interaction mode as a new interaction mode adopted for the local segmentation result to be corrected; if the detected result is failed, counting the correction times of the local segmentation result and the correction random number of the target image each time; and comparing the correction random number with the interaction probability according to the correction times, and selecting a new interaction mode adopted by each local segmentation result to be corrected according to the comparison result.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the interaction probability includes a picture frame interaction probability and a line drawing interaction probability, and the interaction selecting unit is further configured to: if the correction times are preset first times, comparing the corresponding correction random number with the scribing interaction probability, and if the corresponding correction random number is smaller than the scribing interaction probability, selecting the second interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; if the correction times are preset second times, comparing the magnitude between the corresponding correction random number and the picture frame interaction probability, and if the corresponding correction random number is smaller than the picture frame interaction probability, selecting the third interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; and if the correction times are not the first times or the second times, selecting the first interaction mode as a new interaction mode adopted by the local segmentation result to be corrected at this time, wherein the first times are greater than the second times.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the analog dividing unit is further configured to: if the selected new interactive mode is the first interactive mode or the second interactive mode, selecting a maximum edge area with segmentation errors in a local segmentation result to be corrected of the target image each time, and simulating scribing operation on a local contour with a preset size proportion in the segmentation error area to be used as new segmentation information; and if the selected new interaction mode is the third interaction mode, generating an external matrix frame in the local segmentation result to be corrected of the target image each time, and adding a disturbance effect in the external matrix frame as new segmentation information.
A third aspect of the present invention provides an interactive image segmentation apparatus comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the interactive image segmentation apparatus to perform the interactive image segmentation method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the above-mentioned interactive image segmentation method.
According to the technical scheme provided by the invention, a preset region mapping strategy is selected to encode the segmentation information according to an interactive mode adopted when a user interactively segments a target image to obtain encoded information for inputting an interactive segmentation model, and the target image is segmented according to the input encoded information in the interactive segmentation model to obtain a local segmentation result; at this time, the local segmentation result obtained by the first segmentation may have an edge contour which is not accurate enough, so that other suitable interactive modes can be selected to correct the local segmentation result, the segmentation result with clear edge contour is obtained step by step and is used as the interactive segmentation result of the target image, the edge contour of the target segmentation result of the image is more accurate by using and modifying the edge contour in a superposition mode of different interactive modes, the interactive segmentation mode of the image can be used in a cross mode, and the target segmentation of the image is more flexible.
Drawings
FIG. 1 is a diagram of an embodiment of an interactive image segmentation method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another embodiment of an interactive image segmentation method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an embodiment of an interactive image segmentation apparatus according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another embodiment of an interactive image segmentation apparatus according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an embodiment of an interactive image segmentation apparatus according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an interactive image segmentation method, an interactive image segmentation device, interactive image segmentation equipment and a storage medium, wherein an interactive mode and segmentation information adopted when a target image is interactively segmented are obtained; according to the interaction mode, selecting a preset region mapping strategy, and coding the segmentation information to obtain coding information; according to the coding information, a preset interactive segmentation model is utilized to segment a target object of the target image to obtain a local segmentation result; and carrying out satisfaction degree detection on the local segmentation result, and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image. The invention improves the flexibility of image target interactive segmentation.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For understanding, a detailed flow of an embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of an interactive image segmentation method according to an embodiment of the present invention includes:
101. acquiring an interactive mode and segmentation information adopted when a target image is interactively segmented;
it is to be understood that the execution subject of the present invention may be an interactive image segmentation apparatus, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
In this embodiment, a corresponding interactive manner is selected in the front-end client to perform interactive segmentation on the target image, one or more target objects are segmented from the target image, and for each segmented target object, at least point-by-point position coordinates of a segmentation boundary of the segmented target object are identified, so as to generate segmentation information corresponding to each target. The used interaction modes and segmentation information are directly obtained from the front-end client, wherein the interaction modes can include click, line drawing, picture frame drawing and the like, and the number of the interaction modes can include one or more of the click, line drawing and the like, for example, the target image is jointly segmented by using the click and line drawing interaction modes.
In the embodiment, the provided interactive segmentation method supports three interactive modes of clicking, marking and drawing, and can realize the segmentation of the target by adopting different interactive strategies in different scenes; in addition, the three interaction modes are combined to obtain a more accurate result than single interaction, and the segmentation effect on the edge of the target object is mainly improved; meanwhile, the interaction times required by the user can be reduced, and the labeling efficiency is improved.
102. Selecting a preset region mapping strategy according to the interaction mode, and coding the segmentation information to obtain coding information;
in this embodiment, after the selected interaction mode for the target image is obtained, the selected interaction mode is encoded, for example, the clicked interaction mode is fixedly encoded into an a field, the scribed interaction mode is fixedly encoded into a B field, the frame interaction mode is fixedly encoded into a C field, then a corresponding region mapping policy is selected according to the input encoded field, then the segmentation information is encoded, the point-by-point position coordinates of the segmentation boundary at least included in the segmentation information are converted from the screen into the target image, and the segmentation range of each target object in the target image is determined.
Specifically, the interactive encoding may use any method, for example, for the click interaction mode, the segmentation information may be encoded by using a distance transformation, a disc encoding, or the like; for the line drawing interactive mode, the line length and the line width shown in the segmentation information can be subjected to self-adaptive scaling; for the interactive mode of the picture frame, the whole boundary frame represented in the segmentation information can be coded, or only the boundary of the frame or a group of diagonal vertexes can be coded. The method is not limited specifically, and the area mapping policy of each interaction mode may also be adjusted according to the service requirement.
103. According to the coding information, a preset interactive segmentation model is utilized to segment the target object of the target image to obtain a local segmentation result;
in this embodiment, after the segmentation information is encoded according to the selected interactive mode, the obtained encoded information may determine the segmentation range of each target object in the target image. In the first segmentation, the interactive segmentation model is directly segmented according to the segmentation range of each target object represented in the coding information, and the segmentation result under the first interactive segmentation is obtained. In the first segmentation, in order to reduce the manual time and improve the automatic segmentation efficiency, only the approximate segmentation range of each target object in the target image is segmented in an interactive mode of clicking, scribing or drawing frames once or more.
104. And carrying out satisfaction degree detection on the local segmentation result, and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
In this embodiment, only the approximate range of each target object in the target image is segmented for the local segmentation result, and here, the satisfaction detection of the local segmentation result is also performed, for example, whether the front-end selection is satisfied or not is performed, or the interactive segmentation model automatically identifies the local segmentation result according to the real sample, so as to obtain the satisfaction detection result. The accuracy of the first segmentation of the target image is represented by the result of the satisfaction detection. The higher the satisfaction, the higher the determination accuracy, and the lower the satisfaction, the lower the determination accuracy. And then, according to the result of the satisfaction detection, the local segmentation result is iterated again by using the interactive segmentation model again to correct the local segmentation result on the target image until the preset number of iterations or the result of the satisfaction detection reaches a set threshold value, and the local segmentation result corrected for the last time is taken as the interactive segmentation result of the target image.
Specifically, in the process of correcting the local segmentation result, the aforementioned interaction mode is still selected, one or more interaction modes are selected according to the set conditions for each correction to superimpose interactive segmentation on the original local segmentation result, and the edge of the target object on the local segmentation result is corrected according to the local segmentation result obtained by the last interactive segmentation, so that the target object is closer to the real segmentation boundary of the target object after the edge correction each time. When the local segmentation result is corrected by adopting different interactive modes, the interactive operation of a user is simulated corresponding to different edge correction strategies, and the area range of the edge is increased or decreased or reduced.
In the embodiment of the invention, a preset region mapping strategy is selected to encode the segmentation information according to an interactive mode adopted when a user interactively segments a target image to obtain encoded information for inputting an interactive segmentation model, and the target image is segmented according to the input encoded information in the interactive segmentation model to obtain a local segmentation result; at this time, the local segmentation result obtained by the first segmentation may have an edge contour which is not accurate enough, so that other suitable interactive modes can be selected to correct the local segmentation result, the segmentation result with clear edge contour is obtained step by step and is used as the interactive segmentation result of the target image, the edge contour of the target segmentation result of the image is more accurate by using and modifying the edge contour in a superposition mode of different interactive modes, the interactive segmentation mode of the image can be used in a cross mode, and the target segmentation of the image is more flexible.
Referring to fig. 2, a second embodiment of the interactive image segmentation method according to the embodiment of the present invention includes:
201. acquiring an interactive mode and segmentation information adopted when a target image is interactively segmented;
202. if the interactive mode is a first interactive mode, extracting click information in the segmentation information, and performing distance transformation coding in the target image by using the first region mapping strategy according to the click information to obtain first coding information;
203. if the interactive mode is a second interactive mode, extracting scribing information in the segmentation information, and performing adaptive scaling coding in the target image by using the second region mapping strategy according to the scribing information to obtain second coding information;
204. if the interactive mode is a third interactive mode, extracting picture frame information in the segmentation information, and performing frame coding on the target image by using the third region mapping strategy according to the picture frame information to obtain third coding information, wherein the coding information comprises first coding information, second coding information and third coding information;
in this embodiment, the interactive mode of the interactive segmentation of the target image may include at least three interactive modes, namely a first interactive mode, a second interactive mode and a third interactive mode. And aiming at the first interaction mode, taking the segmentation information as input, coding each edge pixel coordinate in the segmentation information through distance transformation by a preset first region mapping strategy, inputting each input edge pixel into a zero pixel which is closest to (preset distance) in the target image, and generating an edge image. Specifically, an n × n array mask may be used, and a point opposite to the center of the mask is defined to set a distance from an input pixel for encoding, so as to obtain first encoded information. In addition, the distance transformation may be performed by using a disc coding method to generate the first encoded information.
In this embodiment, for the second interaction mode, by using a preset second region mapping policy, in a region formed by each split edge in the split information, adaptive scaling is performed on a line length and a line width, and the split edge is adaptively scaled to an actual size of the target image according to a screen size, so as to obtain second coding information.
In this embodiment, for the third interaction mode, through a preset third region mapping policy, the position coordinates of the whole frame pixel point or the frame diagonal pixel point in the segmentation information may be obtained, and then the position coordinates of the whole frame pixel point are encoded into the position coordinates of the target object boundary in the target image, or the frame diagonal pixel point coordinates are encoded in the target image, so as to obtain third encoded information, and then the segmentation boundary may be represented by the horizontally and vertically extending sides of the frame diagonal pixel point.
205. According to the coding information, a preset interactive segmentation model is utilized to segment the target object of the target image to obtain a local segmentation result;
206. carrying out satisfaction detection on the local segmentation result, and selecting a new interaction mode adopted for the local segmentation result to be corrected each time according to the detection result and a preset interaction probability;
in this embodiment, the local segmentation result to be modified each time is modified by using an interactive segmentation model, which may be a model such as f-BRS, RITM, edgeFlow, or the like. The feature extraction network can adopt networks such as HRNet, OCRNet, segFormer and HRViT. Interaction probabilities of different interaction modes are preset, the current times of correction are combined, the results of detection of different satisfaction degrees are compared with the interaction probabilities, and which new interaction mode is selected is determined to correct the local segmentation result generated last time. The process of specifically selecting a new interaction mode is as follows:
6.1 If the detected result is passed, selecting the second interaction mode as a new interaction mode adopted for the local segmentation result to be corrected;
6.2 If the detected result is failed, counting the number of times of correcting the partial segmentation result and the random number of correcting the target image each time;
6.3 According to the correction times, comparing the correction random number with the interaction probability, and selecting a new interaction mode adopted by each local segmentation result to be corrected according to the comparison result.
Specifically, for example, for each correction iteration, the probability of using the frame and scribe line interaction mode is set to be P box 、P line Randomly setting the cycle number in the current iteration; the click is taken as a main interaction mode, and the simulation generation strategy can adopt the existing method. The interactive mode of the frame is considered to reduce the number of times of early-stage clicking, and the interactive mode of the line drawing can be used for the later-stage edge correction, so that for the correction times of the first i-th cycle, if the correction random number P of the satisfaction detection<P box Selecting an interactive mode of the picture frame; for the number of corrections of the last j cycles, the random number P is detected if satisfaction<P line The line interaction is selected. If it is notIf the corrected random numbers of both are overlapped, the latter is considered to be preferred.
Further, if the correction times are preset first times, comparing the corresponding correction random number with the scribing interaction probability, and if the corresponding correction random number is smaller than the scribing interaction probability, selecting the second interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; if the correction times are preset second times, comparing the magnitude between the corresponding correction random number and the picture frame interaction probability, and if the corresponding correction random number is smaller than the picture frame interaction probability, selecting the third interaction mode as a new interaction mode adopted by the local segmentation result to be corrected; and if the correction times are not the first times or the second times, selecting the first interaction mode as a new interaction mode adopted by the local segmentation result to be corrected, wherein the first times are greater than the second times.
207. Based on each local segmentation result to be corrected, adopting a selected new interaction mode to execute interactive segmentation simulation operation in the target image to obtain new segmentation information;
208. and according to the new segmentation information, performing edge correction on a local segmentation result to be corrected to obtain an interactive segmentation result of the target image.
In this embodiment, in the process of correcting the local segmentation result in the target image, the segmentation correction is performed on the target object boundary simulation user according to the adopted interaction mode, in the actual operation, the scribing simulation operation in the second interaction mode and the frame simulation operation in the third interaction mode are mainly adopted, and each new interaction mode for performing the correction performs the interactive segmentation simulation operation corresponding to the policy, which is specifically as follows:
7.1 If the selected new interactive mode is the first interactive mode or the second interactive mode, selecting a maximum edge area with segmentation errors from each local segmentation result to be corrected of the target image, and simulating scribing a local contour with preset size proportion in the segmentation error area to be used as new segmentation information;
7.2 If the selected new interaction mode is the third interaction mode, generating an external matrix frame in the local segmentation result to be corrected of the target image each time, and adding a disturbance effect in the external matrix frame to be used as new segmentation information.
In this embodiment, the local segmentation result and the real segmentation result include missing or excessive edge segmentation error regions, and for the first interaction mode and the second interaction mode, only the largest edge region with one or more segmentation errors may be processed in each correction. And simulating scribing operation at the line position where the preset size proportion is expanded or contracted in each maximum edge region to generate new segmentation information, and correcting the boundary of the segmentation error region once or for multiple times to ensure that the local segmentation result of the target object is more and more accurate.
In this embodiment, for the third interaction mode, an external matrix frame is generated according to a set size in the local segmentation result obtained in the previous time, and the frame boundary of the target object is gradually enlarged or reduced to correct the boundary of the local segmentation result obtained in the previous time, and so on, until the obtained result of satisfaction detection meets the condition, it may be determined that the boundary correction is completed. In addition, after the external matrix frame is added every time, a disturbance effect is added to the external matrix frame to simulate the interactive operation of the user picture frame.
With reference to fig. 3, the following describes an interactive image segmentation apparatus according to an embodiment of the present invention, and an embodiment of the interactive image segmentation apparatus according to an embodiment of the present invention includes:
an obtaining module 301, configured to obtain an interactive mode and segmentation information used when performing interactive segmentation on a target image;
the encoding module 302 is configured to select a preset region mapping strategy according to the interaction manner, and encode the segmentation information to obtain encoded information;
a segmentation module 303, configured to perform target object segmentation on the target image according to the coding information by using a preset interactive segmentation model to obtain a local segmentation result;
and the correcting module 304 is configured to perform satisfaction detection on the local segmentation result, and perform at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
In the embodiment of the invention, a preset region mapping strategy is selected to encode the segmentation information according to an interactive mode adopted when a user interactively segments a target image to obtain encoded information for inputting an interactive segmentation model, and the target image is segmented according to the input encoded information in the interactive segmentation model to obtain a local segmentation result; at this time, the local segmentation result obtained by the first segmentation may have an edge contour which is not accurate enough, so that other suitable interactive modes can be selected to correct the local segmentation result, the segmentation result with clear edge contour is obtained step by step and is used as the interactive segmentation result of the target image, the edge contour of the target segmentation result of the image is more accurate by using and modifying the edge contour in a superposition mode of different interactive modes, the interactive segmentation mode of the image can be used in a cross mode, and the target segmentation of the image is more flexible.
Referring to fig. 4, another embodiment of an interactive image segmentation apparatus according to an embodiment of the present invention includes:
an obtaining module 301, configured to obtain an interactive mode and segmentation information used when performing interactive segmentation on a target image;
the encoding module 302 is configured to select a preset region mapping strategy according to the interaction manner, and encode the segmentation information to obtain encoded information;
a segmentation module 303, configured to perform target object segmentation on the target image according to the coding information by using a preset interactive segmentation model to obtain a local segmentation result;
and the correcting module 304 is configured to perform satisfaction detection on the local segmentation result, and perform at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
Specifically, the preset region mapping policy includes a first region mapping policy, a second region mapping policy, and a third region mapping policy, and the encoding module 302 includes:
a click encoding unit 3021, configured to extract click information from the segmentation information if the interaction mode is a first interaction mode, and perform distance transform encoding in the target image according to the click information and by using the first region mapping policy to obtain first encoding information;
a line marking and encoding unit 3022, configured to extract line marking information in the segmentation information if the interactive mode is a second interactive mode, and perform adaptive scaling and encoding in the target image according to the line marking information and by using the second region mapping policy to obtain second encoding information;
a frame coding unit 3023, configured to extract frame information in the segmentation information if the interactive mode is a third interactive mode, and perform frame coding in the target image according to the frame information by using the third region mapping policy to obtain third coding information, where the coding information includes first coding information, second coding information, and third coding information.
Specifically, the modification module 304 includes:
an interaction selecting unit 3041, configured to select, according to the detection result and according to a preset interaction probability, a new interaction manner to be used for each local segmentation result to be corrected;
a simulation segmentation unit 3042, configured to execute an interactive segmentation simulation operation in the target image in a selected new interaction manner based on each local segmentation result to be corrected, so as to obtain new segmentation information;
an edge correction unit 3043, configured to perform edge correction on the local segmentation result to be corrected according to the new segmentation information, to obtain an interactive segmentation result of the target image.
Specifically, the interaction selecting unit 3041 is further configured to:
if the detected result is passed, selecting the second interaction mode as a new interaction mode adopted for the local segmentation result to be corrected;
if the detected result is failed, counting the correction times of the local segmentation result and the correction random number of the target image each time;
and comparing the correction random number with the interaction probability according to the correction times, and selecting a new interaction mode adopted by each local segmentation result to be corrected according to the comparison result.
Specifically, the interaction probability includes a picture frame interaction probability and a line drawing interaction probability, and the interaction selecting unit 3041 is further configured to:
if the correction times are preset first times, comparing the corresponding correction random number with the scribing interaction probability, and if the corresponding correction random number is smaller than the scribing interaction probability, selecting the second interaction mode as a new interaction mode adopted by the local segmentation result to be corrected;
if the correction times are preset second times, comparing the magnitude between the corresponding correction random number and the picture frame interaction probability, and if the corresponding correction random number is smaller than the picture frame interaction probability, selecting the third interaction mode as a new interaction mode adopted by the local segmentation result to be corrected;
and if the correction times are not the first times or the second times, selecting the first interaction mode as a new interaction mode adopted by the local segmentation result to be corrected at this time, wherein the first times are greater than the second times.
Specifically, the analog dividing unit 3042 is further configured to:
if the selected new interactive mode is the first interactive mode or the second interactive mode, selecting a maximum edge area with segmentation errors in a local segmentation result to be corrected of the target image each time, and simulating scribing operation on a local contour with a preset size proportion in the segmentation error area to be used as new segmentation information;
and if the selected new interaction mode is the third interaction mode, generating an external matrix frame in the local segmentation result to be corrected of the target image each time, and adding a disturbance effect in the external matrix frame as new segmentation information.
Fig. 3 and fig. 4 above describe the interactive image segmentation apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the interactive image segmentation apparatus in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 5 is a schematic structural diagram of an interactive image segmentation apparatus 500 according to an embodiment of the present invention, which may have relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a sequence of instruction operations for the interactive image segmentation apparatus 500. Still further, processor 510 may be configured to communicate with storage medium 530 to execute a series of instruction operations in storage medium 530 on interactive image segmentation apparatus 500.
The interactive image segmentation apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows server, mac OS X, unix, linux, freeBSD, and so on. It will be appreciated by those skilled in the art that the configuration of the interactive image segmentation apparatus shown in fig. 5 does not constitute a limitation of the interactive image segmentation apparatus and may comprise more or less components than those shown, or some components may be combined, or a different arrangement of components.
The present invention also provides an interactive image segmentation apparatus, the computer apparatus comprising a memory and a processor, the memory having stored therein computer-readable instructions, which, when executed by the processor, cause the processor to perform the steps of the interactive image segmentation method in the embodiments described above.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, and which may also be a volatile computer readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the interactive image segmentation method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An interactive image segmentation method, characterized in that the interactive image segmentation method comprises:
acquiring an interactive mode and segmentation information adopted when a target image is interactively segmented;
selecting a preset region mapping strategy according to the interaction mode, and coding the segmentation information to obtain coding information;
according to the coding information, a preset interactive segmentation model is utilized to segment the target object of the target image to obtain a local segmentation result;
and carrying out satisfaction detection on the local segmentation result, and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
2. The interactive image segmentation method according to claim 1, wherein the preset region mapping policy includes a first region mapping policy, a second region mapping policy, and a third region mapping policy, and the selecting the preset region mapping policy according to the interaction manner and encoding the segmentation information to obtain the encoded information includes:
if the interactive mode is a first interactive mode, extracting click information in the segmentation information, and performing distance transformation coding in the target image by using the first region mapping strategy according to the click information to obtain first coding information;
if the interactive mode is a second interactive mode, extracting scribing information in the segmentation information, and performing adaptive scaling coding in the target image by using the second region mapping strategy according to the scribing information to obtain second coding information;
if the interactive mode is a third interactive mode, extracting picture frame information in the segmentation information, and performing frame coding on the target image by using the third region mapping strategy according to the picture frame information to obtain third coding information, wherein the coding information comprises first coding information, second coding information and third coding information.
3. The interactive image segmentation method of claim 2, wherein the modifying the local segmentation result at least once according to the detection result to obtain the interactive segmentation result of the target image comprises:
selecting a new interaction mode adopted for each local segmentation result to be corrected according to the detection result and the preset interaction probability;
based on each local segmentation result to be corrected, adopting a selected new interaction mode to execute interactive segmentation simulation operation in the target image to obtain new segmentation information;
and according to the new segmentation information, performing edge correction on a local segmentation result to be corrected to obtain an interactive segmentation result of the target image.
4. The interactive image segmentation method of claim 3, wherein the selecting, according to the detection result and according to the preset interaction probability, a new interaction mode adopted for each local segmentation result to be corrected comprises:
if the detected result is passed, selecting the second interaction mode as a new interaction mode adopted for the local segmentation result to be corrected;
if the detected result is failed, counting the correction times of the local segmentation result and the correction random number of the target image each time;
and comparing the correction random number with the interaction probability according to the correction times, and selecting a new interaction mode adopted by each local segmentation result to be corrected according to the comparison result.
5. The interactive image segmentation method of claim 4, wherein the interaction probability comprises a frame interaction probability and a line interaction probability, and the comparing the modified random number and the interaction probability according to the modification times and selecting a new interaction mode for each local segmentation result to be modified according to the comparison result comprises:
if the correction times are preset first times, comparing the corresponding correction random number with the scribing interaction probability, and if the corresponding correction random number is smaller than the scribing interaction probability, selecting the second interaction mode as a new interaction mode adopted by the local segmentation result to be corrected;
if the correction times are preset second times, comparing the magnitude between the corresponding correction random number and the picture frame interaction probability, and if the corresponding correction random number is smaller than the picture frame interaction probability, selecting the third interaction mode as a new interaction mode adopted by the local segmentation result to be corrected;
and if the correction times are not the first times or the second times, selecting the first interaction mode as a new interaction mode adopted by the local segmentation result to be corrected, wherein the first times are greater than the second times.
6. The interactive image segmentation method according to claim 3, wherein the performing an interactive segmentation simulation operation on the target image based on each local segmentation result to be corrected by using a selected new interactive mode to obtain new segmentation information comprises:
if the selected new interactive mode is the first interactive mode or the second interactive mode, selecting a maximum edge area with segmentation errors in a local segmentation result to be corrected of the target image each time, and simulating scribing operation on a local contour with a preset size proportion in the segmentation error area to be used as new segmentation information;
and if the selected new interaction mode is the third interaction mode, generating an external matrix frame in the local segmentation result to be corrected of the target image each time, and adding a disturbance effect in the external matrix frame as new segmentation information.
7. An interactive image segmentation apparatus, characterized in that the interactive image segmentation apparatus comprises:
the acquisition module is used for acquiring an interactive mode and segmentation information adopted when the target image is interactively segmented;
the coding module is used for selecting a preset region mapping strategy according to the interaction mode and coding the segmentation information to obtain coding information;
the segmentation module is used for segmenting the target object of the target image by utilizing a preset interactive segmentation model according to the coding information to obtain a local segmentation result;
and the correction module is used for carrying out satisfaction detection on the local segmentation result and carrying out at least one correction on the local segmentation result according to the detection result to obtain an interactive segmentation result of the target image.
8. The interactive image segmentation apparatus of claim 7, wherein the modification module comprises:
the interaction selection unit is used for selecting a new interaction mode adopted for each local segmentation result to be corrected according to the detection result and the preset interaction probability;
the simulation segmentation unit is used for executing interactive segmentation simulation operation in the target image by adopting a selected new interactive mode based on each local segmentation result to be corrected to obtain new segmentation information;
and the edge correction unit is used for performing edge correction on the local segmentation result to be corrected according to the new segmentation information to obtain an interactive segmentation result of the target image.
9. An interactive image segmentation apparatus, characterized in that the interactive image segmentation apparatus comprises: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the interactive image segmentation apparatus to perform the steps of the interactive image segmentation method of any one of claims 1-6.
10. A computer-readable storage medium having instructions stored thereon, which when executed by a processor implement the steps of the interactive image segmentation method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211203622.5A CN115830046A (en) | 2022-09-29 | 2022-09-29 | Interactive image segmentation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211203622.5A CN115830046A (en) | 2022-09-29 | 2022-09-29 | Interactive image segmentation method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115830046A true CN115830046A (en) | 2023-03-21 |
Family
ID=85524234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211203622.5A Pending CN115830046A (en) | 2022-09-29 | 2022-09-29 | Interactive image segmentation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115830046A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237382A (en) * | 2023-11-08 | 2023-12-15 | 湖南自兴智慧医疗科技有限公司 | Chromosome image complex region segmentation method and device based on interactive guidance |
-
2022
- 2022-09-29 CN CN202211203622.5A patent/CN115830046A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117237382A (en) * | 2023-11-08 | 2023-12-15 | 湖南自兴智慧医疗科技有限公司 | Chromosome image complex region segmentation method and device based on interactive guidance |
| CN117237382B (en) * | 2023-11-08 | 2024-02-23 | 湖南自兴智慧医疗科技有限公司 | Chromosome image complex region segmentation method and device based on interactive guidance |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210004629A1 (en) | End-to-end text recognition method and apparatus, computer device and readable medium | |
| CN111191649A (en) | Method and equipment for identifying bent multi-line text image | |
| CN110310305B (en) | Target tracking method and device based on BSSD detection and Kalman filtering | |
| US20210248760A1 (en) | Feature processing method and device for motion trajectory, and computer storage medium | |
| US20230237777A1 (en) | Information processing apparatus, learning apparatus, image recognition apparatus, information processing method, learning method, image recognition method, and non-transitory-computer-readable storage medium | |
| CN115830046A (en) | Interactive image segmentation method, device, equipment and storage medium | |
| CN116843834A (en) | Three-dimensional face reconstruction and six-degree-of-freedom pose estimation method, device and equipment | |
| CN115713633A (en) | Visual SLAM method, system and storage medium based on deep learning in dynamic scene | |
| CN113486879B (en) | Image area suggestion frame detection method, device, equipment and storage medium | |
| Rana et al. | Dynamite: Dynamic query bootstrapping for multi-object interactive segmentation transformer | |
| CN110705554B (en) | Image processing method and device | |
| CN117611679A (en) | Ping-pong table calibrating method, calibrating device and storage medium | |
| CN112084804A (en) | Working method for intelligently acquiring complementary pixels aiming at information-missing bar codes | |
| CN112084855A (en) | Outlier elimination method for video stream based on improved RANSAC method | |
| US20230071291A1 (en) | System and method for a precise semantic segmentation | |
| US20230360262A1 (en) | Object pose recognition method based on triangulation and probability weighted ransac algorithm | |
| CN113971686A (en) | Target tracking method based on background restoration and capsule network | |
| CN112418344A (en) | Training method, target detection method, medium and electronic device | |
| CN112347865A (en) | Bill correction method based on key point detection | |
| CN111260723A (en) | Barycenter positioning method of bar and terminal equipment | |
| CN117237441B (en) | Sub-pixel positioning method, sub-pixel positioning system, electronic equipment and medium | |
| CN116486427B (en) | Text recognition method, device, equipment and storage medium | |
| CN113139625B (en) | Model training method, electronic equipment and storage medium thereof | |
| CN113158855B (en) | Remote sensing image auxiliary processing method and device based on online learning | |
| JP7344916B2 (en) | Information processing device, information processing method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |