CN111612791A - Image segmentation method and device, electronic equipment and storage medium - Google Patents

Image segmentation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111612791A
CN111612791A CN202010403656.3A CN202010403656A CN111612791A CN 111612791 A CN111612791 A CN 111612791A CN 202010403656 A CN202010403656 A CN 202010403656A CN 111612791 A CN111612791 A CN 111612791A
Authority
CN
China
Prior art keywords
image
resolution
target
segmentation
filtered
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010403656.3A
Other languages
Chinese (zh)
Other versions
CN111612791B (en
Inventor
李马丁
徐青
章佳杰
郑云飞
于冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Dajia Internet Information Technology Co Ltd
Original Assignee
Beijing Dajia Internet Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dajia Internet Information Technology Co Ltd filed Critical Beijing Dajia Internet Information Technology Co Ltd
Priority to CN202010403656.3A priority Critical patent/CN111612791B/en
Publication of CN111612791A publication Critical patent/CN111612791A/en
Application granted granted Critical
Publication of CN111612791B publication Critical patent/CN111612791B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The disclosure relates to an image segmentation method, an image segmentation device, electronic equipment and a storage medium, relates to the technical field of internet, and is used for solving the problem of low image segmentation efficiency in the related technology, and the image segmentation method comprises the following steps: acquiring an image set to be processed, wherein the image set comprises an original image and at least one target image, and the target image is obtained by down-sampling according to an image with a resolution larger than that of the image set in the image set; sequentially acquiring segmented images corresponding to the resolutions according to the sequence of the resolutions from small to large, and performing edge filtering processing on the segmented images corresponding to the resolutions according to the images corresponding to the resolutions in the image set to obtain filtered images corresponding to the resolutions; and acquiring a target segmentation image corresponding to the original image according to the filtering image with the maximum resolution. According to the image segmentation method and device, the target image with the minimum resolution is subjected to image segmentation, the calculation amount is small, and the processes of downsampling and the like are easy to accelerate, so that the image segmentation efficiency can be effectively improved.

Description

Image segmentation method and device, electronic equipment and storage medium
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to an image segmentation method and apparatus, an electronic device, and a storage medium.
Background
With the development of artificial intelligence, the application field of image processing is more and more extensive, for example, the core of high and new technologies such as face recognition, fingerprint recognition, vehicle license plate recognition, Chinese character recognition, medical image recognition and the like is related knowledge such as image processing, and the image segmentation technology is a key technology in the image processing link.
Image segmentation is a technique and process that divides an image into several specific regions with unique properties and proposes an object of interest. It is a key step from image processing to image analysis. The related image segmentation methods are mainly classified into the following categories: a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, a wavelet transform-based segmentation method, a neural network-based segmentation method, a particular theory-based segmentation method, and the like. From a mathematical point of view, image segmentation is the process of dividing a digital image into mutually disjoint regions. The process of image segmentation is also a labeling process, i.e. pixels belonging to the same region are assigned the same number. However, the image segmentation method in the related art is inefficient.
Disclosure of Invention
The present disclosure provides an image segmentation method, an image segmentation apparatus, an electronic device, and a storage medium, so as to at least solve the problem of low image segmentation efficiency in the related art. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided an image segmentation method, including:
acquiring an image set to be processed, wherein the image set comprises an original image and at least one target image, the target image is obtained by down-sampling according to an image with a resolution greater than that of the target image in the image set, and the resolution of the original image is the maximum;
sequentially acquiring segmented images corresponding to the resolutions according to the sequence of the resolutions from small to large, and performing edge filtering processing on the segmented images corresponding to the resolutions according to the images corresponding to the resolutions in the image set to obtain filtered images corresponding to the resolutions, wherein the segmented image with the minimum resolution is obtained by performing image segmentation on a target image with the minimum resolution in the image set, and the segmented images with other resolutions are obtained by performing up-sampling on the filtered image obtained last time;
and acquiring a target segmentation image corresponding to the original image according to the filtering image with the maximum resolution.
In an optional implementation manner, the performing, according to the image corresponding to each resolution in the image set, edge filtering on the segmented image corresponding to each resolution to obtain a filtered image corresponding to each resolution specifically includes:
regarding any resolution, taking the image in the image set corresponding to the resolution as a guide image;
and performing guiding filtering on the segmentation image corresponding to the resolution according to the guiding image to obtain a filtering image corresponding to the resolution.
In an optional implementation manner, after performing edge filtering processing on segmented images corresponding to each resolution according to an image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution, before acquiring a target segmented image corresponding to the original image according to a filtered image corresponding to a maximum resolution in the image set, specifically includes:
comparing the images in the image set corresponding to the resolution with the filtered images corresponding to the resolution;
and carrying out fuzzy processing on the filtered image corresponding to the resolution according to the comparison result.
In an optional implementation manner, the performing, according to the image corresponding to each resolution in the image set, edge filtering on the segmented image corresponding to each resolution to obtain a filtered image corresponding to each resolution includes:
for any resolution, sequentially carrying out edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution according to the image in the image set corresponding to the resolution to obtain a filtering image corresponding to the resolution;
when the edge filtering processing is carried out on the first target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution; when the edge filtering processing is performed on other target segmentation objects, the edge filtering processing is performed on the filtered image after the edge filtering processing is performed last time.
In an optional implementation manner, the performing, according to the image corresponding to each resolution in the image set, edge filtering on the segmented image corresponding to each resolution to obtain a filtered image corresponding to each resolution includes:
aiming at any resolution, according to the image in the image set corresponding to the resolution, performing edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution to obtain a filtering image corresponding to each target segmentation object; when the edge filtering processing is carried out on each target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution;
and taking the filtered image corresponding to each target segmentation object as the filtered image corresponding to the resolution.
In an alternative embodiment, there is one filtered image of maximum resolution;
the obtaining of the target segmentation image corresponding to the original image according to the filtered image with the maximum resolution includes:
and carrying out binarization processing on the filtered image with the maximum resolution to obtain a target segmentation image corresponding to the original image.
In an alternative embodiment, the maximum resolution of the filtered image is multiple;
the obtaining of the target segmentation image corresponding to the original image according to the filtered image with the maximum resolution includes:
determining the pixel value of each pixel point in the target filtering image according to the position of each target segmentation object in the corresponding filtering image;
and carrying out binarization processing on the target filtering image to obtain a target segmentation image corresponding to the original image.
In an optional implementation manner, the determining a pixel value of each pixel point in the target filtered image according to a position of each target segmentation object in the corresponding filtered image includes:
and aiming at any pixel point, taking the maximum value in the pixel values of the pixel points in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
In an optional implementation manner, the determining a pixel value of each pixel point in the target filtered image according to a position of each target segmentation object in the corresponding filtered image includes:
for a target pixel point, the pixel value of the target pixel point position in the image obtained after the segmentation image with the minimum resolution is up-sampled to the maximum resolution is taken as the pixel value of the pixel point at the corresponding position in the target filtering image, wherein the target pixel point is the pixel point at the position where the pixel value in the filtering image corresponding to each target segmentation object is the preset pixel value; or
And regarding non-target pixel points, taking the maximum value in the pixel values of the non-target pixel point positions in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
According to a second aspect of the embodiments of the present disclosure, there is provided an image segmentation apparatus including:
the image processing device comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is configured to acquire an image set to be processed, the image set comprises an original image and at least one target image, the target image is obtained by down-sampling images with a resolution higher than that of the target image in the image set, and the resolution of the original image is the maximum;
the processing unit is configured to sequentially acquire segmented images corresponding to the resolutions according to the sequence of the resolutions from small to large, and perform edge filtering processing on the segmented images corresponding to the resolutions according to the images corresponding to the resolutions in the image set to obtain filtered images corresponding to the resolutions, wherein the segmented image with the minimum resolution is obtained by performing image segmentation on a target image with the minimum resolution in the image set, and the segmented images with other resolutions are obtained by performing up-sampling on the filtered image obtained last time;
and the second acquisition unit is configured to execute filtering image according to the maximum resolution and acquire a target segmentation image corresponding to the original image.
In an alternative embodiment, the processing unit is specifically configured to perform:
regarding any resolution, taking the image in the image set corresponding to the resolution as a guide image;
and performing guiding filtering on the segmentation image corresponding to the resolution according to the guiding image to obtain a filtering image corresponding to the resolution.
In an optional implementation manner, after the processing unit performs edge filtering processing on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution, before the second obtaining unit obtains the target segmented image corresponding to the original image according to the filtered image corresponding to the maximum resolution in the image set, the processing unit is further configured to perform:
comparing the images in the image set corresponding to the resolution with the filtered images corresponding to the resolution;
and carrying out fuzzy processing on the filtered image corresponding to the resolution according to the comparison result.
In an alternative embodiment, the processing unit is specifically configured to perform:
for any resolution, sequentially carrying out edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution according to the image in the image set corresponding to the resolution to obtain a filtering image corresponding to the resolution;
when the edge filtering processing is carried out on the first target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution; when the edge filtering processing is performed on other target segmentation objects, the edge filtering processing is performed on the filtered image after the edge filtering processing is performed last time.
In an alternative embodiment, the processing unit is specifically configured to perform:
aiming at any resolution, according to the image in the image set corresponding to the resolution, performing edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution to obtain a filtering image corresponding to each target segmentation object; when the edge filtering processing is carried out on each target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution;
and taking the filtered image corresponding to each target segmentation object as the filtered image corresponding to the resolution.
In an alternative embodiment, there is one filtered image of maximum resolution;
the second acquisition unit is specifically configured to perform:
and carrying out binarization processing on the filtered image with the maximum resolution to obtain a target segmentation image corresponding to the original image.
In an alternative embodiment, the maximum resolution of the filtered image is multiple;
the second acquisition unit is specifically configured to perform:
determining the pixel value of each pixel point in the target filtering image according to the position of each target segmentation object in the corresponding filtering image;
and carrying out binarization processing on the target filtering image to obtain a target segmentation image corresponding to the original image.
In an alternative embodiment, the second obtaining unit is specifically configured to perform:
and aiming at any pixel point, taking the maximum value in the pixel values of the pixel points in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
In an alternative embodiment, the second obtaining unit is specifically configured to perform:
for a target pixel point, the pixel value of the target pixel point position in the image obtained after the segmentation image with the minimum resolution is up-sampled to the maximum resolution is taken as the pixel value of the pixel point at the corresponding position in the target filtering image, wherein the target pixel point is the pixel point at the position where the pixel value in the filtering image corresponding to each target segmentation object is the preset pixel value; or
And regarding non-target pixel points, taking the maximum value in the pixel values of the non-target pixel point positions in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the image segmentation method according to any one of the first aspect of the embodiments of the present disclosure.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the image segmentation method according to any one of the first aspect of the embodiments of the present disclosure.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product, which, when run on an electronic device, causes the electronic device to perform a method that implements any of the above first aspect and the first aspect of embodiments of the present disclosure may relate to.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
since the embodiment of the present disclosure may obtain a plurality of target images by down-sampling an original image or other target images, and only the target image with the minimum resolution is image-segmented, and the segmented images corresponding to other images are all obtained by processing the segmented result of the target image with the minimum resolution, since the target image with the minimum resolution is much smaller than the resolution of the original image with the maximum resolution, a lot of computation may be saved in the most time-consuming image segmentation process, and then after obtaining the segmented image with the minimum resolution, the segmented image may be subjected to certain boundary segmentation, up-sampling, and other processing, so as to obtain the segmented target image with the resolution consistent with the original image resolution, wherein the down-sampling, up-sampling, boundary segmentation, and the like, may be effectively accelerated by hardware lines, and compared with a general image segmentation algorithm, the calculated amount is smaller, so that the performance can be greatly improved, and the image segmentation efficiency is effectively improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a schematic diagram illustrating an application scenario in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a method of image segmentation in accordance with an exemplary embodiment;
FIG. 3 is a schematic diagram illustrating an image segmentation pyramid in accordance with an exemplary embodiment;
FIG. 4 is a flow diagram illustrating a complete method of first image segmentation in accordance with an exemplary embodiment;
FIG. 5 is a flow diagram illustrating a complete method of second image segmentation in accordance with an exemplary embodiment;
FIG. 6 is a block diagram illustrating an image segmentation apparatus according to an exemplary embodiment;
FIG. 7 is a block diagram illustrating an electronic device in accordance with an exemplary embodiment;
FIG. 8 is a block diagram illustrating a computing device in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Some of the words that appear in the text are explained below:
1. the term "and/or" in the embodiments of the present disclosure describes an association relationship of associated objects, and means that there may be three relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
2. The term "electronic device" in the embodiments of the present disclosure may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.
3. The term "image pyramid" in the embodiments of the present disclosure is a kind of multi-scale representation of an image, and is an effective but conceptually simple structure for interpreting an image in multiple resolutions. A pyramid of an image is a series of image sets of progressively lower resolution arranged in a pyramid shape and derived from the same original image. It is obtained by down-sampling in steps, and sampling is not stopped until a certain end condition is reached. We compare the images one level at a time to a pyramid, with the higher the level, the smaller the image and the lower the resolution. A pyramid of an image is a series of sets of images arranged in a pyramid shape with progressively lower resolutions. The bottom of the pyramid is a high resolution representation of the image to be processed, while the top is an approximation of the low resolution. When moving to the upper layer of the pyramid, the size and resolution are reduced.
4. The term "edge-preserving filtering" in the embodiments of the present disclosure refers to a filtering method that preserves the edges (detail information) of an image. Guided Filtering (Guided Filtering) and Bilateral Filtering (BF), least squares Filtering (WLS) are three large Edge-preserving (Edge-preserving) filters. Of course, the function of the guide filtering is not just edge preservation, but it becomes an edge preservation filter only when the guide map is the original. It has corresponding application in image defogging and image matting.
5. The term "guided filtering" in the embodiments of the present disclosure is to use a guided image as a filtering content image, implement local linear function expression on the guided image, implement various linear transformations, and output a deformed guided filtering image. The guide image may be different from or identical to the target image, as desired. Suppose I is a guide image, p is a target image, q is a guide filtered output image, and the guide filtering describes the relationship between the guide image I and the output image q as a local linear model. For this algorithm, when I ═ p, i.e., the original picture and the guide picture are the same side picture, the algorithm becomes an edge-preserving filter.
The application scenario described in the embodiment of the present disclosure is for more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not form a limitation on the technical solution provided in the embodiment of the present disclosure, and as a person having ordinary skill in the art knows, with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present disclosure is also applicable to similar technical problems. Wherein, in the description of the present disclosure, unless otherwise indicated, "plurality" means.
The following briefly introduces an application scenario of the embodiment of the present disclosure:
fig. 1 is a schematic view of an application scenario of the embodiment of the present disclosure. The application scenario diagram includes two terminal devices 110 and a server 130, and the terminal devices 110 can log in the related interface 120. The terminal device 110 and the server 130 can communicate with each other through a communication network.
In an alternative embodiment, the communication network is a wired network or a wireless network.
In the embodiment of the present disclosure, the terminal device 110 is an electronic device used by a user, and the electronic device may be a computer device having a certain computing capability and running instant messaging software and a website or social software and a website, such as a personal computer, a mobile phone, a tablet computer, a notebook, an e-book reader, and the like. Each terminal device 110 is connected to a server 130 through a wireless network, and the server 130 is a server or a server cluster or a cloud computing center formed by a plurality of servers, or is a virtualization platform.
Optionally, the server 130 may also have an image database that may store a large number of images.
In the embodiment of the present disclosure, the terminal device 110 may directly segment an image according to the image segmentation method in the embodiment of the present disclosure, and display the image to the user through the interface 120; or, when receiving a request triggered by a user, the terminal device 110 sends a local image to the server 130, the server 130 receives the image and then segments the image, and sends the segmentation result to the terminal device 110, and the terminal device 110 displays the segmentation result to the user through the interface 120. Optionally, when the terminal device 110 detects a request triggered by a user, the terminal device 110 may also send a request to the server 130, the server 130 searches for an image according to the request sent by the terminal device, performs image segmentation on the searched image, sends the segmentation result to the terminal device 110, and then the terminal device 110 displays the image to the user through the interface 120, and so on. Among other things, image segmentation techniques may be applied in many scenarios, such as medical image segmentation, face detection, and so on.
The following describes an image segmentation method according to an embodiment of the present disclosure in detail:
fig. 2 is a flowchart illustrating an image segmentation method according to an exemplary embodiment, as shown in fig. 2, including the following steps.
In step S21, acquiring an image set to be processed, where the image set includes an original image and at least one target image, the target image is obtained by down-sampling an image of the image set that is greater than a resolution of the target image, and the resolution of the original image is the largest;
in step S22, sequentially acquiring segmented images corresponding to each resolution in the order of resolution from small to large, and performing edge filtering processing on the segmented images corresponding to each resolution according to the images corresponding to each resolution in the image set to obtain filtered images corresponding to each resolution, where the segmented image of the minimum resolution is obtained by image segmentation of the target image of the minimum resolution in the image set, and the segmented images of other resolutions are obtained by upsampling the filtered image obtained last time;
the target segmentation object refers to an object that needs to be segmented when segmenting the image content, for example, in the case of binary classification, the image can be divided into a foreground and a background except the foreground, where the foreground is the target segmentation object. Taking face detection as an example, the face can be regarded as a foreground, and other parts can be regarded as a background, and the face is a target segmentation object at the moment. In the multi-classification case, there are a plurality of target segmentation objects, for example, when an image is classified into three, and the image is divided into three parts of sky, grassland and road, the sky, grassland and road all belong to the target segmentation objects.
In the embodiment of the present disclosure, when images are sorted according to the size of the image resolution, the images may be sorted according to the order of the image resolution from low to high or from high to low, which is not specifically limited herein. Based on the above process, an image pyramid, an inverted pyramid, or the like can be constructed.
It should be noted that, in the embodiment of the present disclosure, sorting images according to the size of the image resolution is only a feasible sorting manner, and in addition, sorting images according to the size, the dimension, the area, and the like of the images may achieve the same effect, that is, in the embodiment of the present disclosure, the image resolution may be equally replaced by descriptions that can represent the size of the images, such as the image dimension, the image area, and the like, and is not limited specifically herein.
In step S23, a target segmented image corresponding to the original image is acquired from the filtered image of the maximum resolution.
According to the scheme, a plurality of target images can be obtained based on down-sampling, only the target image with the minimum resolution is subjected to image segmentation, the segmented images corresponding to other images are obtained by processing the segmented image obtained by the target image with the minimum resolution, the target image with the minimum resolution is much smaller than the original image with the maximum resolution, so that a lot of calculation amount can be saved in the most time-consuming image segmentation process, then after the segmented image with the minimum resolution is obtained, the segmented image is subjected to certain boundary segmentation, up-sampling and other processing, so that the target segmented image with the resolution consistent with the original image resolution can be obtained, wherein the down-sampling, up-sampling, boundary segmentation and the like can be effectively accelerated through a hardware line, and compared with a common image segmentation algorithm, the calculation amount is smaller, so that the performance can be greatly improved, the efficiency of image segmentation is effectively improved.
In the embodiment of the present disclosure, the target image is obtained by down-sampling, and specifically, the target image may be obtained by down-sampling the original image or may be obtained by down-sampling another target image having a resolution higher than that of the target image.
Taking the image pyramid shown in fig. 3 as an example, the pyramid has a total of n layers, and the resolution of the image gradually decreases from bottom to top. The image of the bottom layer (the n layer) is an original image and has the largest resolution, the images of the rest n-1 layers are target images, and the target image resolution of the top layer (the 1 st layer) is the smallest. It is assumed that in each adjacent two-layer image, the resolution of the next layer image is twice that of the previous layer image. Assuming that the resolutions of the layers from top to bottom are respectively: k1<K2<K3<…<KnOf which the mostResolution of the bottom layer is KnThe resolution of the top layer is K1
Taking the target image shown in the n-2 th layer as an example, the resolution of the target image is Kn-2The image having a resolution greater than that of the target image has an original image of the nth layer (resolution K)n) And target image of layer n-1 (resolution K)n-1) Therefore, the target image can be obtained by performing down-sampling of 0.25 times from the original image of the nth layer, or can be obtained by performing down-sampling of 0.5 times from the image of the (n-1) th layer.
In the embodiment of the present disclosure, the image set includes at least two images, an original image and at least one target image, where the resolution of different images in the image set is different, and the resolution of the original image is the largest. In the embodiment of the present disclosure, the lower sampling multiple may be set as appropriate, and is generally 0.5.
When only one target image is included in the image set, that is, when two images are included in the image set, there is no iterative process in step S22; when the number of target images included in the image set is greater than one, that is, when at least three images are included in the image set, an iterative process exists in step S22. That is, the simplest case is when only two images are included in the image set, and the number of iterations increases as the number of images in the image set increases.
The following describes the implementation of step S22 in detail according to the number of images in the image set:
example one: the image set only comprises two images, and at the moment, the image set only comprises one target image, and the target image is obtained by downsampling an original image. In this case, a two-layer image pyramid can be obtained by sorting the images according to the resolution of the images. When images are sorted by resolution, the image of the smallest resolution (layer 1) is adjacent to the image of the largest resolution (layer 2).
The following takes two-class segmentation as an example (for example, foreground segmentation and image content segmentation into two classes), and describes a specific implementation process of step S22:
firstly, performing operations such as image segmentation on a target image with the minimum resolution (namely, an image of a layer 1) in an image pyramid to obtain an image segmentation result with the minimum resolution, namely, a segmented image with the minimum resolution;
then, according to the target image (the image of the layer 1) with the minimum resolution in the image set, carrying out edge filtering processing on the segmentation image with the minimum resolution to obtain a filtering image with the minimum resolution;
then, the filtered image with the minimum resolution is up-sampled to obtain a segmented image with the resolution size consistent with that of the layer 2 image (namely consistent with that of the original image);
then, according to the image (namely the original image) of the 2 nd layer, carrying out edge filtering processing on the segmentation image corresponding to the 2 nd layer to obtain a filtering image with the maximum resolution;
and finally, acquiring a target segmentation image corresponding to the original image based on the filtering image with the maximum resolution.
Example two, assuming that the image set includes at least three images, that is, the image set includes an original image and at least two target images, in this case, sorting the images according to the size of the image resolution may obtain an image pyramid of at least three layers, where the image with the minimum resolution is located at the top layer, and the image with the maximum resolution is located at the bottom layer. In this case, step S22 is an iterative process, and the specific implementation manner of the process of sequentially obtaining the segmented images corresponding to each resolution in the order from small to large in resolution, and performing edge filtering on the segmented images corresponding to each resolution according to the images corresponding to each resolution in the image set to obtain the filtered images corresponding to each resolution is as follows:
firstly, performing operations such as image segmentation on a target image with the minimum resolution (namely, an image of a layer 1) in an image pyramid to obtain an image segmentation result with the minimum resolution, namely, a segmented image with the minimum resolution;
then, according to the target image with the minimum resolution in the image set, performing edge filtering processing on the segmented image with the minimum resolution to obtain a filtered image with the minimum resolution; performing primary up-sampling on the filtering image with the minimum resolution ratio until the size of the filtering image is consistent with that of the image of the layer 2, and obtaining a segmentation image corresponding to the layer 2; performing edge filtering processing on the obtained segmented image according to the layer 2 image to obtain a filtered image corresponding to the layer 2 image;
then, performing up-sampling on the obtained filtering image until the size of the filtering image is consistent with that of the 3 rd layer image, and obtaining a segmentation image corresponding to the 3 rd layer image; performing edge filtering processing on the obtained segmented image according to the 3 rd layer image to obtain a filtering image corresponding to the 3 rd layer image;
continuously repeating the steps until the bottom layer, namely the nth layer is reached, and obtaining a filtering image corresponding to the nth layer image, namely a filtering image with the maximum resolution;
and finally, acquiring a target segmentation image corresponding to the original image based on the filtered image with the maximum resolution.
The following details are given for the implementation of S22 and S23:
in an optional implementation manner, when performing edge filtering processing on a segmented image corresponding to each resolution according to an image corresponding to each resolution in an image set to obtain a filtered image corresponding to each resolution, a specific process is as follows:
regarding any resolution, taking the image in the image set corresponding to the resolution as a guide image; and performing guiding filtering on the segmentation image corresponding to the resolution according to the guiding image to obtain a filtering image corresponding to the resolution.
That is, a method of guided filtering is adopted, an image in an image set corresponding to the resolution is taken as a guide image, that is, a guide map, and the guide filtering is performed on the segmented image corresponding to the resolution, so that the filtered image corresponding to the resolution obtained after filtering is substantially similar to the segmented image corresponding to the resolution, but the texture part is similar to the guide image, and the effect of edge smoothing can be realized through edge-guided filtering, so that the segmentation boundary becomes clearer.
It should be noted that, besides the guided filtering, other edge-preserving filtering methods are also applicable to the embodiments of the present disclosure, and the image is filtered while preserving detail information such as edges in the image.
In an optional implementation manner, after performing edge filtering processing on a segmented image corresponding to each resolution according to an image corresponding to each resolution in an image set to obtain a filtered image corresponding to each resolution, before acquiring a target segmented image corresponding to an original image according to a filtered image corresponding to a maximum resolution in the image set, specifically include: comparing the image in the image set corresponding to the resolution with the filtering image corresponding to the resolution; and carrying out fuzzy processing on the filtered image corresponding to the resolution according to the comparison result.
In the embodiment of the present disclosure, after a filtered image corresponding to any resolution is obtained through edge-oriented filtering or other filtering manners, the filtered image may be further compared with an image (an original image or a target image) corresponding to the resolution in an image set, and an edge portion of a conflict occurring in the image corresponding to the resolution in the image set is subjected to blurring processing according to a comparison result, so that the edge portion of a final result is kept consistent with the edge portion of the image corresponding to the resolution in the image set as much as possible, and accuracy of image segmentation is ensured.
After the above process, the filtered image with the maximum resolution, that is, the filtered image with the resolution consistent with that of the original image, can be obtained, and then the target segmentation image corresponding to the original image can be determined based on the filtered image.
It should be noted that, in the above processes, for the case of two classifications, there is one divided image or one filtered image corresponding to each resolution; in the multi-classification case, the original image will include at least two target segmentation objects, and for each resolution, one or more segmentation images or filtering images corresponding to each resolution may be used, depending on the filtering method. When the number of filtered images is different, the manner in which the target segmented image of the original image is determined is also different. The following is a detailed description:
in an optional embodiment, when performing edge filtering processing on a segmented image corresponding to each resolution according to an image corresponding to each resolution in an image set to obtain a filtered image corresponding to each resolution, the following two filtering methods may be adopted:
the method comprises the steps that in a first filtering mode, aiming at any resolution, according to images in an image set corresponding to the resolution, edge filtering processing is sequentially carried out on each target segmentation object in segmentation images corresponding to the resolution, and then filtering images corresponding to the resolution are obtained; when the edge filtering processing is carried out on the first target segmentation object, the edge filtering processing is carried out on a segmentation image corresponding to the resolution; when the edge filtering processing is performed on other target segmentation objects, the edge filtering processing is performed on the filtered image after the edge filtering processing is performed last time.
For any one resolution, e.g. Kn(bottom layer of image pyramid):
taking the two classifications as an example, the target segmentation object only includes one, for example, foreground, and at this time, only the edge of the foreground in the segmentation image needs to be filtered, so that the filtered image corresponding to the resolution can be obtained.
Taking multi-classification as an example, the target segmented objects include at least two objects, such as sky, grassland and roads, at this time, the edge of the first target segmented object may be filtered on the segmented image, and then the edge of the second target segmented object may be filtered on the filtering result, and finally the edge of the third target segmented object may be filtered on the filtering result of the edge of the second target segmented object, so as to obtain the filtered image corresponding to the resolution.
In this way, in both the binary and multi-class cases, each resolution corresponds to only one filtered image. That is to say, the image segmentation method in the embodiment of the present disclosure is not only suitable for the case of two-class classification, but also suitable for the case of multiple-class classification, and can effectively improve the efficiency of image segmentation.
In the above embodiment, the order of the target segmentation objects may be preset or random, and is not particularly limited herein.
In the second filtering mode, aiming at any resolution, according to the image in the image set corresponding to the resolution, edge filtering processing is carried out on each target segmentation object in the segmentation image corresponding to the resolution to obtain a filtering image corresponding to each target segmentation object; when the edge filtering processing is carried out on each target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution; and taking the filtered image corresponding to each target segmentation object as the filtered image corresponding to the resolution.
For any one resolution, e.g. Kn(bottom layer of image pyramid):
taking the two classifications listed in the first filtering method as an example, the target segmentation object only includes one, and at this time, the filtering process is only required to be performed on the edge of the foreground in the segmented image, so that the filtered image corresponding to the resolution can be obtained.
Still taking the multi-classification listed in the first filtering manner as an example, the target segmentation objects include sky, grassland, and roads, and for each target segmentation object, corresponding filtering processing needs to be performed on the segmentation image corresponding to each target segmentation object to obtain a filtering image corresponding to each target segmentation object.
Specifically, in this embodiment, even in the 1 st layer, a plurality of target segmented objects correspond to only one segmented image, and in each layer other than the 1 st layer, each target segmented object has a corresponding segmented image. For example at a resolution of K2In the process, namely, layer 2, aiming at 3 target segmentation objects, respectively carrying out filtering processing on the edges of the 3 target segmentation objects on a segmentation image to obtain filtered images corresponding to the 3 target segmentation objects, wherein each target segmentation object corresponds to one filtered image, then carrying out up-sampling on the 3 filtered images to obtain 3 segmentation images corresponding to the layer 3 image, each segmentation image corresponds to one target segmentation object, at the moment, filtering processing can be carried out on the edges of the target segmentation objects corresponding to the target segmentation objects on the segmentation images corresponding to the target segmentation objects to obtain the filtered images corresponding to the target segmentation objects,continuously iterating the above process at a resolution of KnThen there will be 3 sheets with resolution KnThe segmentation images respectively correspond to 3 target segmentation objects, filtering processing is carried out on the edge of the sky on the first segmentation image, then a filtering image corresponding to the sky can be obtained, similarly, filtering processing is carried out on the edge of the grassland on the second segmentation image, then a filtering image corresponding to the grassland can be obtained, and filtering processing is carried out on the edge of the highway on the third segmentation image, then a filtering image corresponding to the highway can be obtained.
In this manner, one target segmented object corresponds to one filtered image, so that in the case of multi-classification, there are a plurality of target segmented objects, and thus one resolution corresponds to a plurality of filtered images. In the embodiment of the disclosure, the multi-classification is only required to be regarded as a process of multiple two classifications, and similar processing can be performed on only one class in turn, so that the acceleration of the multi-classification image segmentation algorithm is realized.
As can be seen from the above embodiments, one or more filtered images corresponding to one resolution may be provided; therefore, the manner of obtaining the target segmented image corresponding to the original image is different according to the difference of the number of the filtered images corresponding to the maximum resolution. Based on this, the following describes the manner of obtaining the target segmented image corresponding to the original image.
In case one, one filtered image corresponding to one resolution is one.
In this case, the maximum resolution filter image is also one, and when the target segmentation image corresponding to the original image is acquired according to the maximum resolution filter image, the maximum resolution filter image may be directly subjected to binarization processing to acquire the target segmentation image corresponding to the original image.
Namely, a filtered image obtained by edge-oriented filtering at the bottom layer of the image pyramid needs to be subjected to binarization processing through threshold processing, so as to obtain a final target segmentation image.
In case two, a plurality of filtered images are provided for one resolution.
When a target segmentation image corresponding to an original image is obtained according to the filtered image with the maximum resolution, firstly, the pixel value of each pixel point in the target filtered image is determined according to the position of each target segmentation object in the corresponding filtered image; and then directly carrying out binarization processing on the target filtering image to obtain a target segmentation image corresponding to the original image.
In the embodiment of the present disclosure, there are many ways to obtain a target filtered image based on multiple filtered images corresponding to the maximum resolution, two of which are listed below:
in an optional implementation manner, when determining the pixel value of each pixel point in the target filtered image according to the position of each target segmentation object in the corresponding filtered image, the following manner may be adopted:
and aiming at any pixel point, taking the maximum value in the pixel values of the pixel points in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
For resolution KnStill taking the above listed multi-classification case as an example, the sky, the grassland and the highway respectively correspond to a filter image, which is assumed to be a1, a2 and A3, where the 3 filter images can be understood as a probability map, the pixel value of each pixel point in a1 represents the probability that the pixel point is sky, similarly, the pixel value of each pixel point in a2 represents the probability that the pixel point is grassland, the pixel value of each pixel point in A3 represents the probability that the pixel point is highway, for a pixel point at any position, the pixel value of the pixel point in a1 is a1, the pixel value in a2 is a2, the pixel value in A3 is A3, and it is assumed that a1 is a1>a2>a3, determining that the pixel value of the pixel point in the target filtering image is a 1; and determining the pixel value of the pixel point at each position in the target filtering image by adopting the same mode aiming at other pixel points to obtain the target filtering image.
In the above embodiment, the target filtered image can be determined based on the pixel values of the pixels in the filtered image corresponding to each target segmentation object, and a simple and efficient method for determining the target filtered image is provided.
In another optional implementation manner, when determining the pixel value of each pixel point in the target filtered image according to the position of each target segmentation object in the corresponding filtered image, the following manner may be adopted:
for a target pixel point, the pixel value of the position of the target pixel point in the image obtained after the segmentation image with the minimum resolution is up-sampled to the maximum resolution is used as the pixel value of the pixel point at the corresponding position in the target filtering image, wherein the target pixel point is the pixel point at the position where the pixel value in the filtering image corresponding to each target segmentation object is the preset pixel value; and regarding the non-target pixel points, taking the maximum value in the pixel values of the non-target pixel point positions in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
In this way, the pixel points in one image are divided into two categories, namely a target pixel point and a non-target pixel point, and the multi-classification case listed in the above embodiment is still taken as an example below, for 3 filter images a1, a2 and A3, if the pixel values of the pixel points at certain positions in the 3 images are the same and are all preset pixel values (e.g., 0), it indicates that the pixel point is the target pixel point, otherwise, the pixel point is the non-target pixel point.
The preset pixel value of 0 can represent that the probability that the pixel point is a target segmentation object is 0, therefore, when the pixel values of the pixel points at the same position in 3 images are all 0, the pixel point is represented to belong to neither sky nor grassland nor highway, and at the moment, the pixel point can be up-sampled to K according to the segmentation image with the minimum resolution rationAnd determining the pixel value of the position in the target filtering image according to the pixel value of the pixel point of the position in the obtained image. For the non-target pixel, the determination may be performed by the method listed in the previous embodiment.
For example, for the target pixel point X1, the pixel values at this position in A1, A2, A3 are all 0, but the segmented image of the minimum resolution is upsampled to KnThe pixel value at the position X1 in the obtained image is X, and at this time, the pixel value at the position X1 in the target filtering image can be determined to be X; to is directed atThe non-target pixel point X2 has the pixel values of b1, b2 and b3 in A1, A2 and A3 respectively, and at the moment, the pixel value of b3 is B3538>b2>b1, so that the pixel value at the position X2 in the target filtered image is b3, and based on the mode, the pixel value of the pixel point at each position in the target filtered image can be determined, so as to obtain the target filtered image.
In the foregoing embodiment, the target filtered image can be determined based on the pixel values of the pixels in the filtered image corresponding to each target segmented object, and a simple and efficient method for determining a target filtered image is also provided. In addition, in the mode, the pixel value of the target pixel point is determined by utilizing the segmentation image with the minimum resolution, and the accuracy is improved.
It should be noted that the image segmentation method is applicable to any task of assigning a label to a pixel, for example, foreground segmentation, that is, a task of assigning two kinds of labels, namely, foreground and background, to a pixel, that is, any image segmentation task may be accelerated by the method in the embodiment of the present disclosure. For a multi-classification image segmentation method, such as semantic segmentation of an image, similar processing may be performed on only one class in turn, that is, the segmentation methods listed in the above embodiments, and the like. The method is suitable for any segmentation algorithm, and the operations such as downsampling, edge-oriented filtering and the like are very easy to accelerate through hardware, and compared with a common image segmentation algorithm, the method has the advantages that the calculation amount is smaller, and therefore the performance can be greatly improved. In addition, the edge-oriented filtering may also be replaced by other filtering methods that maintain edges, such as bilateral filtering, least square filtering, etc., and is not limited herein.
The following takes two-class segmentation as an example to describe the complete method of image segmentation. Fig. 4 is a flowchart illustrating a complete method for image segmentation according to an exemplary embodiment, which specifically includes the following steps:
s41: the method comprises the steps that down-sampling is conducted on an original image to obtain an image pyramid, wherein the image pyramid comprises three layers in total, the image resolution of the topmost layer is the minimum, and the image resolution of the bottommost layer is the maximum;
s42: performing image segmentation operation on the top-most image to obtain a segmented image corresponding to the top-most image;
s43: performing edge-oriented filtering on the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer;
s44: up-sampling the filtering image corresponding to the topmost layer to obtain a segmentation image corresponding to the middle layer;
s45: performing edge-oriented filtering on the segmented image corresponding to the middle layer according to the image of the middle layer to obtain a filtered image corresponding to the middle layer;
s46: up-sampling the filtering image corresponding to the middle layer to obtain a segmentation image corresponding to the topmost layer;
s47: performing edge-oriented filtering on the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer;
s48: and carrying out binarization processing on the topmost filtering image to obtain a target segmentation image of the original image.
Fig. 4 illustrates a specific flow when the image segmentation method in the embodiment of the present disclosure is applied to the binary classification, and in the case of the multi-classification, it is necessary to treat the image segmentation method as a plurality of binary classifications.
In the following, a multi-class segmentation is taken as an example to describe a complete method for image segmentation, for example, target segmentation objects are an a object, a B object and a C object respectively, and image contents are divided into three classes. Fig. 5 is a flowchart illustrating a complete method for image segmentation according to an exemplary embodiment, which specifically includes the following steps:
s51: the method comprises the steps that down-sampling is conducted on an original image to obtain an image pyramid, wherein the image pyramid comprises three layers in total, the image resolution of the topmost layer is the minimum, and the image resolution of the bottommost layer is the maximum;
s52: performing image segmentation operation on the top-most image to obtain a segmented image corresponding to the top-most image;
s53: performing edge-oriented filtering on the object A in the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer;
s54: up-sampling the filtering image corresponding to the topmost layer to obtain a segmentation image corresponding to the middle layer;
s55: performing edge-oriented filtering on the object A in the segmentation image corresponding to the middle layer according to the image of the middle layer to obtain a filtering image corresponding to the middle layer;
s56: up-sampling the filtering image corresponding to the middle layer to obtain a segmentation image corresponding to the topmost layer;
s57: performing edge-oriented filtering on the object A in the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer, wherein the image is a filtered image with the maximum resolution corresponding to the object A;
s53': performing edge-oriented filtering on the B object in the segmented image corresponding to the topmost layer according to the topmost layer image to obtain a filtered image corresponding to the topmost layer;
s54': up-sampling the filtering image corresponding to the topmost layer to obtain a segmentation image corresponding to the middle layer;
s55': according to the image of the middle layer, carrying out edge-oriented filtering on the object B in the segmentation image corresponding to the middle layer to obtain a filtering image corresponding to the middle layer;
s56': up-sampling the filtering image corresponding to the middle layer to obtain a segmentation image corresponding to the topmost layer;
s57': performing edge-oriented filtering on the B object in the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer, wherein the image is a filtered image with the maximum resolution corresponding to the B object;
s53': performing edge-oriented filtering on the C object in the segmented image corresponding to the topmost layer according to the topmost layer image to obtain a filtered image corresponding to the topmost layer;
s54': up-sampling the filtering image corresponding to the topmost layer to obtain a segmentation image corresponding to the middle layer;
s55': performing edge-oriented filtering on the object C in the segmentation image corresponding to the middle layer according to the image of the middle layer to obtain a filtering image corresponding to the middle layer;
s56': up-sampling the filtering image corresponding to the middle layer to obtain a segmentation image corresponding to the topmost layer;
s57': performing edge-oriented filtering on the C object in the segmented image corresponding to the topmost layer according to the image of the topmost layer to obtain a filtered image corresponding to the topmost layer, wherein the image is a filtered image with the maximum resolution corresponding to the C object;
s58: according to the filtered images with the maximum resolution ratios corresponding to the object A, the object B and the object C, obtaining the pixel value of each pixel point in the target filtered image;
s59: and carrying out binarization processing on the target filtering image to obtain a target segmentation image of the original image.
Based on the same inventive concept, an image segmentation apparatus is also provided in the embodiments of the present disclosure, fig. 6 is a schematic diagram of an image segmentation apparatus 600 shown according to an exemplary embodiment, and referring to fig. 6, the apparatus includes a first acquisition unit 601, a processing unit 602, and a second acquisition unit 603.
A first obtaining unit 601, configured to perform obtaining of an image set to be processed, where the image set includes an original image and at least one target image, the target image is obtained by down-sampling an image of the image set with a resolution greater than that of the target image, and a resolution of the original image is the largest;
a processing unit 602, configured to sequentially acquire segmented images corresponding to respective resolutions in an order from small to large in resolution, and perform edge filtering processing on the segmented images corresponding to the respective resolutions according to the images corresponding to the respective resolutions in the image set to obtain filtered images corresponding to the respective resolutions, where the segmented image of the minimum resolution is obtained by performing image segmentation on a target image of the minimum resolution in the image set, and the segmented images of other resolutions are obtained by performing upsampling on the filtered image obtained last time;
a second obtaining unit 603 configured to obtain a target segmentation image corresponding to the original image according to the filtered image with the maximum resolution.
In an optional implementation, the processing unit 602 is specifically configured to perform:
regarding any resolution, taking the image in the image set corresponding to the resolution as a guide image;
and performing guiding filtering on the segmentation image corresponding to the resolution according to the guiding image to obtain a filtering image corresponding to the resolution.
In an optional implementation manner, after the processing unit 602 performs edge filtering processing on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution, before the second obtaining unit 603 obtains the target segmented image corresponding to the original image according to the filtered image corresponding to the maximum resolution in the image set, the processing unit 602 is further configured to perform:
comparing the images in the image set corresponding to the resolution with the filtered images corresponding to the resolution;
and carrying out fuzzy processing on the filtered image corresponding to the resolution according to the comparison result.
In an optional implementation, the processing unit 602 is specifically configured to perform:
for any resolution, sequentially carrying out edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution according to the image in the image set corresponding to the resolution to obtain a filtering image corresponding to the resolution;
when the edge filtering processing is carried out on the first target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution; when the edge filtering processing is performed on other target segmentation objects, the edge filtering processing is performed on the filtered image after the edge filtering processing is performed last time.
In an optional implementation, the processing unit 602 is specifically configured to perform:
aiming at any resolution, according to the image in the image set corresponding to the resolution, performing edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution to obtain a filtering image corresponding to each target segmentation object; when the edge filtering processing is carried out on each target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution;
and taking the filtered image corresponding to each target segmentation object as the filtered image corresponding to the resolution.
In an alternative embodiment, there is one filtered image of maximum resolution;
the second obtaining unit 603 is specifically configured to perform:
and carrying out binarization processing on the filtered image with the maximum resolution to obtain a target segmentation image corresponding to the original image.
In an alternative embodiment, the maximum resolution of the filtered image is multiple;
the second obtaining unit 603 is specifically configured to perform:
determining the pixel value of each pixel point in the target filtering image according to the position of each target segmentation object in the corresponding filtering image;
and carrying out binarization processing on the target filtering image to obtain a target segmentation image corresponding to the original image.
In an optional implementation manner, the second obtaining unit 603 is specifically configured to perform:
and aiming at any pixel point, taking the maximum value in the pixel values of the pixel points in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
In an optional implementation manner, the second obtaining unit 603 is specifically configured to perform:
for a target pixel point, the pixel value of the target pixel point position in the image obtained after the segmentation image with the minimum resolution is up-sampled to the maximum resolution is taken as the pixel value of the pixel point at the corresponding position in the target filtering image, wherein the target pixel point is the pixel point at the position where the pixel value in the filtering image corresponding to each target segmentation object is the preset pixel value; or
And regarding non-target pixel points, taking the maximum value in the pixel values of the non-target pixel point positions in the filtering image corresponding to each target segmentation object as the pixel value of the pixel point at the corresponding position in the target filtering image.
For convenience of description, the above parts are separately described as modules (or units) according to functional division. Of course, the functionality of the various modules (or units) may be implemented in the same one or more pieces of software or hardware in practicing the disclosure.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
Fig. 7 is a block diagram illustrating an electronic device 700 according to an example embodiment, the apparatus comprising:
a processor 710;
a memory 720 for storing instructions executable by the processor 710;
wherein the processor 710 is configured to execute the instructions to implement the image segmentation method in the embodiments of the present disclosure.
In an exemplary embodiment, a storage medium comprising instructions, such as the memory 720 comprising instructions, executable by the processor 710 of the electronic device 700 to perform the above-described method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In some possible implementations, a computing device according to the present disclosure may include at least one processing unit, and at least one memory unit. Wherein the storage unit stores program code which, when executed by the processing unit, causes the processing unit to perform the steps in the image segmentation methods according to various exemplary embodiments of the present disclosure described above in this specification. For example, the processing unit may perform the steps as shown in fig. 2.
The computing device 80 according to this embodiment of the disclosure is described below with reference to fig. 8. The computing device 80 of fig. 8 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 8, computing device 80 is embodied in the form of a general purpose computing device. Components of computing device 80 may include, but are not limited to: the at least one processing unit 81, the at least one memory unit 82, and a bus 83 connecting the various system components (including the memory unit 82 and the processing unit 81).
Bus 83 represents one or more of any of several types of bus structures, including a memory bus or memory control module, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
The storage unit 82 may include readable media in the form of volatile memory, such as a Random Access Memory (RAM)821 and/or a cache storage unit 822, and may further include a Read Only Memory (ROM) 823.
The storage unit 82 may also include a program/utility 825 having a set (at least one) of program modules 824, such program modules 824 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The computing device 80 may also communicate with one or more external devices 84 (e.g., keyboard, pointing device, etc.), may also communicate with one or more devices that enable a user to interact with the computing device 80, and/or may communicate with any devices (e.g., router, modem, etc.) that enable the computing device 80 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interfaces 85. Also, computing device 80 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) through network adapter 86. As shown, network adapter 86 communicates with other modules for computing device 80 over bus 83. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computing device 80, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
In some possible embodiments, various aspects of the image segmentation method provided by the present disclosure may also be implemented in the form of a program product including program code for causing a computer device to perform the steps in the image segmentation method according to various exemplary embodiments of the present disclosure described above in this specification when the program product is run on the computer device, for example, the computer device may perform the steps as shown in fig. 2.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The program product of embodiments of the present disclosure may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a computing device. However, the program product of the present disclosure is not limited thereto, and in this document, the readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with a command execution system, apparatus, or device.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a command execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on the user equipment, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.
Further, while the operations of the disclosed methods are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present disclosure have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the disclosure.
It will be apparent to those skilled in the art that various changes and modifications can be made in the present disclosure without departing from the spirit and scope of the disclosure. Thus, if such modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include such modifications and variations as well.

Claims (10)

1. An image segmentation method, comprising:
acquiring an image set to be processed, wherein the image set comprises an original image and at least one target image, the target image is obtained by down-sampling according to an image with a resolution greater than that of the target image in the image set, and the resolution of the original image is the maximum;
sequentially acquiring segmented images corresponding to the resolutions according to the sequence of the resolutions from small to large, and performing edge filtering processing on the segmented images corresponding to the resolutions according to the images corresponding to the resolutions in the image set to obtain filtered images corresponding to the resolutions, wherein the segmented image with the minimum resolution is obtained by performing image segmentation on a target image with the minimum resolution in the image set, and the segmented images with other resolutions are obtained by performing up-sampling on the filtered image obtained last time;
and acquiring a target segmentation image corresponding to the original image according to the filtering image with the maximum resolution.
2. The method according to claim 1, wherein the performing edge filtering processing on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution specifically includes:
regarding any resolution, taking the image in the image set corresponding to the resolution as a guide image;
and performing guiding filtering on the segmentation image corresponding to the resolution according to the guiding image to obtain a filtering image corresponding to the resolution.
3. The method according to claim 1, wherein after performing edge filtering processing on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution, before acquiring the target segmented image corresponding to the original image according to the filtered image corresponding to the maximum resolution in the image set, specifically comprises:
comparing the images in the image set corresponding to the resolution with the filtered images corresponding to the resolution;
and carrying out fuzzy processing on the filtered image corresponding to the resolution according to the comparison result.
4. The method of claim 1, wherein performing edge filtering on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution comprises:
for any resolution, sequentially carrying out edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution according to the image in the image set corresponding to the resolution to obtain a filtering image corresponding to the resolution;
when the edge filtering processing is carried out on the first target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution; when the edge filtering processing is performed on other target segmentation objects, the edge filtering processing is performed on the filtered image after the edge filtering processing is performed last time.
5. The method of claim 1, wherein performing edge filtering on the segmented image corresponding to each resolution according to the image corresponding to each resolution in the image set to obtain a filtered image corresponding to each resolution comprises:
aiming at any resolution, according to the image in the image set corresponding to the resolution, performing edge filtering processing on each target segmentation object in the segmentation image corresponding to the resolution to obtain a filtering image corresponding to each target segmentation object; when the edge filtering processing is carried out on each target segmentation object, the edge filtering processing is carried out on the segmentation image corresponding to the resolution;
and taking the filtered image corresponding to each target segmentation object as the filtered image corresponding to the resolution.
6. The method of any of claims 1 to 5, wherein there is one filtered image of maximum resolution;
the obtaining of the target segmentation image corresponding to the original image according to the filtered image with the maximum resolution includes:
and carrying out binarization processing on the filtered image with the maximum resolution to obtain a target segmentation image corresponding to the original image.
7. The method according to any one of claims 1 to 5, wherein the maximum resolution filtered image is plural;
the obtaining of the target segmentation image corresponding to the original image according to the filtered image with the maximum resolution includes:
determining the pixel value of each pixel point in the target filtering image according to the position of each target segmentation object in the corresponding filtering image;
and carrying out binarization processing on the target filtering image to obtain a target segmentation image corresponding to the original image.
8. An image segmentation apparatus, comprising:
the image processing device comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is configured to acquire an image set to be processed, the image set comprises an original image and at least one target image, the target image is obtained by down-sampling images with a resolution higher than that of the target image in the image set, and the resolution of the original image is the maximum;
the processing unit is configured to sequentially acquire segmented images corresponding to the resolutions according to the sequence of the resolutions from small to large, and perform edge filtering processing on the segmented images corresponding to the resolutions according to the images corresponding to the resolutions in the image set to obtain filtered images corresponding to the resolutions, wherein the segmented image with the minimum resolution is obtained by performing image segmentation on a target image with the minimum resolution in the image set, and the segmented images with other resolutions are obtained by performing up-sampling on the filtered image obtained last time;
and the second acquisition unit is configured to execute filtering image according to the maximum resolution and acquire a target segmentation image corresponding to the original image.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the image segmentation method according to any one of claims 1 to 7.
10. A storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the image segmentation method of any one of claims 1 to 7.
CN202010403656.3A 2020-05-13 2020-05-13 Image segmentation method, device, electronic equipment and storage medium Active CN111612791B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010403656.3A CN111612791B (en) 2020-05-13 2020-05-13 Image segmentation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010403656.3A CN111612791B (en) 2020-05-13 2020-05-13 Image segmentation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111612791A true CN111612791A (en) 2020-09-01
CN111612791B CN111612791B (en) 2023-11-28

Family

ID=72200199

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010403656.3A Active CN111612791B (en) 2020-05-13 2020-05-13 Image segmentation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111612791B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800094A (en) * 2012-07-13 2012-11-28 南京邮电大学 Fast color image segmentation method
US20180061091A1 (en) * 2016-08-31 2018-03-01 International Business Machines Corporation Anatomy segmentation through low-resolution multi-atlas label fusion and corrective learning
CN110084818A (en) * 2019-04-29 2019-08-02 清华大学深圳研究生院 Dynamic down-sampled images dividing method
CN110197491A (en) * 2019-05-17 2019-09-03 上海联影智能医疗科技有限公司 Image partition method, device, equipment and storage medium
CN110866878A (en) * 2019-11-13 2020-03-06 首都师范大学 Multi-scale denoising method for low-dose X-ray CT image
CN110930416A (en) * 2019-11-25 2020-03-27 宁波大学 MRI image prostate segmentation method based on U-shaped network

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102800094A (en) * 2012-07-13 2012-11-28 南京邮电大学 Fast color image segmentation method
US20180061091A1 (en) * 2016-08-31 2018-03-01 International Business Machines Corporation Anatomy segmentation through low-resolution multi-atlas label fusion and corrective learning
CN110084818A (en) * 2019-04-29 2019-08-02 清华大学深圳研究生院 Dynamic down-sampled images dividing method
CN110197491A (en) * 2019-05-17 2019-09-03 上海联影智能医疗科技有限公司 Image partition method, device, equipment and storage medium
CN110866878A (en) * 2019-11-13 2020-03-06 首都师范大学 Multi-scale denoising method for low-dose X-ray CT image
CN110930416A (en) * 2019-11-25 2020-03-27 宁波大学 MRI image prostate segmentation method based on U-shaped network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YANGYANG SONG等: "Fast two-stage segmentation based on local correntropy-based K-means clustering", 《2017 IEEE 9TH INTERNATIONAL CONFERENCE ON COMMUNICATION SOFTWARE AND NETWORKS (ICCSN)》 *
刘海华等: "基于小波变换的多分辨率图像分割", 《计算机工程与应用》 *
许利显;潘建寿;唐宏震;: "多分辨率下的彩色图像分割方法", 现代电子技术, no. 12 *

Also Published As

Publication number Publication date
CN111612791B (en) 2023-11-28

Similar Documents

Publication Publication Date Title
US10229499B2 (en) Skin lesion segmentation using deep convolution networks guided by local unsupervised learning
US10373312B2 (en) Automated skin lesion segmentation using deep side layers
AU2017302248B2 (en) Label and field identification without optical character recognition (OCR)
US20190385054A1 (en) Text field detection using neural networks
US10936914B2 (en) Convolutional neural network with augmentation features
CN111488826A (en) Text recognition method and device, electronic equipment and storage medium
AU2021354030B2 (en) Processing images using self-attention based neural networks
Singh et al. Adaptive binarization of severely degraded and non-uniformly illuminated documents
CN111914654B (en) Text layout analysis method, device, equipment and medium
US11600088B2 (en) Utilizing machine learning and image filtering techniques to detect and analyze handwritten text
JP2021135993A (en) Text recognition method, text recognition apparatus, electronic device, and storage medium
CN112949767A (en) Sample image increment, image detection model training and image detection method
CN111104941B (en) Image direction correction method and device and electronic equipment
CN115731422A (en) Training method, classification method and device of multi-label classification model
CN115240203A (en) Service data processing method, device, equipment and storage medium
KR102026280B1 (en) Method and system for scene text detection using deep learning
Dhar et al. Interval type-2 fuzzy set and human vision based multi-scale geometric analysis for text-graphics segmentation
CN111612791B (en) Image segmentation method, device, electronic equipment and storage medium
CN115620315A (en) Handwritten text detection method, device, server and storage medium
Cai et al. Bank card and ID card number recognition in Android financial APP
AU2022221413A1 (en) Domo v2: on-device object detection and instance segmentation for object selection
CN111612804B (en) Image segmentation method, device, electronic equipment and storage medium
CN114627456A (en) Bill text information detection method, device and system
Kusetogullari Unsupervised text binarization in handwritten historical documents using k-means clustering
CN117423116B (en) Training method of text detection model, text detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant