CN115512145A - Image segmentation method and device, vehicle and storage medium - Google Patents
Image segmentation method and device, vehicle and storage medium Download PDFInfo
- Publication number
- CN115512145A CN115512145A CN202211249080.5A CN202211249080A CN115512145A CN 115512145 A CN115512145 A CN 115512145A CN 202211249080 A CN202211249080 A CN 202211249080A CN 115512145 A CN115512145 A CN 115512145A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- superpixel
- fusion
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
- G06V10/7625—Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image segmentation method, an image segmentation device, a vehicle and a storage medium. The method comprises the following steps: obtaining the original a target image; pre-segmenting an original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity; clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to realize image segmentation. In order to adapt to the irregular shape of a target contour in an image and fully utilize color information when the information of the target contour is reserved, the fuzzy C-means clustering algorithm is promoted to a super-pixel level, super-pixel fusion operation based on color similarity is embedded, local color information and gradient information are introduced into the fuzzy C-means clustering algorithm to complete regularization of target color distribution, in addition, a multi-dimensional characteristic fusion method is adopted to update a membership matrix and is embedded in the clustering process, and the robustness of an image segmentation method is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image segmentation method, an image segmentation apparatus, a vehicle, and a storage medium.
Background
The intelligent internet automobile technology is widely used in the aspects of ensuring driving safety, simplifying travel, saving energy, reducing emission and the like, wherein the computer vision technology plays a vital role in perception and understanding of the surrounding environment of the driver, and the objective and accurate prompt of the computer vision technology is of great help to make a correct decision and reasonably plan for the driver. In order to comprehensively understand the surrounding environment and the target posture state, a high requirement is required for accurate extraction of the external driving environment.
In general, the image information captured by the vehicle video sensor is a three-channel color picture, which is more spatial information and gradient information than a grayscale picture. Due to the influence of uneven illumination intensity, the color distribution of the same target in a color image is irregular, and therefore, a great challenge still exists in how to completely and accurately detect the target.
Disclosure of Invention
The invention provides an image segmentation method, an image segmentation device, a vehicle and a storage medium, which are used for realizing image segmentation and accurately identifying a target object in an image.
According to an aspect of the present invention, there is provided an image segmentation method, including:
acquiring an original target image;
pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity;
clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to achieve image segmentation.
Optionally, the pre-segmenting the original target image by using an improved watershed method to obtain a target superpixel image includes:
performing multi-dimensional gradient fusion and watershed operation on the original target image to generate an initial superpixel image;
and marking the color median of the initial superpixel blocks in the initial superpixel image to obtain a target superpixel image.
Optionally, the performing multi-dimensional gradient fusion and watershed operations on the original target image to generate an initial superpixel image includes:
determining fusion gradient images of the original target image in at least two different dimensions;
carrying out multi-dimensional fusion on each fusion gradient image to obtain a multi-dimensional gradient fusion image;
and carrying out watershed operation on the multi-dimensional gradient fusion image to obtain an initial superpixel image.
Optionally, the color median marking the initial superpixel block in the initial superpixel image to obtain the target superpixel image includes:
determining, for each initial superpixel block in the initial superpixel image, a median color intensity value for the initial superpixel block;
assigning the color intensity median value to all pixel points in the initial superpixel block to obtain a target superpixel block;
and reconstructing each target superpixel block to obtain a target superpixel image.
Optionally, the clustering the target super-pixel blocks in the target super-pixel image based on the multi-dimensional feature fusion metric to implement image segmentation includes:
initializing at least two clustering centers, and determining a membership value of each target superpixel block in the target superpixel image to each clustering center, wherein the membership value is obtained by fusing the color similarity and the spatial distance of the target superpixel block to the clustering centers;
determining a value of a target function by combining a pre-constructed target function according to each membership value;
when the value of the objective function is larger than a preset clustering threshold, updating the membership value of each clustering center and each target superpixel block to each clustering center based on the objective function minimization principle until the newly determined value of the objective function is smaller than the preset clustering threshold;
and clustering each target superpixel block based on the updated membership value to realize image segmentation.
Optionally, the membership value is positively correlated with the color similarity.
Optionally, the membership value is inversely related to the spatial distance.
According to another aspect of the present invention, there is provided an image segmentation apparatus including:
the image acquisition module is used for acquiring an original target image;
the super-pixel pre-segmentation module is used for pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity;
and the multi-feature fusion clustering module is used for clustering the target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion measurement so as to realize image segmentation.
According to another aspect of the present invention, there is provided a vehicle including:
the sensor is used for acquiring an original target image;
at least one controller; and
a memory communicatively coupled to the at least one controller; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one controller to enable the at least one controller to perform the image segmentation method according to any of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the image segmentation method according to any one of the embodiments of the present invention when the computer instructions are executed.
According to the technical scheme of the embodiment of the invention, an original target image is obtained; pre-segmenting an original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity; clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to realize image segmentation. In order to adapt to the irregular shape of the target contour in the image and fully utilize color information when the target contour information is reserved, the embodiment of the invention adopts an improved watershed method to upgrade the fuzzy C-means clustering algorithm to a super-pixel level and embed super-pixel fusion operation based on color similarity, and introduces local color information and gradient information into the super-pixel fusion operation to finish the regularization of target color distribution.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of an image segmentation method according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating a principle of multi-dimensional gradient fusion in an image segmentation method according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating a principle of color median labeling in an image segmentation method according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a principle of clustering in an image segmentation method according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an image segmentation apparatus according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of a vehicle implementing the image segmentation method of the embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," "original," "initial," "target," and the like in the description and claims of the invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a flowchart of an embodiment of the present invention, which provides an image segmentation method, where the embodiment is applicable to segmenting an image to identify a target object in the image, and the method may be performed by an image segmentation apparatus, which may be implemented in a form of hardware and/or software, and the image segmentation apparatus may be configured in a vehicle controller. As shown in fig. 1, the method includes:
and S110, acquiring an original target image.
The image segmentation method can be applied to an intelligent driving scene of a vehicle, and can be used for intelligently identifying the target object after image segmentation by carrying out image segmentation on the image acquired by the vehicle sensor and assisting a driver to make a correct decision and reasonably plan.
In this embodiment, the original target image may be an environmental image around the vehicle collected by the vehicle collector.
S120, pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity.
In general, the conventional watershed method fuses similar regions of an image based on intensity variation and based on similarity between pixels. Under the condition of uneven illumination intensity, different parts of the same target in the image present complex color distribution, so that it is difficult to select a proper classifier to accurately extract the target in the color real image, and a low-precision segmentation result is obtained only by an original image segmentation clustering method. In the embodiment, after the improved watershed operation is combined with the clustering method, the clustering algorithm is promoted to a super-pixel level so as to improve the robustness of the algorithm to chromatic aberration.
Firstly, the watershed algorithm is improved, an original target image is pre-segmented to generate a superpixel block, image contour information is reserved, and then, in order to overcome an over-segmentation phenomenon generated by the traditional watershed operation, a superpixel fusion module based on color similarity is reconstructed to obtain a target superpixel image. The local color formation information and gradient information are integrated into a watershed algorithm to regularize the color distribution.
Optionally, S120 may be implemented by:
s1201, carrying out multi-dimensional gradient fusion and watershed operation on the original target image to generate an initial superpixel image.
Further, S1201 may be implemented by the following specific method: determining fusion gradient images of an original target image in at least two different dimensions; carrying out multi-dimensional fusion on each fusion gradient image to obtain a multi-dimensional gradient fusion image; and carrying out watershed operation on the multi-dimensional gradient fusion image to obtain an initial superpixel image.
S1202, color median marking is carried out on the initial superpixel blocks in the initial superpixel image, and a target superpixel image is obtained.
Further, S1202 may be implemented by the following specific method: determining a color intensity median of the initial superpixel blocks for each initial superpixel block in the initial superpixel image; assigning the color intensity median value to all pixel points in the initial superpixel block to obtain a target superpixel block; and reconstructing each target superpixel block to obtain a target superpixel image.
In this embodiment, the super-pixel merging module based on color similarity is designed into two parts: on the one hand, multi-dimensional gradient fusion is proposed to introduce gradient information; color median markers, on the other hand, are embedded into the watershed algorithm to incorporate local color information. In general, the conventional watershed algorithm relies too much on gradient changes from the original image, but the irregular color intensity will result in a weak gradient change increase, so that the region of interest cannot be extracted to the maximum extent. Therefore, in order to improve the pre-segmentation effect, the invention provides multi-dimensional gradient fusion to improve the watershed algorithm. In general, the conventional gradient expression can be expressed as:
where f may represent the original target image, G x 、G y Can represent the horizontal and vertical gradient obtained by Sobel operator, and G (f) can represent G x 、G y And fusing the gradient images. In the multi-dimensional gradient fusion module, gradients of other dimensions can be fused on the basis of G (f) to obtain more gradient information. For example, gradient information of two other dimensions is fused on the basis of G (f), and the formula can be expressed as follows:
FIG. 2 is a schematic diagram illustrating a principle of multi-dimensional gradient fusion in an image segmentation method according to an embodiment of the present invention, wherein G (f) 1 ) The graph of fig. 2 (a) may be represented to maintain sharp gradient changes and weaken slight gradient changes; g (f) 2 ) The (b) diagram in fig. 2 may be represented, which may be a fused gradient image at a standard scale; g (f) 3 ) The (c) diagram in fig. 2 can be represented, which is introduced to also intensify the sharp gradient change; f G A multi-dimensional gradient fusion image obtained by multi-dimensionally fusing the respective fusion gradient images, such as the (d) diagram in fig. 2, can be represented.
This embodiment may utilize F G Performing a watershed operation on the basis of which an initial superpixel image is obtained:
F w (ξ 1 ,ξ 2 ,…ξ n )＝watershed(F G )，
C(ξ i )＝median[C(a 1 ),C(a 2 ),…C(a m )]，
wherein, a j ∈ξ i ，j∈[1,m]，F W Can represent an initial superpixel image, { ξ, { obtained over a watershed operation 1 ,ξ 2 ,…ξ n Can represent different initial superpixel blocks, C (ξ) i ) Can represent the color intensity of the respective super-pixel block, { a 1 ,a 2 ,…a m Can represent the block xi i At all the pixels in the tree, the mean operation can represent that the pixel is { a } 1 ,a 2 ,…a m The color intensities of the } are ordered and median. Because the chromatic aberration can be characterized as an ultra-pixel block with an extremely small size, after an extremely small area is restrained by multi-dimensional gradient fusion, the color median of each ultra-pixel block is introduced, the influence of uneven color intensity distribution can be reduced by combining local color information with a clustering method, and partial data is regularized. Fig. 3 is a schematic diagram of a principle of a color median marker in an image segmentation method according to an embodiment of the present invention, and as shown in fig. 3, an operation flow of the color median marker in this embodiment may be as follows:
the ChannelR, channelG, and ChannelB may respectively represent pixel values of three color channels before the color median mark, and in fig. 3, one of the red channel ChannelR has an initial superpixel block with uneven color distribution, where the pixel color intensity values are sequenced [0,221,221,225,228,228,230,234,237], and it is easy to know that the color median is 228. Subsequently, the color intensity median is assigned to all the pixel points in the initial super-pixel block, as shown in the pixel table (c) in fig. 4, that is, the super-pixel block into which the local color information is introduced, and it can be seen that the intensity distribution in the pixel block has been regularized. After the color median mark is applied to the whole super-pixel map, the complex color distribution is suppressed, as shown in the graph (e) in fig. 3, the uneven color distribution is eliminated, and the target contour information is completely stored. The color median marking method provided by the embodiment can be used for dealing with irregular color distribution in a color image, and the sensitivity of a clustering algorithm to intensity change is reduced.
S130, clustering target superpixel blocks in the target superpixel image based on the multi-dimensional feature fusion metric so as to realize image segmentation.
In classical fuzzy set theory, membership is defined by the membership of an element to each cluster center, and the membership function value of each element is bounded between [0,1 ]. Mathematically, the fuzzy set F is defined as:
F＝{(x,μ F (x))|x∈E}，
wherein mu F (x) I.e. the membership value of element x to the corpus E, for each element x, mu F (x) Are all in [0,1]In the meantime.
The traditional fuzzy C-means clustering algorithm (FCM) divides the pixels in the image into C fuzzy clusters, and in each iteration, both the cluster center and membership value are updated, and thus the objective function value is minimized. The feature vectors are classified by minimizing an objective function, which expression can be expressed as:
calculating J by updating membership degree matrix U and clustering center V f Where x is the minimum value of k Is the pixel k, c in the image i Representing the cluster center i by evaluating the pixel x k And cluster center c i Membership relationship between to calculate membership function value u ik . In addition, the first and second substrates are,
the following formula gives u ik And c i In which dis (x) k ,c i ) Can be expressed as a pixel x k And cluster center c i Euclidean distance between:
although the FCM is widely applied in the field of image segmentation, as shown in the above formula, it ignores local information and spatial information in the iterative process, and because the real color image is affected by factors such as illumination intensity, the color distribution and intensity in the same target are not uniform, which results in that the FCM cannot completely adapt to the complex color distribution of the color real image, and the clustering effect is not ideal.
To improve the FCM ability to segment true color images, the present embodiment embeds a multi-feature adaptive fusion metric into the membership update process, which is designed as a combination of color similarity and spatial distance. Specifically, the metric fuses the similarity between the superpixel blocks and the cluster centers on top of a simple euclidean metric, and the multi-feature fusion metric improves the robustness of the algorithm to uneven color distribution.
Optionally, S130 may be implemented by the following steps:
s1301, initializing at least two clustering centers, and determining the membership value of each target superpixel block in the target superpixel image to each clustering center, wherein the membership value is obtained by fusing the color similarity and the spatial distance of the target superpixel block to the clustering centers.
Further, the membership value and the color similarity are positively correlated; the membership value is inversely related to the spatial distance.
And S1302, determining a value of the objective function according to each membership value and by combining a pre-constructed objective function.
And S1303, when the value of the target function is greater than the preset clustering threshold, updating the membership value of each clustering center and each target superpixel block to each clustering center based on the target function minimization principle until the newly determined value of the target function is less than the preset clustering threshold.
And S1304, clustering the target super-pixel blocks based on the updated membership value, and realizing image segmentation.
In general, the traditional FCM updates the membership only according to the euclidean distance between the pixel and the cluster center, which results in poor adaptability to the complex structure of the real color image and incomplete target extraction. Although the watershed algorithm is improved to optimize the color distribution in the embodiment, the color composition between super-pixel blocks is still complex, and the image segmentation only by using a simple euclidean distance still causes the generation of a misclassification phenomenon. In order to improve the partitioning capability of the FCM, the invention provides a multi-feature self-adaptive fusion module for calculating a membership matrix and further utilizing spatial information.
In this embodiment, the color similarity and the spatial distance distribution are simultaneously considered in the membership evaluation process. In addition, in order to automatically harmonize the proportion of each feature, the embodiment also designs an adaptive fusion rule of multiple features. The metric method of this embodiment ensures that only superpixel blocks with high intensity color similarity and distance similarity have higher membership values. The optimization of the multidimensional characteristics not only improves the precision of the segmentation result, but also improves the robustness of the clustering algorithm to the irregular color intensity.
Fig. 4 is a schematic diagram illustrating a principle of clustering in an image segmentation method according to an embodiment of the present invention, and as shown in fig. 4, since a harmonic mean can automatically normalize all similarities without prior knowledge and focuses on a low membership value, the embodiment may use a multi-feature adaptive fusion method based on a harmonic mean to fuse color similarities and spatial distances. Specifically, the membership degree measurement method may be defined as:
the measurement method provided by the embodiment comprises two parts: first part D ki May be a distance measure, second part S ki Can represent local color information, and the two specific design schemes can be as follows:
D ki ＝exp(-dis 2 (ξ k ,v i ))，
S ki ＝exp(-simu 2 ||ξ k ,v i ||)，
wherein D is ki Can be space distance information based on Euclidean distance similarity, and Simu operation index quantity target superpixel block xi k And a clustering center v i The color similarity between them. The specific calculation method of the simul operation can be as follows:
wherein R is k And R i Can respectively represent target superpixel blocks xi k And a clustering center v i R channel value, Δ R, Δ G, and Δ B may represent RGB three-channel color variation amounts, respectively. A smaller amount of color change will result in a larger simul value.
Cause simulu>0 and dis>0, easy to know D ki ∈(0,1]、G ki ∈(0,1]. As the distance between the target superpixel and the cluster center decreases, D ki Will increase, which means that superpixel blocks that are close to the cluster center distance have a larger weight value. Similarly, if a target superpixel block is not close to the cluster center, the color similarity is high, M ki The value will also be larger and therefore the target superpixel point will be considered as a member of this class.
From the perspective of an objective function, the embedding of the multi-feature adaptive fusion metric can be expressed as the replacement of a distance variable, and therefore, the method proposed by the embodiment has stronger adaptability to the complex structure and the fuzzy edge of a real color image.
The embodiment can perform clustering on the target superpixel graph, and directly input the centroid of each superpixel into the clustering process. The objective function of the clustering method of this embodiment can be expressed as:
this embodiment may utilize the partial differential variable u by the Lagrange multiplier method ik And v i The partial differential equation is solved to minimize the objective function. By calculation, the membership degree update formula of the present embodiment can be expressed as:
the cluster center update formula can be expressed as:
by u in the above formula ik I.e. the objective function J can be minimized.
According to the technical scheme of the embodiment of the invention, an original target image is obtained; pre-segmenting an original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity; clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to realize image segmentation. In order to adapt to the irregular shape of the target contour in the image and fully utilize color information when the target contour information is reserved, the embodiment of the invention adopts an improved watershed method to upgrade a fuzzy C-means clustering algorithm to a superpixel level, embeds superpixel fusion operation based on color similarity, introduces local color information and gradient information into the super-pixel fusion operation, completes the regularization of target color distribution, and in addition, adopts a multi-dimensional characteristic fusion method to update a membership matrix and embeds the membership matrix in the clustering process, thereby improving the robustness of the fuzzy C-means clustering algorithm to uneven color change and optimizing the image segmentation effect.
Example two
Fig. 5 is a schematic structural diagram of an image segmentation apparatus according to a second embodiment of the present invention. As shown in fig. 5, the apparatus includes:
an image acquisition module 510 for acquiring an original target image;
a super-pixel pre-segmentation module 520, configured to pre-segment the original target image by using an improved watershed method to obtain a target super-pixel image, where the improved watershed method regularizes color distribution of the multi-dimensional gradient fusion image based on color similarity;
a multi-feature fusion clustering module 530, configured to cluster the target superpixel blocks in the target superpixel image based on a multi-dimensional feature fusion metric, so as to implement image segmentation.
Optionally, the super-pixel pre-segmentation module 520 includes:
the initial superpixel generating unit is used for carrying out multi-dimensional gradient fusion and watershed operation on the original target image to generate an initial superpixel image;
and the target super-pixel generating unit is used for marking the color median of the initial super-pixel blocks in the initial super-pixel image to obtain a target super-pixel image.
Optionally, the initial super-pixel generating unit is specifically configured to:
determining fusion gradient images of the original target image in at least two different dimensions;
carrying out multi-dimensional fusion on each fusion gradient image to obtain a multi-dimensional gradient fusion image;
and carrying out watershed operation on the multi-dimensional gradient fusion image to obtain an initial superpixel image.
Optionally, the target super-pixel generating unit is specifically configured to:
determining, for each initial superpixel block in the initial superpixel image, a median color intensity value for the initial superpixel block;
assigning the color intensity median value to all pixel points in the initial superpixel block to obtain a target superpixel block;
and reconstructing each target superpixel block to obtain a target superpixel image.
Optionally, the multi-feature fusion clustering module 530 is specifically configured to:
initializing at least two clustering centers, and determining a membership value of each target superpixel block in the target superpixel image to each clustering center, wherein the membership value is obtained by fusing the color similarity and the spatial distance of the target superpixel block to the clustering centers;
determining a value of a target function by combining a pre-constructed target function according to each membership value;
when the value of the objective function is larger than a preset clustering threshold, updating the membership value of each clustering center and each target superpixel block to each clustering center based on the objective function minimization principle until the newly determined value of the objective function is smaller than the preset clustering threshold;
and clustering each target superpixel block based on the updated membership value to realize image segmentation.
Optionally, the membership value is positively correlated with the color similarity.
Optionally, the membership value is inversely related to the spatial distance.
The image segmentation device provided by the embodiment of the invention can execute the image segmentation method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE III
Fig. 6 is a block diagram of a vehicle according to a third embodiment of the present invention, as shown in fig. 6, the vehicle includes a controller 610, a memory 620, a sensor 630, an input device 640, and an output device 650; the number of controllers 610 may be one or more, and one controller 610 is illustrated in fig. 6; the number of sensors 630 in the vehicle may be one or more, with one sensor 630 being exemplified in fig. 6; the controller 610, memory 620, sensors 630, input device 640, and output device 650 in the vehicle may be connected by a bus or other means, as exemplified by the bus connection in fig. 6.
The memory 620, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the image segmentation method in the embodiments of the present invention (e.g., the image acquisition module 510, the super-pixel pre-segmentation module 520, and the multi-feature fusion clustering module 530 in the image segmentation apparatus). The controller 610 executes various functional applications and data processing of the vehicle, i.e., implements the image segmentation method described above, by executing software programs, instructions, and modules stored in the memory 620.
The memory 620 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 620 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 620 may further include memory located remotely from the controller 610, which may be connected to the vehicle over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 640 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the vehicle. The output device 650 may include a display device such as a display screen.
Example four
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, where the computer-executable instructions are executed by a computer controller to perform an image segmentation method, and the method includes:
acquiring an original target image;
pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity;
clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to achieve image segmentation.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the image segmentation method provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An image segmentation method, comprising:
acquiring an original target image;
pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity;
clustering target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion metrics to achieve image segmentation.
2. The method of claim 1, wherein said pre-segmenting said original target image using a modified watershed method to obtain a target superpixel image comprises:
performing multi-dimensional gradient fusion and watershed operation on the original target image to generate an initial superpixel image;
and marking the color median of the initial superpixel blocks in the initial superpixel image to obtain a target superpixel image.
3. The method of claim 2, wherein performing a multi-dimensional gradient fusion and watershed operation on the original target image to generate an initial superpixel image comprises:
determining fusion gradient images of the original target image in at least two different dimensions;
carrying out multi-dimensional fusion on each fusion gradient image to obtain a multi-dimensional gradient fusion image;
and carrying out watershed operation on the multi-dimensional gradient fusion image to obtain an initial superpixel image.
4. The method of claim 2, wherein said color median labeling initial superpixel blocks in said initial superpixel image to obtain a target superpixel image comprises:
determining, for each initial superpixel block in the initial superpixel image, a median color intensity value for the initial superpixel block;
assigning the color intensity median value to all pixel points in the initial superpixel block to obtain a target superpixel block;
and reconstructing each target superpixel block to obtain a target superpixel image.
5. The method of claim 1, wherein clustering target superpixel blocks in the target superpixel image based on a multi-dimensional feature fusion metric to achieve image segmentation comprises:
initializing at least two clustering centers, and determining a membership value of each target superpixel block in the target superpixel image to each clustering center, wherein the membership value is obtained by fusing the color similarity and the spatial distance of the target superpixel block to the clustering centers;
according to each membership value, determining a value of a target function by combining a pre-constructed target function;
when the value of the objective function is larger than a preset clustering threshold, updating the membership value of each clustering center and each target superpixel block to each clustering center based on the objective function minimization principle until the newly determined value of the objective function is smaller than the preset clustering threshold;
and clustering each target superpixel block based on the updated membership value to realize image segmentation.
6. The method of claim 5,
the membership value is positively correlated with the color similarity.
7. The method of claim 5,
the membership value is inversely related to the spatial distance.
8. An image segmentation apparatus, comprising:
the image acquisition module is used for acquiring an original target image;
the super-pixel pre-segmentation module is used for pre-segmenting the original target image by adopting an improved watershed method to obtain a target super-pixel image, wherein the improved watershed method regularizes the color distribution of the multi-dimensional gradient fusion image based on color similarity;
and the multi-feature fusion clustering module is used for clustering the target superpixel blocks in the target superpixel image based on multi-dimensional feature fusion measurement so as to realize image segmentation.
9. A vehicle, characterized in that the vehicle comprises:
the sensor is used for acquiring an original target image;
at least one controller; and
a memory communicatively coupled to the at least one controller; wherein, the first and the second end of the pipe are connected with each other,
the memory stores a computer program executable by the at least one controller to enable the at least one controller to perform the image segmentation method of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a processor to perform the image segmentation method of any one of claims 1-7 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211249080.5A CN115512145A (en) | 2022-10-12 | 2022-10-12 | Image segmentation method and device, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211249080.5A CN115512145A (en) | 2022-10-12 | 2022-10-12 | Image segmentation method and device, vehicle and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115512145A true CN115512145A (en) | 2022-12-23 |
Family
ID=84509305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211249080.5A Pending CN115512145A (en) | 2022-10-12 | 2022-10-12 | Image segmentation method and device, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115512145A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984145A (en) * | 2023-02-27 | 2023-04-18 | 湖南城市学院 | Accurate fishway fish passing identification method |
| CN117974273A (en) * | 2024-03-28 | 2024-05-03 | 环球数科集团有限公司 | E-commerce product display system based on augmented reality |
-
2022
- 2022-10-12 CN CN202211249080.5A patent/CN115512145A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115984145A (en) * | 2023-02-27 | 2023-04-18 | 湖南城市学院 | Accurate fishway fish passing identification method |
| CN115984145B (en) * | 2023-02-27 | 2024-02-02 | 湖南城市学院 | Precise fish-passing identification method for fishway |
| CN117974273A (en) * | 2024-03-28 | 2024-05-03 | 环球数科集团有限公司 | E-commerce product display system based on augmented reality |
| CN117974273B (en) * | 2024-03-28 | 2024-05-28 | 环球数科集团有限公司 | E-commerce product display system based on augmented reality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ibrahim et al. | Image segmentation methods based on superpixel techniques: A survey | |
| CN115512145A (en) | Image segmentation method and device, vehicle and storage medium | |
| US9483835B2 (en) | Depth value restoration method and system | |
| CN109522908A (en) | Image significance detection method based on area label fusion | |
| CN110796026A (en) | Pedestrian re-identification method based on global feature stitching | |
| CN109977997B (en) | Image target detection and segmentation method based on convolutional neural network rapid robustness | |
| CN109961506A (en) | A kind of fusion improves the local scene three-dimensional reconstruction method of Census figure | |
| CN105469111B (en) | The object classification method of small sample set based on improved MFA and transfer learning | |
| CN110992381A (en) | Moving target background segmentation method based on improved Vibe + algorithm | |
| CN110443822A (en) | A kind of high score remote sensing target fine extracting method of semanteme edge auxiliary | |
| CN104700071B (en) | A kind of extracting method of panorama sketch road profile | |
| US9619729B2 (en) | Density measuring device, density measuring method, and computer program product | |
| CN105809673B (en) | Video foreground dividing method based on SURF algorithm and the maximum similar area of merging | |
| WO2014004271A2 (en) | Method and system for use of intrinsic images in an automotive driver-vehicle-assistance device | |
| Xu et al. | Weakly supervised deep semantic segmentation using CNN and ELM with semantic candidate regions | |
| CN108377374A (en) | Method and system for generating depth information related to an image | |
| CN107516061B (en) | Image classification method and system | |
| CN115641583B (en) | Point cloud detection method, system and medium based on self-supervision and active learning | |
| CN111274926A (en) | Image data screening method and device, computer equipment and storage medium | |
| CN111028261A (en) | High-precision semi-automatic image data annotation method, electronic device and storage medium | |
| Wang et al. | Combining semantic scene priors and haze removal for single image depth estimation | |
| CN111429512A (en) | Image processing method and device, storage medium and processor | |
| CN111274964A (en) | Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle | |
| CN113705579A (en) | Automatic image annotation method driven by visual saliency | |
| CN103871089A (en) | Image superpixel meshing method based on fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |