CN108596893B - An image processing method and system - Google Patents

Abstract

The invention discloses an image processing method and system. The method is suitable for an image processing device. The device includes an image acquisition module, an image preprocessing module, an image processing module, and an image segmentation module. The method includes the following steps: S1 image acquisition, image acquisition through the image acquisition module; S2 image preprocessing, de-drying, contrast enhancement and image morphological fabrication of the collected images, completing the pre-processing of image data, and providing high-quality image data for subsequent main processing processes ; S3 image processing, including image saliency target detection processing, and smoothing and morphological operation processing; S4 image segmentation, through adaptive threshold segmentation, the final image in S3 is subjected to target segmentation to obtain the region of interest. The method and system of the present invention can simultaneously perform visual display of detection and segmentation, provide users with real-time detection and segmentation effects, improve user experience, and improve operation efficiency.

Description

An image processing method and system

technical field

The present invention relates to an image processing method and system, in particular to a method and system for rapid detection of salient objects in static images and video sequences.

Background technique

The image processing device and method are widely used in production, management, military, medical and many other fields. Image processing technology, especially the detection and segmentation technology in image processing technology, in the application of target detection and segmentation, allows us to obtain useful information more conveniently in many aspects, so as to make more accurate judgments on the situation. The wide application of image processing technology can make our life more convenient and save more cost and resources.

The target detection and segmentation algorithm proposed by the present invention can quickly detect and segment the salient target of the collected input image, and quickly obtain the target area that people are interested in. In many scenarios, such as intelligent surveillance videos, the observer needs to quickly extract the target person or the area of the main marker, and this algorithm can be used to quickly extract the relevant area. For another example, in the acquisition of effective information from big data, how to quickly obtain the most effective information is a problem that many data experts have been considering, and the related model based on the visual attention mechanism can quickly follow the principle of visual saliency. The significant information is retrieved in the database to realize the rapid extraction of effective information. In fact, it is not only of great application value in these two aspects, but also of great significance in such aspects as pathological detection, data compression, and automatic focusing of photography.

The existing similar technologies have the following disadvantages:

First, if the image texture is complex and the contrast is relatively low, the existing detection algorithms based on the definition of image contrast cannot give good target area information, so that we cannot obtain accurate target information.

Second, the existing recognized model with the best detection effect (MBD) is difficult to give a better detection result based on the design idea of the algorithm itself when the target to be detected is in contact with the boundary of the image.

Third, the popular deep learning models rarely have cross-scenario applicability. When the scene changes, new models need to be trained according to different data sets due to different targets, and deep learning-based models require a lot of data. and time resources.

The above shortcomings are the points that the saliency detection model in the existing technology is difficult to take into account, and it is also a problem that everyone is actively exploring and solving. If these problems cannot be solved to the greatest extent, it will be difficult to provide researchers or consumers with accurate and effective target information.

Among the published papers and realized applications, some directly use the image color contrast to define the saliency of pixels or regions based on the color between pixels or regions (distinguishability of human vision), and Mingming is more representative. Cheng's HC and RC algorithms, but the problem of low contrast proposed above is still difficult to solve, and the effect is not obvious in detection and segmentation. There are also many researchers who consider the overall topological information of the image and use the correlation between pixels and regions to define saliency, which greatly improves the effect. For example, Jianming Zhang's detection algorithm based on scan traversal of minimum obstacle distance , and obtained good results on major data sets, but the existing problems are still low contrast and the detection target is in contact with the boundary, especially when the detection target is in contact with the boundary, the performance of the algorithm is greatly reduced. Other researchers who have studied interactive segmentation algorithms proposed that manual tags (foreground and background information) can be added to images in the manner of Trimap and Strokes, and then pixel-based target detection and segmentation are performed according to the information of interest provided by users. It's OK, but requires human participation markers. For the recently popular deep learning solution to this problem, it is based on convolutional neural network training. The classic is Huchuan Lu's "SaliencyDetection with Recurrent Fully Convolutional Networks." (European Conferenceon Computer Vision. Springer, Cham, 2016: 825-841.) The transformation of the fully connected layer at the back end of the classic CNN model; and Mingming Cheng's "Deeply Supervised Salient Object Detection with Short Connections" (IEEE TPAMI, 2018). However, the detection behind the deep learning network will result in insufficient scene migration, strong specificity for specific targets, and large demand for time and data resources, making it difficult to meet different application scenarios.

SUMMARY OF THE INVENTION

In view of the above-mentioned defects and deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a general target detection algorithm, which can detect salient targets on the image data collected in various scenes, combined with the method of image processing. Processing and segmentation operations, and finally get the precise target area.

The technical scheme of the present invention is realized as follows:

An image processing method, the method is applicable to an image processing device, the device includes an image acquisition module, an image preprocessing module, an image processing module, and an image segmentation module; the method includes the following steps:

S1, image acquisition, image acquisition is performed through an image acquisition module;

S2, image preprocessing, perform denoising, contrast enhancement and image morphological operations on the collected images, complete the pre-processing of image data, and provide high-quality image data for subsequent main processing processes;

S3, image processing, including image saliency target detection processing, and smoothing and morphological operation processing;

S4, image segmentation, through adaptive threshold segmentation, target segmentation is performed on the final image in S3 to obtain a region of interest.

Preferably, the image saliency target detection in the above step S3 mainly includes the selection of seed points, the determination of the diffusion order, and the image fusion of different channels;

Further, the step of image saliency target detection is:

A1, perform BMBD algorithm processing;

A2, based on the processing result of A1, and then enter the FMBD algorithm processing;

A3, fuse the two saliency detection maps obtained in A1 and s2 to obtain the BF_MBDS map of saliency detection.

Preferably, the step of the BMBD algorithm processing is,

B1, preprocess the input image, including denoising and color space conversion;

B2, perform channel separation on the processed image;

B3, initial seed point setting, set the image surrounding as seed points, and perform diffusion processing based on the seed points;

B4, adopt the method of four neighborhoods to spread from the surrounding to the center, and perform a circle-by-circle diffusion update;

B5, in the next round of diffusion, a new update is also performed for the pixels updated in the previous round of diffusion, until all the pixels are updated and traversed;

B6, respectively obtain the diffusion images BMBD_L, BMBD_a and BMBD_b of the three channels;

B7, fuse the images BMBD_L, BMBD_a, and BMBD_b obtained in S6 by taking the maximum value to obtain the BMBD_Lab map;

B8, further process the BMBD_Lab image obtained in B7, mainly including smoothing and contrast enhancement, to obtain the final BMBD_map detection image.

Preferably, the step of the FMBD algorithm processing is,

C1, preprocess the input image, including denoising and color space conversion;

C2, perform channel separation on the processed image;

C3, initial seed point setting, based on the above-mentioned BMBD algorithm detection map, select the pixels with the highest confidence, that is, the highest saliency value, and use these pixels as the initial seed points for diffusion processing;

C4, take the method of four neighborhoods to spread from the surrounding to the center, and perform a circle-by-circle diffusion update;

C5, for the pixels updated in the previous round of diffusion, in the next round of diffusion, a new update is also performed until all the pixels are updated and traversed;

C6, respectively obtain three channel diffusion images FMBD_L, FMBD_a, FMBD_b;

C7, fuse the images FMBD_L, FMBD_a, and FMBD_b obtained in S6 by taking the maximum value to obtain the FMBD_Lab map;

C8, further process the FMBD_Lab map obtained in C7, mainly including smoothing and contrast enhancement, to obtain the final FMBD_map detection image.

An image processing system, comprising an image acquisition device, an image transmission module, an image processing device, an image display module and an image storage module:

The image acquisition device performs data acquisition in a scenario where the user needs to perform detection and segmentation, and can set single-frame image acquisition of data and video acquisition of different video sequences according to user requirements;

The image transmission module is used for transmitting the above-mentioned collected data;

The image processing device uses the image processing method of claim 1 to process the image transmitted from the image transmission module;

The image storage module is used to store the image data transmitted by the image transmission module and the intermediate data in the processing process of the image processing device;

The image display module displays the processed result in real time for the user to observe.

Preferably, the processing process of the image processing device mainly includes the following steps:

D1, preprocess the transmission data to provide high-quality data for the subsequent detection algorithm;

D2, according to the detection algorithm described in claim 1, the image transmitted by the transmission module is processed to obtain the detection image;

D3, perform segmentation processing on the detection image obtained in S2 to obtain a final segmentation map.

Preferably, the image transmission module adopts wired and wireless data transmission modes to meet the needs of different users under different application conditions.

The beneficial effects of the present invention are:

1. The image processing method and system of the present invention can simultaneously perform visual display of detection and segmentation, and can provide users with real-time detection and segmentation effects, improve user experience, and improve operational efficiency.

2. The image processing method and system of the present invention can be applied to different application scenarios, and can perform target detection and segmentation processing in multiple scenarios.

3. The system adopts the long-distance wireless transmission mode, which greatly increases the freedom of space.

Description of drawings

Figure 1 is a flow chart of the BMBD algorithm.

Accompanying drawing 2 is the flow chart of FMBD algorithm.

Figure 3 is an example of the diffusion map of the BMBD algorithm.

Figure 4 is an example of a diffusion map of the FMBD algorithm.

Figure 5 is a block diagram of the overall flow of BF_MBD.

Figure 6 is a system block diagram of the present invention.

Detailed ways

The present invention is described in further detail below in conjunction with the accompanying drawings and specific embodiments:

As shown in Figures 1 and 2, an image processing method is applicable to an image processing device, the device includes an image acquisition module, an image preprocessing module, an image processing module, and an image segmentation module, and the method includes the following step:

S1, image acquisition, image acquisition is performed through an image acquisition module;

S2, image preprocessing, perform denoising, contrast enhancement and image morphological operations on the collected images, complete the pre-processing of image data, and provide high-quality image data for subsequent main processing processes;

S3, image processing, including image saliency target detection processing, and smoothing and morphological operation processing;

S4, image segmentation, through adaptive threshold segmentation, target segmentation is performed on the final image in S3 to obtain a region of interest.

As shown in FIG. 5 , further, the image saliency target detection described in S3 mainly includes the selection of seed points, the determination of the diffusion order, and the image fusion of different channels. Wherein, the step of image saliency target detection is:

A1, perform BMBD algorithm processing;

A2, based on the processing result of A1, and then perform FMBD algorithm processing;

A3, fuse the two saliency detection maps obtained in A1 and A2 to obtain a BF_MBDS map of saliency detection.

As shown in Figure 1, the steps of the BMBD algorithm processing are:

B1, preprocess the input image, including denoising and color space conversion;

B2, perform channel separation on the processed image;

B3, initial seed point setting, set the image surrounding as seed points, and perform diffusion processing based on the seed points;

B4, adopt the method of four neighborhoods to spread from the surrounding to the center, and perform a circle-by-circle diffusion update;

B5, in the next round of diffusion, a new update is also performed for the pixels updated in the previous round of diffusion, until all the pixels are updated and traversed;

B6, respectively obtain the diffusion images BMBD_L, BMBD_a and BMBD_b of the three channels;

B7, fuse the images BMBD_L, BMBD_a, and BMBD_b obtained in B6 by taking the maximum value to obtain the BMBD_Lab map;

B8, further process the BMBD_Lab image obtained in B7, mainly including smoothing and contrast enhancement, to obtain the final BMBD_map detection image.

As shown in Figure 2, the steps of the FMBD algorithm processing are:

C1, preprocess the input image, including denoising and color space conversion;

C2, perform channel separation on the processed image;

C3, initial seed point setting, based on the above-mentioned BMBD algorithm detection map, select the pixels with the highest confidence, that is, the highest saliency value, and use these pixels as the initial seed points for diffusion processing;

C4, take the method of four neighborhoods to spread from the surrounding to the center, and perform a circle-by-circle diffusion update;

C5, for the pixels updated in the previous round of diffusion, in the next round of diffusion, a new update is also performed until all the pixels are updated and traversed;

C6, respectively obtain three channel diffusion images FMBD_L, FMBD_a, FMBD_b;

C7, fuse the images FMBD_L, FMBD_a, and FMBD_b obtained in C6 by taking the maximum value to obtain the FMBD_Lab map;

C8, further process the FMBD_Lab map obtained in C7, mainly including smoothing and contrast enhancement, to obtain the final FMBD_map detection image.

As shown in FIG. 6, an image processing system includes an image acquisition device 601, an image transmission module 602, an image processing module 603, an image storage module 604 and an image display module 605, wherein,

The image acquisition device 601 performs data acquisition in a scenario where the user needs to perform detection and segmentation, and can set single-frame image acquisition of data and video acquisition of different video sequences according to user requirements;

The image transmission module 602 is used to transmit the above collected data

The image processing module 603 uses the image processing method of claim 1 to process the image transmitted from the image transmission module;

The image storage module 604 is used to store the image data transmitted by the image transmission module and the intermediate data in the process of the image processing device;

The image display module 605 displays the processing result in real time for the user to observe.

Further, the processing process of the image processing module mainly includes the following steps:

D1, preprocess the transmission data to provide high-quality data for the subsequent detection algorithm;

D2, according to the detection algorithm described in claim 1, the image transmitted by the transmission module is processed to obtain the detection image;

D3, perform segmentation processing on the detection image obtained in D2 to obtain a final segmentation map.

Further, the image transmission module adopts wired and wireless data transmission modes to meet the needs of different users under different application conditions.

As shown in Figure 3, the design idea of the BMBD algorithm in the embodiment is as follows:

The full name of BMBD is Background Minimum Barrier Distance, which is a detection method based on the minimum obstacle distance of background information. This method takes into account the influence of background information on detection, so in the selection of seed points, the pixel points around the image are set as seed points; then based on the surrounding seed points, the inner circle is updated, and the specific update method adopts four neighborhoods. update method.

The point marked 0 in the figure is the pixel point in the background we selected as the seed point. Next, the idea of visual diffusion in visual attention is used to carry out the diffusion operation of the four neighborhoods, and the point marked 1 for the first diffusion is obtained. This point has been diffused once, and its initial BMBD value is updated for the first time. The next update follows in the same way, but unlike the first one, when the area marked 2 is updated for the second diffusion, the area marked 1 is also updated again at the same time. This idea of diffusion is in line with the idea of repeated diffusion of water in reality, just like in reality, the water on the surrounding mountains converges to the central basin. The idea of diffusion in our algorithm. This idea of diffusion is proposed by us and is an important innovation in our visual saliency detection algorithm.

As shown in Figure 4, the design idea of the FMBD algorithm in the embodiment is as follows:

The full name of FMBD is Foreground Minimum Barrier Distance, which is a detection method based on the minimum obstacle distance of foreground (target) information. This method takes into account the influence of foreground information on detection, so in the selection of seed points, the pixel points of some foreground objects in the image are set as seed points; The specific method adopts the four-neighbor update method.

The point marked 0 in the figure is the initialization seed point. The idea of spreading from the center to the surrounding area is based on the idea of spring overflow, which can well solve the detection problem of the contact between the detection target and the image boundary. The selection of the center seed point is a more critical issue. The saliency detection target provided by the BMBD algorithm based on the background information diffusion algorithm is used. The partial area here is obtained by morphological operations in image processing. A small part of pixels is the central area of high saliency value in saliency detection (indirectly indicating that this part is the foreground area with the greatest probability). This area can be used as foreground information to set seed points and perform diffusion operations to the surrounding area. The diffusion will not stop until the diffusion is complete and sufficient, and the final saliency diffusion detection map will be obtained.

As shown in Figure 5, the fusion of the BMBD algorithm and the FMBD algorithm to generate the final BF_MBDS image design idea and specific implementation are as follows:

The fusion algorithm mainly used in this part is based on the idea of maximum value. The multi-channel saliency value is used for sum operation on multiple channels of a single algorithm, which is defined as the saliency value of this point, which are BMBD_value, FMBD_value, and the next is fusion. For the saliency detection maps of different algorithms, we adopt a simple maximum value idea, that is, BF_MBDS_value=BMBD_value+FMBD_value. So far, we have obtained a visual saliency detection map of an image, which is a grayscale image of the detection. The target area has a high saliency value and can be clearly distinguished. Next, an appropriate segmentation algorithm can be used for segmentation to obtain a standard two. value image.

The specific implementation (pseudo code) of the algorithm is:

For each pixel in each lap update, it is defined as follows:

And the definition of β _I (P _y (x)) is as follows:

β _I (P _y (x))=max{U(y),I(x)}-min{L(y),I(x)}

It should be pointed out that the specific implementation (pseudo code) of the above-mentioned embodiments and algorithms is not a limitation of the present invention, and the present invention is not limited to the above-mentioned examples. Changes, modifications, additions or substitutions should all belong to the protection scope of the present invention.

Claims (5)

1. An image processing method is applicable to an image processing device, the device comprises an image acquisition module, an image preprocessing module, an image processing module and an image segmentation module, and the method is characterized by comprising the following steps:

s1, acquiring images, and acquiring images through an image acquisition module;

s2, image preprocessing, namely, carrying out denoising, contrast enhancement and image morphology operation on the acquired image, completing the preprocessing of the image data and providing high-quality image data for the subsequent main processing process;

s3, image processing, including image saliency target detection processing, and smoothing and morphological operation processing;

s4, image segmentation, namely, performing target segmentation on the final image in the S3 through adaptive threshold segmentation to obtain a region of interest;

wherein the step of detecting the image salient object comprises the following steps,

a1, performing BMBD algorithm processing;

a2, based on the processing result of A1, performing FMBD algorithm processing again;

a3, fusing the two significance detection graphs obtained in A1 and A2 to obtain a BF _ MBDS graph for significance detection;

the steps of the BMBD algorithm processing are,

b1, preprocessing the input image, including denoising and color space conversion;

b2, performing channel separation on the processed image;

b3, setting initial seed points, setting the periphery of the image as seed points, and performing diffusion treatment based on the seed points;

b4, diffusing from the periphery to the center in a four-neighborhood mode, and performing diffusion updating for one circle;

b5, when the pixel point of the previous round of diffusion updating is in the next round of diffusion, the new updating is carried out in the same way until all the pixel points are updated and traversed;

b6, respectively obtaining diffusion images BMBD _ L, BMBD _ a and BMBD _ B of the three channels;

b7, fusing the images BMBD _ L, BMBD _ a and BMBD _ B obtained in B6 in a maximum value mode to obtain a BMBD _ Lab image;

b8, further processing the BMBD _ Lab image obtained in B7, mainly comprising smoothing processing and contrast enhancement, and obtaining a final BMBD _ map detection image;

the FMBD algorithm process has the steps of,

c1, preprocessing the input image, including denoising and color space conversion;

c2, channel separation is carried out on the processed image;

c3, setting an initial seed point, selecting the pixels with the confidence coefficient, namely the highest significance value based on the BMBD algorithm detection graph, and performing diffusion treatment by taking the pixel points as initialized seed points;

c4, diffusing from the periphery to the center in a four-neighborhood mode, and performing diffusion updating for one circle;

c5, when the pixel point of the previous diffusion updating is in the next diffusion, the new updating is carried out in the same way until all the pixel points are updated and traversed;

c6, obtaining diffusion images FMBD _ L, FMBD _ a and FMBD _ b of the three channels respectively;

c7, fusing the images FMBD _ L, FMBD _ a and FMBD _ b obtained in the C6 in a mode of taking the maximum value to obtain an FMBD _ Lab image;

and C8, further processing the FMBD _ Lab image obtained in the C7, mainly comprising smoothing processing and contrast enhancement, and obtaining a final FMBD _ map detection image.

2. The image processing method according to claim 1, characterized in that: the detection of the image salient object in the step S3 mainly includes selection of seed points, determination of diffusion order, and image fusion of different channels.

3. An image processing system comprises an image acquisition device, an image transmission module, an image processing device, an image display module and an image storage module, and is characterized in that:

the image acquisition equipment acquires data in a scene that a user needs to detect and segment, and can set single-frame image acquisition of the data and video acquisition of different video sequences according to the user requirements;

the image transmission module is used for transmitting the acquired data;

the image processing device processes the image transmitted by the image transmission module by adopting the image processing method of claim 1;

the image storage module is used for storing the image data transmitted by the image transmission module and the intermediate data in the processing process of the image processing device;

and the image display module displays the processed result in real time for a user to observe.

4. The image processing system according to claim 3, characterized in that: the processing procedure of the image processing apparatus mainly includes the following steps,

d1, preprocessing the transmission data to provide high-quality data for a later detection algorithm;

d2, processing the image transmitted by the transmission module according to the detection algorithm of claim 1 to obtain a detection image;

and D3, performing segmentation processing on the detection image obtained in the step D2 to obtain a final segmentation map.

5. The image processing system according to claim 3, characterized in that: the image transmission module adopts two data transmission modes of wire and wireless, and meets the requirements of different users under different application conditions.

Images