TWI613903B - Apparatus and method for combining with wavelet transformer and edge detector to generate a depth map from a single image - Google Patents

Abstract

A device and method for establishing a single image depth map combining wavelet transform and edge detection, in order to provide a conversion system for converting a 2-dimensional (2D) planar image signal into a 3-dimensional (3D) stereoscopic image signal image In the depth map generated by the depth map creation method of the present invention, each image block is assigned a depth value. The method of the present invention includes the following steps: S1 input the original image; S2 performs wavelet transformation on the original image and Edge detection; S3 creates a defocus map based on the wavelet conversion results and edge detection results; S4 performs depth prediction based on the wavelet coefficient values of the wavelet conversion; S5 corresponds to the depth value to the defocus map to perform depth diffusion; S6 Finally, a depth map is generated.

Description

Device and method for establishing single image depth map combining wavelet transform and edge detection

The invention relates to a video information system, in particular to a depth map generating device and method for converting two-dimensional image data into three-dimensional image data.

Since the launch of Avatar 3D movies in 2009, people have begun to pursue the entertainment effects of 3D display technology. In 2010, the 3D broadcast World Cup football game, until the 2016 virtual reality helmet, all show that our entertainment industry has changed from 2D Like 3D, people are no longer satisfied with the image effects brought by 2D, and they are beginning to pursue 3D display technology. At present, because of the commercialization of 3D display technology and the increasing services related to 3D content, relative users Demand has also increased, however, there has been no significant progress in the development of 3D content. In contrast, there are quite a large number of 2D images or videos, and the images taken by individuals are also 2D images, which are waiting to be effectively used. For conversion into 3D video applications.

For this reason, some inventors have invented such as Chinese Patent Publication No. CN 103559701 A "Depth Estimation Method of Two-Dimensional Single-View Images Based on DCT Coefficient Entropy", which proposes to perform depth prediction in a single image with depth of field, For each pixel in the image to be processed, Take the pixel as the center to capture the window as a sub-image, and after performing wavelet transformation on these sub-images, quantize the wavelet coefficient value in the image, and then calculate the coefficient entropy to be the blur of the pixel, and then Linear mapping maps the entropy value to an 8-bit depth range to obtain a pixel-level depth map. In addition, some inventors proposed the Chinese Patent Announcement No. CN 10247539B "Video image 2D to 3D method", which uses wavelet transform to predict the depth of a single image with depth of field, by performing wavelet transform on the original image to Extract high-frequency coefficients in the image and divide the image into several blocks, then count the number of non-zero coefficients in each block as the blur degree of the block, and at the same time, based on the color characteristics of the original image, the original image The color is divided into three types of pixel sets, and then the ambiguity of each pixel set is compared to calculate the average value, the pixel set corresponding to the maximum value is used as the foreground, the pixel set corresponding to the next largest value is regarded as the middle scene, and the minimum corresponding pixel value is Seen as the background, the system of preset depth of field finally assigns different depth values to the foreground, middle scene, and background to obtain a depth map.

According to the prior art disclosed above, the prior art and the conventional depth map generation method have related shortcomings. For a window centered on a pixel, the size of the window setting needs to be manually set and cannot be automatically adjusted according to different images. . Furthermore, the color features of the original image are used to divide the image into a set of three types of pixels, and only the image is divided into the foreground, the middle scene, and the background. Obviously, the rich images we usually see have different levels of depth information, which leads to The correct depth map cannot be generated.

In view of this, the present inventors have been engaged in related industries and research for many years, and have conducted research and analysis on the existing depth map generation methods, with a view to inventing a depth that improves the conventional shortcomings The method of generating a picture. Therefore, the main purpose of the present invention is a device and method for creating a single image depth map in combination with wavelet transformation and edge detection that does not require human interference and conforms to the depth information viewed by the human eye.

To achieve the above purpose, the device and method for creating a single image depth map combining wavelet transform and edge detection according to the present invention has an image capture unit for capturing or inputting an original image; an image analysis unit , And presents an information link with the image capturing unit to perform image analysis on the original image captured or input by the image capturing unit, wherein the image analysis can execute an image analysis algorithm, and the image analysis calculation The method can be wavelet transformation, edge detection, etc .; an image synthesis unit, which is connected to the image analysis unit, and when the image analysis unit executes the image analysis algorithm, the image synthesis unit can perform image synthesis on the results of the image analysis unit analysis To generate a defocus map; a depth calculation unit, which is connected to the image synthesis unit for information, when the image synthesis unit generates the defocus map, the depth calculation unit performs a depth prediction based on the analysis result of the image analysis unit Algorithm, and the result of depth prediction is mapped to the defocus map, so that the depth calculation unit can be based on the depth prediction The results correspond to the defocus map, performing a deep diffusion algorithm, and the depth of diffusion algorithm may be a Laplacian interpolation technique or a global interpolation algorithm, it is not limited thereto.

1‧‧‧ Combined wavelet transform and edge detection to create a single image depth map device

11‧‧‧Image capture unit

12‧‧‧Image analysis unit

13‧‧‧Image synthesis unit

14‧‧‧Depth calculation unit

S1‧‧‧Enter the original image steps

S2‧‧‧Image analysis steps

S22‧‧‧edge detection steps

S23‧‧‧Wavelet conversion steps

S231‧‧‧Convert to grayscale image

S232‧‧‧Find the local maximum straight step

S233‧‧‧Local maximum value corresponds to wavelet coefficient value

S234‧‧‧ Threshold calculation result

S3‧‧‧ Steps to create defocus map

S4‧‧‧Depth prediction steps

S41‧‧‧Number of local maximum histogram steps

S42‧‧‧ Steps to create a window based on the number

S43‧‧‧ Step of calculating the ambiguity of the wavelet transform result

S44‧‧‧Depth prediction result step

S5‧‧‧Deep diffusion step

S6‧‧‧Produce depth map steps

Figure 1 is a schematic diagram of the present invention.

Figure 2 is a flow chart of the steps of the present invention.

Figure 3 is a schematic diagram of the implementation of the present invention.

Figure 4 is a schematic diagram of the implementation of the present invention (1).

Figure 5 is a schematic diagram of the implementation of the present invention (2).

Figure 6 is a schematic diagram of an embodiment of the present invention.

Fig. 7 is a schematic diagram (1) of an embodiment of the present invention.

Figure 8 is a schematic diagram (2) of the implementation of the present invention.

In the description of the following specification, the term "depth map" refers to a two-dimensional matrix of depth values, and each depth value in the matrix corresponds to the relative position of a scene, and each depth value represents a specific reference position For the distance to each relative position of the scene, if each pixel of a 2D image has its own depth value, the 2D image can be displayed using 3D technology.

In order to enable your reviewing committee to have a better understanding and understanding of the purpose, technical means and effects of the present invention, I would like to use the preferred embodiment together with the schematic description.

Please refer to "Figure 1". The figure shows a schematic diagram of the structure of the present invention. The image acquisition unit 11, an image analysis unit 12, an image synthesis unit 13, and a depth calculation unit 14 are constructed, wherein the image acquisition unit 11 can acquire an original image, and the original image For a 2D image or video, the image analysis unit 12 and the image acquisition unit 11 present an information link to receive the original image and execute a plurality of image analysis algorithms, wherein the image analysis algorithm may be One or a combination of a wavelet transform algorithm and an edge detection algorithm, and the wavelet transform algorithm can be discrete wavelet transform or continuous wavelet transform, and the edge detection algorithm can be Roberts Cross operator, Prewitt calculation , Sobel operator, Canny operator, compass operator, Marr-Hildreth, wavelet transformation, but any edge detection algorithm that can detect the original image is within the scope of the implementation of the present invention, but not This is the limit. The image synthesis unit 13 and the image analysis unit 12 present an information link for performing an image synthesis on the image results analyzed by the image analysis unit 12 to generate a defocused image. The depth calculation unit 14 and the image synthesis unit 13 present an information link. After performing a depth prediction algorithm based on the image results analyzed by the image analysis unit 12, the depth prediction algorithm to the result is performed by the image synthesis unit 13 After synthesis, the depth calculation unit 14 successively executes a depth diffusion algorithm, and the depth diffusion algorithm may be a Laplace interpolation technique or a global interpolation algorithm, which is not limited to The original image is matched with a depth map.

As mentioned above, and please refer to "Figure 2", the figure shows the flow chart of the steps of the present invention. The original image input by the unit 11. An image analysis step S2, which includes a wavelet transformation step S23 and an edge detection step S22, which performs a wavelet transformation analysis and an edge detection analysis on the original image, wherein the wavelet transformation step S23 performs a wavelet transformation Algorithm to generate a wavelet transform analysis result, and the wavelet transform algorithm can be one of a discrete wavelet transform or a continuous wavelet transform, which is not limited thereto, and the edge detection step S22 is executed An edge detection algorithm to produce an edge detection result, and the edge detection algorithm can be Roberts Cross operator, Prewitt operator, Sobel operator, Canny operator, compass operator, Marr-Hildreth, One of the wavelet transforms, and please refer to "Figure 3". The figure shows the schematic diagram of the implementation of the present invention, as shown in the figure is the result of wavelet transform analysis binarization. A step S3 of creating a defocus map, which is the image synthesis unit 13 transforming the wavelet analysis result and the The results of the edge detection are combined to produce a defocused map. Please refer to "Figure 4" for the matching. The figure shows the schematic diagram (1) of the implementation of the present invention. The defocused map is shown in the figure. The synthesis is that the result of the edge detection corresponds to the result of the wavelet transform analysis to extract the coefficient values of the pixels in the edge detection result in the result of the wavelet transform analysis. A depth prediction step S4 is for the depth calculation unit 14 to perform a depth prediction algorithm based on the wavelet conversion result of the original image after the wavelet transformation on the image analysis step S2 to perform a depth prediction on the original image. The depth calculation unit 14 After executing the depth prediction algorithm, the defocus map generated by the step S3 of creating a defocus map is synthesized by the image synthesis unit 13 to generate a defocus depth map. The synthesis is that the result of the depth prediction corresponds to the result of the defocus map, and the result of the depth prediction is replaced with the defocus map. A depth diffusion step S5 is to execute a depth diffusion algorithm based on the defocused depth map, and the depth diffusion algorithm may be the Laplacian interpolation technique or the global interpolation algorithm. Finally, in the step S6 of generating a depth map, after the depth calculation unit 14 executes the depth diffusion algorithm, a depth map will be generated. Please refer to "Figure 5". The figure shows the schematic diagram of the implementation of the invention (2). The depth map is shown in the figure.

As mentioned above, and please also refer to "Figure 6", the figure shows a schematic diagram of an embodiment of the present invention. As shown in the figure, the depth prediction algorithm executed by the depth prediction step S4 has the following steps: Step S41 of the local maximum number of histograms is to find the number of peaks of the grayscale histogram of the original image in the grayscale value histogram, a window is created based on the number of steps S42, based on the number of peaks A calculation window is established, and a pair of wavelet transformation results are used to calculate the ambiguity S43. Based on the wavelet transformation results, a neighborhood calculation is performed centering on the center pixel of the window, and the wavelet transformation results of the center pixel neighborhood are calculated. The depth prediction result step S44 is to perform depth prediction according to the result of calculating the pixel neighborhood of the center point, that is, the blur degree.

As mentioned above, please also refer to "Figure 7". The figure shows a schematic diagram of an embodiment of the invention (1). As shown in the figure, the wavelet conversion step S23 further includes a wavelet conversion threshold setting step, The step includes a step S231 of converting a grayscale image to convert the original image into a grayscale image. In a step S232 of finding the local maximum value, a gray-scale value histogram is created based on the gray-scale image of the original image, and in the gray-scale value histogram, the gray-scale value at which the peak is located is found. A local maximum value corresponds to the wavelet coefficient value step S233, which locates all grayscale values where the peak value is found, which is located at the position of the original image corresponds to the position of the coefficient value in the wavelet transformation result, and combines all the coefficient values Extract it. In a threshold calculation result step S234, the extracted coefficient values are subjected to numerical analysis to establish a wavelet transform threshold, wherein the numerical analysis can be Simpson's law, and the threshold calculation function is as follows:

Among them, f (x) is all the coefficient values extracted, and among all coefficient values, the top three coefficient values are taken, and the threshold value is calculated through equation (1). After the threshold value calculation is completed, the following Function:

Among them, I (m, n) is the result of wavelet conversion, Th is the calculated threshold, and please refer to "Figure 3", the figure shows the schematic diagram of the implementation of the present invention, as shown in the figure after wavelet conversion and The result of binarization is the result of not setting the threshold, and please refer to "Figure 8". The figure shows the schematic diagram of the implementation of the present invention (two). The figure shows the result after the threshold is established , Set the result greater than or equal to the threshold to 255, which is the white part, and the result less than the threshold to 0, which is the black part.

In summary, the device and method for creating a single image depth map combining wavelet transform and edge detection in the present invention mainly use an image analysis algorithm to perform image analysis on an original image so that a depth calculation unit can After the depth prediction algorithm is executed, the depth diffusion algorithm is executed to generate a depth map. Since the image analysis algorithm is fast and has high accuracy, no complicated calculation is required, so the measurement efficiency is good, and because the present invention does not need to be complicated and The huge calculation, so the cost is relatively reduced, and it can achieve the main purpose of the present invention without artificial interference, and is in line with the depth information viewed by the human eye combined with wavelet transform and edge detection device to create a single image depth map and Its method.

Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the patent application scope of the present invention. Anyone who is familiar with this art will not deviate from the spirit and scope of the present invention, and may make some changes and modifications. Therefore, the protection scope of the present invention is not limited thereto.

S1‧‧‧Enter the original image steps

S2‧‧‧Image analysis steps

S22‧‧‧wavelet conversion steps

S23‧‧‧Edge detection steps

S3‧‧‧ Steps to create defocus map

S4‧‧‧Depth prediction steps

S5‧‧‧Deep diffusion step

S6‧‧‧Produce depth map steps

Claims (8)

A device combining wavelet transform and edge detection to create a single image depth map, which includes: an image capture unit for inputting an original image; an image analysis unit connected to the image capture unit for information To execute an image analysis algorithm, and analyze the original image input by the image capture unit through a wavelet transformation algorithm and an edge detection algorithm, and the wavelet transformation algorithm further includes a wavelet transformation threshold setting step; An image synthesizing unit, which is connected to the image analysis unit for information, and is used for analyzing the plurality of analysis results of the original image by the image analysis unit, performing an image synthesis on the plurality of analysis results, and generating a defocus map; A depth calculation unit, which is linked to the image synthesis unit for information, can execute a depth prediction algorithm based on the plurality of analysis results, after completion, execute a depth diffusion algorithm, and, after the depth diffusion algorithm is executed, can generate A depth map. The device combining wavelet transform and edge detection to create a single image depth map as described in item 1 of the patent application scope, wherein the wavelet transform threshold setting step includes the steps of converting to grayscale image and searching for local The maximum value step, a local maximum value correspond to the wavelet coefficient value and a threshold calculation result. As described in item 1 of the patent application scope, the device combining wavelet transform and edge detection to create a single image depth map, wherein the wavelet transform algorithm can be one of discrete wavelet transform or continuous wavelet transform. Create a single image by combining wavelet transform and edge detection as described in item 1 of the patent application The depth map device, wherein the edge detection algorithm can be one of Roberts Cross operator, Prewitt operator, Sobel operator, Canny operator, compass operator, Marr-Hildreth, wavelet transform. The device for creating a single image depth map combining wavelet transform and edge detection as described in item 1 of the patent scope, wherein the steps of the depth prediction algorithm executed by the depth calculation unit are based on the original image The number of local maximums of a gray-scale value histogram creates a window, and performs a calculation of ambiguity based on the result of the image analysis. A method combining wavelet transform and edge detection to create a single image depth map, which is applied to a device combining wavelet transform and edge detection to create a single image depth map, includes an image capture unit, an image analysis unit, An image synthesis unit and a depth calculation unit, and the units described above are connected to each other in information. The device that combines wavelet transform and edge detection to create a single image depth map, the implementation method includes: a step of inputting original images The image capture unit inputs an original image; an image analysis step, in succession, the image analysis unit executes a wavelet transform algorithm and an edge detection algorithm to analyze the original image input by the image capture unit, and the wavelet The conversion algorithm further includes a wavelet conversion threshold setting step; a defocus map creation step, when the image analysis step is completed, the image synthesis unit performs an image on the result of the original image analyzed by the image analysis step Synthesis to produce a defocus map; a depth prediction step, when the defocus map is generated, the depth Depth calculation unit performs a prediction algorithm based on the results of the image analysis step; step a diffusion depth, when the depth of the depth calculation unit after executing prediction algorithm, A depth diffusion algorithm is executed; a step of generating a depth map is continued, and after the depth diffusion algorithm is completed, a depth map is generated. The method of combining a wavelet transform and edge detection to create a single image depth map as described in item 6 of the patent scope, wherein the step of the depth prediction algorithm is based on a part of a grayscale value histogram of the original image The maximum number is used to establish a window, and a blur degree calculation is performed based on the result of the image analysis. As described in item 6 of the patent application scope, the method of combining wavelet transform and edge detection to create a single image depth map, wherein the depth diffusion algorithm can be a Laplace interpolation technique or a global interpolation algorithm, One of them.