TWM535848U - Apparatus for combining with wavelet transformer and edge detector to generate a depth map from a single image - Google Patents

Description

Device for establishing single image depth map combined with wavelet transform and edge detection

This creation relates to video systems, and more particularly to a Depth Map generation device for converting 2D image data into 3D image data.

Since the launch of Avatar 3D movies in 2009, people have begun to pursue the entertainment effects of 3D display technology. In 2010, 3D broadcasted the World Cup soccer game, and until 2016, the virtual reality helmets all showed that our entertainment industry has turned from 2D. Like 3D, people are not satisfied with the image effects brought by 2D, and are beginning to pursue 3D display technology. At present, because of the commercialization of 3D display technology and the services related to 3D content, the relative users are concerned with 3D. Demand has also increased. However, there has been no significant progress in the development of 3D content. In contrast, there is a considerable amount of 2D video or video, and the images taken by individuals are also 2D images, which are waiting to be effectively utilized. In order to convert to a 3D video application.

For this reason, the inventors have invented, as disclosed in Chinese Patent Publication No. CN 103559701 A, "Two-dimensional single-view image depth estimation method based on DCT coefficient entropy", which proposes depth prediction in a single image having a depth of field, For each pixel in the image to be processed, the window is taken as a sub-image centered on the pixel, and wavelet transform is performed on the sub-images, and the wavelet coefficient values in the image are quantized, and then calculated. The coefficient entropy is used as the ambiguity of the pixel, and then the entropy value is mapped to an 8-bit depth range through a linear mapping to obtain an image. A depth map of the prime level. In addition, the inventor has proposed Chinese Patent Publication No. CN 10247539B "Method for Video Image 2D to 3D", which uses wavelet transform to predict the depth of a single image having a depth of field, and performs wavelet conversion on the original image. Extract the high-frequency coefficients in the image, and divide the image into several blocks, then count the number of non-zero coefficients in each block as the blur of the block, and based on the color features of the original image, perform the original image. The color is divided into three types of pixel sets, and then the ambiguity of each pixel set is compared to obtain a statistical average value, and the pixel set corresponding to the maximum value is used as the foreground, and the pixel set corresponding to the next largest value is regarded as the middle scene, and the minimum corresponding pixel value is As the background, the system with the preset depth of field finally assigns different depth values to the foreground, the middle scene and the background to obtain the depth map.

The prior art is known from the prior art. The conventional depth map generation method has related disadvantages. For a window centered on a pixel, the size of the window setting needs to be manually set, and the image cannot be automatically adjusted according to different images. . Furthermore, using the color features of the original image to divide the image into a collection of three types of pixels, the image is only divided into the foreground, the middle scene and the background. Obviously, the rich images we usually see have different levels of depth information, resulting in different Unable to produce the correct depth map.

In view of this, the creator is based on years of research in related industries and research, and researches and analyzes existing depth map generation methods, and can create a depth map generation method for improving the shortcomings of the conventional knowledge. Therefore, the main purpose of the creation is It is a device that combines wavelet transform and edge detection to establish a single image depth map without manual intervention and conforming to the depth information viewed by the human eye.

In order to achieve the above object, the device for creating a single image depth map combined with wavelet transform and edge detection according to the present invention has an image capturing unit for capturing or inputting a primary image. For example, an image analysis unit is coupled to the image capture unit for performing an image analysis algorithm, and the image analysis algorithm can be one of wavelet transform or edge detection or a combination thereof, and analyze The image capturing unit captures or inputs the original image, and an image synthesizing unit is coupled with the image analyzing unit for analyzing the plurality of analysis results after the image analyzing unit analyzes the original image. The analysis result performs an image synthesis to generate a defocus map, and a depth calculation unit is coupled with the image synthesis unit to perform a depth prediction algorithm according to the plurality of analysis results, and after performing a depth diffusion calculation The method, and, after the depth diffusion algorithm is executed, generates a depth map.

1‧‧‧Device for establishing a single image depth map in combination with wavelet transform and edge detection

11‧‧‧Image capture unit

12‧‧‧Image Analysis Unit

13‧‧‧Image synthesis unit

14‧‧‧Deep calculation unit

S1‧‧‧Input original image steps

S2‧‧‧ image analysis steps

S22‧‧‧Edge detection device steps

S23‧‧‧wavelet conversion device steps

S231‧‧‧ converted to grayscale image

S232‧‧‧ Looking for the local maximum straight step

S233‧‧‧Local maximum corresponds to wavelet coefficient value

S234‧‧‧ threshold calculation results

S3‧‧‧ Establishing a defocusing step

S4‧‧‧Deep prediction device steps

S41‧‧‧Steps for the local maximum of the histogram

S42‧‧‧Create window steps based on number

S43‧‧‧The ambiguity calculation step for wavelet transform results

S44‧‧‧Deep prediction results steps

S5‧‧‧Deep diffusion device steps

S6‧‧‧ Generate depth map steps

Figure 1 is a schematic diagram of the structure of the creation.

Figure 2 is a flow chart of the steps of the creation.

Figure 3 is a schematic diagram of the implementation of this creation.

Figure 4 is a schematic diagram of the implementation of this creation (1).

Figure 5 is a schematic diagram of the implementation of the creation (2).

Figure 6 is a schematic view of an embodiment of the present creation.

Figure 7 is a schematic view (I) of an embodiment of the present invention.

Figure 8 is a schematic view (2) of an embodiment of the present invention.

In the description of the following description, the term "depth map" refers to a two-dimensional matrix of depth values, and each depth value in the matrix corresponds to a relative position of a scene, and each depth value represents a specific reference position. The distance to each relative position of the scene, if a 2D image Each pixel has its own depth value, and the 2D image can be displayed using 3D technology.

In order to enable your review committee to have a better understanding and understanding of the purpose, technical means and efficacy of this creation, please refer to the preferred embodiment with a schematic description.

Please refer to "Figure 1". The figure shows the structure of the creation. As shown in the figure, the device 1 for creating a single image depth map combined with wavelet transform and edge detection is mainly composed of an image. The image capturing unit 11 is configured to capture an original image, and the original image is captured by the image capturing unit 12, an image combining unit 13, and a depth calculating unit 14. The image analysis unit 12 is coupled to the image capturing unit 11 for receiving the original image, and then performing a plurality of image analysis algorithms, wherein the image analysis algorithm can be a wavelet transform, an edge detection or a combination thereof, wherein the wavelet transform can be a discrete wavelet transform or a continuous wavelet transform, and the edge detecting device can be a Roberts Cross operator and a Prewitt operator. , Sobel operator, Canny operator, compass operator, Marr-Hildreth, wavelet transform, but the detection of edge detection in the original image is within the scope of the implementation of the creation, but not Limited. The image synthesizing unit 13 and the image analyzing unit 12 are connected to each other for performing image synthesizing on the image result analyzed by the image analyzing unit 12, thereby generating a defocusing map. The depth calculation unit 14 is connected to the image synthesis unit 13 and performs a depth prediction algorithm according to the image result analyzed by the image analysis unit 12, and then the result is transmitted through the image synthesis unit 13 through the depth prediction device. Then, the depth calculation unit 14 successively performs a depth diffusion algorithm to generate a depth map that is matched with the original image.

As mentioned above, please refer to "Fig. 2", which shows the flow chart of the steps of the creation. As shown in the figure, the implementation steps of the creation are as follows: an input original image step S1, which is The original image input by the image capturing unit 11. An image analysis step S2 includes a wavelet conversion device step S23 and an edge detection device step S22, performing a wavelet transform analysis and an edge detection analysis on the original image, wherein the wavelet transform device step S23 is performed by The image analysis unit 12 performs a wavelet transform algorithm to generate a wavelet transform analysis result, and the wavelet transform algorithm may be one of a discrete wavelet transform or a continuous wavelet transform, and is not limited thereto. The edge detecting device performs an edge detection algorithm by the image analyzing unit 12 to generate an edge detection result, and the edge detection may be a Roberts Cross operator, a Prewitt operator, a Sobel operator, Canny operator, compass operator, Marr-Hildreth, wavelet conversion, and please refer to "3rd picture", the figure shows the implementation of the creation, as shown in the figure for wavelet transformation analysis binarization The result. a defocusing step S3 is established, wherein the image synthesizing unit 13 synthesizes the wavelet transform analysis result and the edge detection result to generate a defocus map, please refer to "4th figure", Shown as a schematic diagram of the implementation of the creation (1), the defocus map is shown in the figure. The synthesizing is that the result of the edge detection corresponds to the wavelet transform analysis result, so as to extract a coefficient value of the pixel point in the edge detection result of the wavelet transform analysis result. a depth prediction device step S4, wherein the depth calculation unit 14 performs a depth prediction algorithm on the wavelet transform result of the wavelet transform on the original image according to the image analysis step S2, to perform a depth prediction on the original image, and the depth calculation After the unit 14 executes the depth prediction algorithm, the defocus map generated by the establishing the defocus map step S3 is synthesized by the image synthesizing unit 13 to generate a defocus depth map. The synthesis is that the result of the depth prediction corresponds to the result of the defocus map, and the result of the depth prediction is replaced with a defocus map. A depth diffusing device step S5 is performed to perform a depth diffusion algorithm according to the defocus depth map. And the depth diffusion algorithm may be the Laplacian interpolation technique or the global interpolation algorithm, and finally, the depth map is generated. In step S6, after the depth calculation unit 14 executes the depth diffusion algorithm, a depth map is generated. Please refer to "figure 5", which shows the implementation diagram of the creation (2), as shown in the figure. The depth map.

As described above, please also refer to "FIG. 6", which shows a schematic diagram of an embodiment of the creation. As shown, the depth prediction means step S4, the depth performed by the depth calculation unit 14 The step of the prediction algorithm further includes a step of the local maximum value of the histogram step S41, and the depth calculation unit 14 is used to find the peak value of the grayscale value histogram of the original image in the histogram of the grayscale value. A window is established according to the number of steps S42, a calculation window is established according to the number of peaks, and a pair of wavelet transformation results are used for ambiguity calculation S43, and a neighborhood calculation is performed centering on the central pixel of the window according to the wavelet transformation result. The wavelet transform result of the neighborhood of the central pixel point is calculated, and a depth prediction result step S44 is performed according to the result of calculating the pixel neighborhood of the center point, that is, the ambiguity.

In the above, please refer to "Figure 7" at the same time. The figure shows a schematic diagram (1) of the embodiment of the creation. As shown in the figure, the wavelet conversion step S23 further includes a step of setting a wavelet switching threshold. The step performed by the image analyzing unit 12 includes a step of converting to a grayscale image, S231, which converts the original image into a grayscale image. A searching for the local maximum step S232 is to establish a grayscale value histogram according to the grayscale image of the original image, and find the grayscale value of the peak in the grayscale value histogram. A local maximum corresponds to the wavelet coefficient value step S233, and all the grayscale values of the found peak are located at the position of the original image corresponding to the coefficient value in the wavelet transform result, and all the coefficient values are Take it out. A threshold value calculation result step S234, the coefficient value obtained is subjected to numerical analysis, and a wavelet switching threshold is established, wherein the numerical analysis may be a Simpson's rule, and the threshold calculation function is as follows: Where f(x) is the value of all the coefficients taken, and the coefficient values of the first three major coefficients of all the coefficient values are calculated by the equation (1). When the threshold value is calculated, the following is satisfied. Function: Among them, I(m,n) is the wavelet conversion result, Th is the calculation threshold, and please refer to “3rd picture”. The figure shows the implementation diagram of the creation, as shown in the figure after wavelet transformation. The result of the binarization is the result of not setting the threshold, and please refer to "8th figure". The figure shows the schematic diagram (2) of the embodiment of the creation. The figure shows the result after the threshold is established. The result of setting the value greater than or equal to the threshold is 255, that is, the white portion, and the result smaller than the threshold is 0, that is, the black portion.

In summary, the device for combining a wavelet transform and edge detection to create a single image depth map mainly performs image analysis on an original image by an image analysis algorithm, so that a depth calculation unit can perform depth prediction. After the algorithm, the deep diffusion algorithm is executed to generate the depth map. Since the image analysis algorithm is fast and accurate, it does not require complicated calculations, so the measurement efficiency is good, and the creation does not need complicated and huge. The operation, so the cost is relatively reduced, and the device that can achieve the main purpose of the creation without manual intervention and conforming to the depth information viewed by the human eye, combined with wavelet transform and edge detection, establishes a single image depth map.

Although the present invention has been disclosed in the above preferred embodiments, it is not intended to limit the scope of the present invention. Anyone skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of this creation is not limited to this.

1‧‧‧Device for establishing a single image depth map in combination with wavelet transform and edge detection

11‧‧‧Image capture unit

12‧‧‧Image Analysis Unit

13‧‧‧Image synthesis unit

14‧‧‧Deep calculation unit

Claims (6)

An apparatus for establishing a single image depth map by combining wavelet transform and edge detection includes: an image capturing unit for inputting an original image; and an image analyzing unit for linking information with the image capturing unit for use Performing an image analysis algorithm and analyzing the original image input by the image capturing unit; an image synthesizing unit is coupled with the image analyzing unit for analyzing the plurality of images after the image analyzing unit analyzes the original image The result of the analysis is performed by performing an image synthesis on the plurality of analysis results to generate a defocus map; a depth calculation unit is coupled with the image synthesis unit to perform a depth prediction algorithm according to the plurality of analysis results, and completes Thereafter, a depth diffusion algorithm is performed, and after the depth diffusion algorithm is executed, a depth map can be generated. The apparatus for establishing a single image depth map by combining wavelet transform and edge detection according to the first aspect of the patent application, wherein the image analysis algorithm may be one of wavelet transform or edge detection or a combination thereof. . The apparatus for establishing a single image depth map by combining wavelet transform and edge detection according to the second aspect of the patent application, wherein the wavelet transform may be one of discrete wavelet transform or continuous wavelet transform. The apparatus for establishing a single image depth map by combining wavelet transform and edge detection according to the second aspect of the patent application, wherein the edge detection may be a Roberts Cross operator, a Prewitt operator, a Sobel operator, and a Canny calculation. Sub, compass, Marr-Hildreth, wavelet transform one of them. The apparatus for establishing a single image depth map by combining the wavelet transform and the edge detection according to the first aspect of the patent application, wherein the step of performing the depth prediction algorithm performed by the depth calculation unit is based on the original image a local maximum number of grayscale value histograms, creating a window And calculate a ambiguity based on the result of the image analysis. The apparatus for establishing a single image depth map by combining wavelet transform and edge detection according to claim 1 of the patent application scope, wherein the depth diffusion algorithm performed by the depth calculation unit may be a Laplacian interpolation technique or A global interpolation algorithm, one of which.