WO2016058359A1

WO2016058359A1 - Method and device for generating three-dimensional image

Info

Publication number: WO2016058359A1
Application number: PCT/CN2015/077900
Authority: WO
Inventors: 张启平; 孙李娜
Original assignee: 华为技术有限公司
Priority date: 2014-10-17
Filing date: 2015-04-30
Publication date: 2016-04-21
Also published as: CN105574926A; CN105574926B

Abstract

The embodiments of the present invention relate to a method and device for generating a three-dimensional image. The method comprises: acquiring a plurality of micro unit images; dividing a plurality of feature regions on each of the micro unit images, a difference value between colour values of any two pixel points in each feature region of the plurality of feature regions being less than or equal to a first threshold value; according to the plurality of feature regions, determining a plurality of region planes, wherein the feature regions included in each region plane belong to the same object or belong to the same source region, and each feature region of the plurality of feature regions only belongs to one region plane of the plurality of region planes; determining a region plane depth value of each region plane; and obtaining a three-dimensional image according to the region plane depth value. By means of the method and device for generating a three-dimensional image in the embodiments of the present invention, a depth value can be more accurately extracted, thereby enabling the generated three-dimensional image to be more accurate and realistic, and an application scenario range to be wider.

Description

Method and apparatus for generating three-dimensional images

The present application claims priority to Chinese Patent Application No. 201410551038.8, entitled "Method and Apparatus for Generating Three-Dimensional Images", filed on October 17, 2014, the entire contents of which is incorporated herein by reference. .

Technical field

The present invention relates to the field of image processing, and more particularly to a method and apparatus for generating a three-dimensional image.

Background technique

Three-dimensional reconstruction is getting more and more attention because of the ability to reconstruct the three-dimensional model information of objects. In the existing 3D reconstruction technology, a method for determining the depth of the anti-aliasing iteration of the viewpoint image is proposed, which mainly solves the problem of deep error. Since the method puts the target of depth determination on the viewpoint image, this inevitably leads to a low depth resolution and cannot obtain an overly complex scene, which results in a limited application range of the method.

In addition, in the field of machine vision research, how to extract the depth information of the target from two two-dimensional images with disparity to reconstruct the three-dimensional contour of the target has always been an important issue, and the computational reconstruction process of the light field imaging It is also a process of restoring three-dimensional information of objects from two-dimensional micro-cell images with different directions and perspective information. There are many similarities between them. Therefore, the computational reconstruction methods for applying depth extraction to the field of light field imaging are endless. However, the existing three-dimensional reconstruction algorithm for light field imaging has a poor reconstruction effect and is difficult to apply in practice.

In the prior art, an improved high-resolution reconstruction method for computing integrated images is proposed, which mainly solves the problem that the existing computational integrated imaging reconstruction method has low resolution and high reconstruction complexity. The method determines the non-zero pixel points in the reconstructed image point by point, and superimposes the color value on the zero pixel points among the adjacent 8 pixel points, and integrates and restores the three-dimensional image of the target. However, the reconstruction method described by the method still results in a two-dimensional image, and cannot restore the depth information of the three-dimensional object, thereby reducing the advantages of light field imaging.

Summary of the invention

The present invention provides a method and apparatus for generating a three-dimensional image, which can extract depth values more accurately and more accurately generate a three-dimensional image.

In a first aspect, a method for generating a three-dimensional image is provided, the method comprising: acquiring a plurality of micro-cell images; dividing a plurality of feature regions on each of the micro-cell images, each of the plurality of feature regions The difference between the color values of any two pixel points is less than or equal to the first threshold; and the plurality of area planes are determined according to the plurality of feature areas, wherein the feature areas included in each of the area planes belong to the same object or belong to the same The source area, each of the plurality of feature areas belongs to only one of the plurality of area planes; the area plane depth value of each area plane is determined; and the three-dimensional image is obtained according to the area plane depth value.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the determining, by the plurality of feature regions, the plurality of region planes includes: determining a first feature region of the plurality of feature regions and the first a contiguous region of a feature region; determining a first joint probability density that the first feature region and the adjacent feature region of the first feature region do not belong to the same object; determining a contiguous feature region of the first feature region and the first feature region a second joint probability density belonging to the same object; determining a neighboring feature region of the first feature region and the first feature region when a ratio of the first joint probability density to the second joint probability density is less than or equal to a second threshold It belongs to the same area plane in the plurality of area planes, and the same area plane includes feature areas belonging to the same object.

With reference to the first aspect, in a second possible implementation manner of the first aspect, determining the plurality of area planes according to the plurality of feature areas, including: determining a second feature area of the plurality of feature areas, and the a contiguous region of the second feature region; a first likelihood ratio of the merge region and the second feature region, a second likelihood ratio of the merge region and the adjacent feature region of the second feature region, the merge region including the second a feature area and an adjacent area of the second feature area; determining the adjacency of the second feature area and the second feature area when the first likelihood ratio and/or the second likelihood ratio is less than or equal to a third threshold The feature area belongs to the same area plane in the plurality of area planes, and the feature area included in the same area plane belongs to the same object.

With reference to the first aspect, in a third possible implementation manner of the first aspect, the determining, by the plurality of feature regions, the plurality of region planes includes: determining, in the image of the first microcell in the plurality of feature regions a third feature area; determining a fourth feature area in the second microcell image of the plurality of feature areas that has the smallest color error value of the third feature area, the second microcell image and the first microcell image And the color error value of the fourth feature area and the third feature area is less than or equal to a fourth threshold; determining that the third feature area and the fourth feature area belong to the same area plane of the plurality of area planes, the same The feature area included in the area plane belongs to the homologous area.

In conjunction with the first aspect, in a fourth possible implementation of the first aspect, the Determining a plurality of area planes includes: determining a fifth feature area of the third micro unit image and a center pixel point of the fifth feature area of the plurality of feature areas; and in the fourth micro unit image, Centering on a pixel point on the same pole line of the central pixel, determining a plurality of regions having the same size and shape as the fifth feature region, the fourth microcell image being adjacent to the third microcell image; Determining, in the plurality of regions, a sixth feature region having a smallest color error value with the fifth feature region, wherein a color error value of the sixth feature region and the fifth feature region is less than or equal to a fifth threshold; determining the fifth feature region and The sixth feature area belongs to the same area plane in the plurality of area planes, and the feature area included in the same area plane belongs to the same area.

In conjunction with the first aspect or any one of the possible implementations of the first to fourth possible implementations of the first aspect, in a fifth possible implementation of the first aspect, the acquiring the plurality of micro-units The image includes: acquiring a light field image by using a light field camera; mapping each pixel point in the light field image to a five-dimensional space to obtain a corresponding mapped pixel point, where the coordinates of the five-dimensional space include: horizontal X-direction coordinates , vertical Y direction coordinate, red component intensity value coordinate, green component intensity value coordinate and blue component intensity value coordinate; the average color value of the highest density region in the neighborhood of the mapped pixel point is determined as the color value of the mapped pixel point; The mapped pixel points that determine the color value determine the plurality of microcell images.

In conjunction with the first aspect or any one of the first to fourth possible implementations of the first aspect, in a sixth possible implementation of the first aspect, the determining each of the regions Determining a plane depth value of the plane includes: determining at least one feature point in the plane of each area; determining a depth value of the at least one feature point; determining a depth value of the area plane of the area of each area, where the plane depth value is An average of depth values of the at least one feature point.

With reference to the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, the acquiring the plurality of micro unit images includes: acquiring the plurality of micro unit images by using a light field camera; Determining the depth value of the at least one feature point includes: determining a center interval of the adjacent lens of the light field camera; determining a distance from a plane where the plurality of microcell images are located to a plane of the light field camera lens array; determining the mth The disparity value of the feature point; the depth value w _m ' of the mth feature point is calculated according to the following formula:

Where t is the center spacing of adjacent lenses of the light field camera; i is the distance from the plane of the plurality of microcell images to the plane of the lens array of the light field camera; d _m is the parallax of the mth feature point value.

In conjunction with the seventh possible implementation of the first aspect, in an eighth possible implementation manner of the first aspect, determining the disparity value of the mth feature point includes: using the mth feature point as The center establishes an original matching block; determines a to-be-matched block in the micro-cell image adjacent to the micro-cell image in which the original matching block is located; and determines an original view of the m-th feature point according to the original matching block and the to-be-matched block a difference; determining, according to the original disparity value, a microcell image to be matched that is farthest from the microcell image where the mth feature point is located, and determining the microcell image of the to-be-matched microcell image and the original matching block The difference in the number of images between the two; according to the original matching block and the matching block in the image of the to-be-matched micro-unit with the smallest color error value of the original matching block, determining the matching disparity value of the m-th feature point; according to the following formula Calculating the exact disparity value d _{m of} the mth feature point:

Where D is the matching disparity value; n is the difference in the number of images.

With reference to the first aspect, or any one of the first to the eighth possible implementation manners of the first aspect, in the ninth possible implementation manner of the first aspect, the acquiring the multiple micro cells The image includes: acquiring, by using a light field camera, the plurality of microcell images; obtaining the three-dimensional image according to the planar depth value of the region, comprising: establishing a three-dimensional coordinate system, where the three-dimensional coordinate system includes an x-axis, a y-axis, and a z-axis; The following formula generates a three-dimensional image in the three-dimensional coordinate system:

Wherein P _j represents coordinate values of the j-th pixel point in the plurality of micro-cell images corresponding to the three-dimensional coordinate system, and C _j represents coordinates of the j-th pixel point corresponding to the center of the microlens in the plurality of micro-cell images a value, X _j represents a coordinate value of the j-th pixel in the plurality of micro-cell images, w _j represents a plane depth value of the region of the region plane where the j-th pixel is located, and i represents the plurality of micro- The distance from the plane of the unit image to the plane of the light field camera lens array, the j being less than or equal to the number of all the pixels in the plurality of microcell images.

In a second aspect, an apparatus for generating a three-dimensional image is provided, the apparatus comprising: an obtaining module, configured to acquire a plurality of micro-cell images; and a dividing module, configured to divide a plurality of feature regions on each of the micro-cell images, The difference between the color values of any two of the plurality of feature regions is less than or equal to the first threshold; the first determining module is configured to determine the plurality of region planes according to the plurality of feature regions, The feature area included in each area plane belongs to the same object or belongs to the same area, and each of the plurality of feature areas belongs to only the multiple area planes. And a second determining module, configured to determine an area plane depth value of each area plane; and a third determining module, configured to obtain a three-dimensional image according to the area plane depth value.

With reference to the second aspect, in a first possible implementation manner of the second aspect, the first determining module is specifically configured to: determine a first feature region of the plurality of feature regions and an adjacent region of the first feature region; Determining a first joint probability density that the first feature region and the adjacent feature region of the first feature region do not belong to the same object; determining that the first feature region and the adjacent feature region of the first feature region belong to a second association of the same object a probability density; when the ratio of the first joint probability density to the second joint probability density is less than or equal to the second threshold, determining that the first feature region and the adjacent feature region of the first feature region belong to the plurality of region planes The same area plane, the feature area included in the same area plane belongs to the same object.

With reference to the second aspect, in a second possible implementation manner of the second aspect, the first determining module is specifically configured to: determine a second feature region of the plurality of feature regions and an adjacent region of the second feature region; Determining a first likelihood ratio of the merged region and the second feature region, a second likelihood ratio of the merged region and the adjacent feature region of the second feature region, the merged region including the second feature region and the second feature a contiguous region of the region; when the first likelihood ratio and/or the second likelihood ratio is less than or equal to the third threshold, determining that the second feature region and the adjacent feature region of the second feature region belong to the plurality of regions The same area plane in the plane, the feature area included in the same area plane belongs to the same object.

With reference to the second aspect, in a third possible implementation manner of the second aspect, the first determining module is specifically configured to: determine a third feature region in the first microcell image of the plurality of feature regions; a fourth feature region of the second microcell image of the plurality of feature regions having the smallest color error value of the third feature region, the second microcell image being adjacent to the first microcell image, the fourth feature region And determining, by the third feature region, a color error value that is less than or equal to a fourth threshold; determining that the third feature region and the fourth feature region belong to the same region plane in the plurality of region planes, and the feature region included in the same region plane belongs to Homologous region.

With reference to the second aspect, in a fourth possible implementation manner of the second aspect, the first determining module is specifically configured to: determine a fifth feature region in the third microcell image of the plurality of feature regions, and the first a central pixel point of the five feature regions; in the fourth microcell image, a plurality of regions having the same size and shape as the fifth feature region are determined centering on a pixel point on the same pole line as the central pixel point a fourth microcell image is adjacent to the third microcell image; and a sixth feature region having a smallest color error value with the fifth feature region is determined in the plurality of regions, the sixth feature region and the sixth feature region The color error value of the five feature regions is less than or equal to the fifth threshold; determining that the fifth feature region and the sixth feature region belong to the same region plane in the plurality of region planes, and the feature region included in the same region plane belongs to the same region .

With reference to the second aspect, or any one of the first to the fourth possible implementation manners of the second aspect, in a fifth possible implementation manner of the second aspect, the acquiring module is specifically used to : acquiring a light field image by using a light field camera; mapping each pixel point in the light field image to a five-dimensional space to obtain a corresponding mapped pixel point, wherein the coordinates of the five-dimensional space include: horizontal X direction coordinate, vertical Y Direction coordinate, red component intensity value coordinate, green component intensity value coordinate and blue component intensity value coordinate; determining an average color value of the highest density region in the neighborhood of the mapped pixel point as a color value of the mapped pixel point; The mapped pixel of the value determines the plurality of microcell images.

With reference to the second aspect, or any one of the first to the fourth possible implementation manners of the second aspect, in a sixth possible implementation manner of the second aspect, the second determining module is specific And determining, by the at least one feature point in the plane of each area, determining a depth value of the at least one feature point, and determining an area plane depth value of the area plane, where the area plane depth value is the at least one feature point The average of the depth values.

With reference to the sixth possible implementation of the second aspect, in a seventh possible implementation manner of the second aspect, the acquiring module is specifically configured to: acquire the multiple micro unit images by using a light field camera; The determining module is specifically configured to: determine a center interval of the adjacent lens of the light field camera; determine a distance from a plane where the plurality of micro unit images are located to a plane of the light field camera lens array; and determine a disparity value of the mth feature point; The depth value w _m ' of the mth feature point is calculated according to the following formula:

Wherein, t for the light field camera lens center spacing of adjacent; for i from the plane where the image of the micro cell to the light-field camera lens plane of the array; disparity value D _m for the m-th feature point .

With reference to the seventh possible implementation of the second aspect, in the eighth possible implementation manner of the second aspect, the second determining module is specifically configured to: establish an original matching block by using the mth feature point; Determining a block to be matched in the microcell image adjacent to the microcell image in which the original matching block is located; determining an original disparity value of the mth feature point according to the original matching block and the to-be-matched block; Determining the image of the microcell to be matched that is farthest from the microcell image where the mth feature point is located, and determining the difference in the number of images between the image of the microcell to be matched and the microcell image where the original matching block is located a value; determining a matching disparity value of the mth feature point according to the original matching block and the matching block with the smallest color error value of the original matching block in the to-be-matched microcell image; and calculating the mth matching according to the following formula The exact disparity value of the block d _m :

With reference to the second aspect, or any one of the first to the eighth possible implementation manners of the second aspect, in the ninth possible implementation manner of the second aspect, the acquiring module is specifically used to Obtaining the plurality of microcell images by using a light field camera; the third determining module is specifically configured to: establish a three-dimensional coordinate system, where the three-dimensional coordinate system includes an x-axis, a y-axis, and a z-axis; and the three-dimensional coordinates are according to the following formula Generate a 3D image within the system:

Based on the foregoing technical solution, the method and apparatus for generating a three-dimensional image according to an embodiment of the present invention, by acquiring a plurality of micro-cell images, dividing the feature regions on the plurality of micro-cell images, combining the feature regions into the region planes, and calculating the depth values of the region planes The three-dimensional image is generated according to the depth value, which avoids the mismatch in the depth value extraction process, thereby more accurately extracting the depth value, thereby making the generated three-dimensional image more accurate and realistic, and the application scene range is wider.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the present invention, Those skilled in the art can also obtain other drawings based on these drawings without paying any creative work.

FIG. 1 is a schematic flow chart of a method of generating a three-dimensional image according to an embodiment of the present invention.

2 is a schematic diagram of a method of generating a three-dimensional image in accordance with an embodiment of the present invention.

3 is a schematic block diagram of an apparatus for generating a three-dimensional image in accordance with an embodiment of the present invention.

4 is another schematic block diagram of an apparatus for generating a three-dimensional image in accordance with an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

FIG. 1 shows a schematic flow diagram of a method 100 of generating a three-dimensional image, which may be performed by a terminal, in accordance with an embodiment of the present invention. As shown in FIG. 1, the method 100 includes:

S110. Acquire multiple micro cell images.

S120, dividing a plurality of feature regions on each of the microcell images, wherein a difference in color values of any two pixel points in each of the plurality of feature regions is less than or equal to a first threshold;

S130. Determine, according to the multiple feature regions, a plurality of region planes, where the feature regions included in each region plane belong to the same object or belong to the same region, and each of the plurality of feature regions belongs to the plurality of feature regions only One of the area planes;

S140. Determine an area plane depth value of each area plane.

S150. Obtain a three-dimensional image according to the planar depth value of the region.

Specifically, the two-dimensional micro-cell image may be acquired by using the light field camera, and the feature region is divided on the micro-cell image, and the pixel point in the feature region satisfies the difference between the color values of any two pixel points is less than or equal to the first A threshold, and each pixel on the microcell image belongs to one of the plurality of feature regions. The region plane is obtained by combining the feature regions, and the depth value in each region plane is calculated. The depth value is used as the depth value of all the pixels in the region plane, and a three-dimensional coordinate system is established, and each depth value is determined according to the depth value of each pixel point. The three-dimensional coordinate values of the pixels, thereby generating a three-dimensional image.

Therefore, the method for generating a three-dimensional image is performed by dividing a feature region on the acquired plurality of microcell images, combining the feature region into a region plane, calculating a depth value of the region plane, and generating a three-dimensional stereoscopic image according to the depth value, The mismatching in the depth value extraction process is avoided, so that the depth value can be extracted more accurately, thereby making the three-dimensional stereo image more accurate and realistic, and the application scene range is wider.

In S110, a plurality of microcell images can be obtained by a light field camera. Specifically, it can pass The light field camera directly captures a light field image, which is a micro cell image array composed of a plurality of micro cell images. In order to make the microcell image array more accurate when dividing the feature region, optionally, the light field image may be mapped by the mean shift method. Specifically, each pixel in the light field image is mapped to a five-dimensional space to obtain a corresponding mapped pixel, and the coordinates of the five-dimensional space include: horizontal X-direction coordinates, vertical Y-direction coordinates, and red component intensity values. Coordinate, green component intensity value coordinate and blue component intensity value coordinate; for each mapped pixel obtained by mapping, obtaining an average color value of a maximum density region within a neighborhood of each mapped pixel; using the color value as the mapped pixel The new color value of the point, and then the color value of the original pixel point is re-determined from the five-dimensional space to obtain a new micro-cell image array. Alternatively, the size of the mapped pixel point neighborhood may be determined based on empirical values.

In S120, a plurality of feature regions may be divided on each microcell image in the microcell image array composed of the acquired plurality of microcell images, such that the pixel points included in each feature region satisfy the color of any two pixel points. The difference in values is less than or equal to the first threshold. After each of the micro cell images divides the plurality of feature regions, the different feature regions do not overlap, and each pixel point belongs to only one of the plurality of feature regions. Optionally, the color value of the pixel may include an RGB (Red Red, Green Green, Blue Blue) value or an HSV (Hue Tone, Saturation Saturation, Value Brightness) value of the pixel, but the present invention is not limited thereto. .

Alternatively, the feature area can be divided by the flooding method. A pixel point that is not divided into a feature area or is not marked as a area plane is arbitrarily selected as a seed point on any one of the micro unit images, and the seed point is used as a new feature area. Gradually finding a pixel point that is smaller than or equal to a first threshold value in a contiguous set of the feature area, for example, the color value may be an RGB value, and the first threshold is a corresponding RGB threshold, in turn Calculating whether the difference between the seed point and the color value of the pixel point in the adjacent set meets less than or equal to the set threshold. Dividing a pixel point that satisfies a difference with a color value of the seed point less than or equal to the first threshold into the feature area, until there is no pixel point less than or equal to the first threshold in the adjacent set of the feature area, A seed point is set and another new feature area is divided, and the process is cycled to divide the microcell image into a plurality of feature areas. Optionally, the contiguous set of the feature regions may include 4 neighborhood pixels, and may also include 8 neighborhood pixels. Optionally, the first threshold may be set according to experience, or may be set according to image processing requirements, and the present invention is not limited thereto. The feature area divided by the flooding method is a continuous area, and the difference of the color values of any two pixel points belonging to the same feature area satisfies less than or equal to the first threshold.

Optionally, the feature region is further divided by the Kmeans algorithm, and the obtained feature region includes a discontinuous region, and each of the feature regions meets a difference between the color values of any two pixels in the same feature region is less than or equal to the first A threshold, the first threshold may be set according to an empirical value.

In the embodiment of the present invention, all the micro unit images may be divided into feature regions at the same time, and may be sequentially divided according to a certain direction, for example, in order from left to right and from top to bottom. The plurality of micro cell images are divided, and the present invention is not limited thereto.

In S130, a plurality of area planes are determined according to the plurality of divided feature regions. Optionally, the plurality of feature regions belonging to the same object may be combined to obtain a region plane by determining whether the plurality of feature regions belong to the same object; and the plurality of feature regions may belong to the same region, and may belong to the same region. Multiple feature regions are combined to obtain a region plane; it is also possible to determine whether multiple feature regions belong to the same object, combine feature regions belonging to the same object, and determine whether multiple feature regions belong to the same region, and belong to the same region. Multiple feature areas of the area are also merged to finally obtain the area plane. Optionally, the feature area may be merged, the area plane may be obtained by combining the feature areas on the position, and the feature area may be merged by marking the feature areas of different positions as the same area plane.

In the embodiment of the present invention, whether the plurality of feature regions belong to the same object can be determined by the following method. In particular, in a micro unit on either image, determines two adjacent feature region respectively adjacent region R of the first region R ₁ and a first characteristic feature region _2. Assuming that the regions in the image have constant gray values and are contaminated by independent, additive, and zero-mean Gaussian noise, the gray values follow a normal distribution. The calculated feature region R ₁ and the adjacent feature region R ₂ respectively include m ₁ and m ₂ pixel points, and have the following two assumptions:

H ₀ : two regions of R ₁ and R ₂ belong to the same object, in which case the gray values of both regions obey a single Gaussian distribution (μ ₀ , σ ₀ ² );

H ₁ : R ₁ and R ₂ do not belong to the same object, in which case the gray values of both regions obey different Gaussian distributions (μ ₁ , σ ₁ ² ) and (μ ₂ , σ ₂ ² ).

In general, the above parameters are unknown, but can be estimated using samples. For example, when the region contains n pixels, the gray value of each pixel is g _i , i=1, 2...n, subject to a normal distribution:

Therefore, the following parameters can be obtained by the following formulas (2) and (3):

That is, the correlation parameters σ ₀ , σ ₁ , σ ₂ in the embodiment of the present invention can be obtained by the formulas (2) and (3).

Therefore, in the case of H ₀ , the joint density

for:

In the case of H ₁ , the joint density

for:

The ratio of the combined densities of H ₁ and H _{0 is} calculated by the following formula (6), which is a likelihood ratio L:

When the L value is less than or equal to the second threshold, it is determined that the feature region R ₁ and the adjacent feature region R ₂ belong to the same object, and when the L value is greater than the second threshold, determining that the feature region R ₁ and the adjacent feature region R _{2 are} not Belong to the same object. Each feature region in each microcell image and the adjacent region of the feature region are judged and combined by the above method to obtain a plurality of region planes. Optionally, the second threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto.

Optionally, in the embodiment of the present invention, the feature region and the adjacent region of the feature region are combined into a merged region, and the feature is determined by calculating a likelihood ratio of the merged region and the adjacent region of the feature region and the feature region. Whether the area and the adjacent area of the feature area belong to the same object. Specifically, selecting any two adjacent feature regions in any one of the plurality of micro cell images is the second feature region R ₁ and the adjacent region R _{2 of the} second feature region, and R ₁ and R ₂ merged into a merged region R _3, according to the above equations (1) - (6) calculating the likelihood ratio L ₃₁ R ₃ and R ₁ are, respectively, R ₃ and R ₂ log likelihood ratio L _32, if L ₃₁ and / or If L _{32 is} less than or equal to the third threshold, it can be determined that R ₁ and R ₂ belong to the same object, otherwise, it can be determined that R ₁ and R ₂ do not belong to the same object. For example, when image processing is required to be relatively accurate, it may be set that when both L ₃₁ and L ₃₂ are less than or equal to the third threshold, it may be determined that R ₁ and R ₂ belong to the same object; otherwise, it may be determined that R ₁ and R _{2 are} not Belong to the same object; when image processing is required to be less precise, it can be set to be that when L ₃₁ or L _{32 is} less than or equal to the third threshold, then it can be determined that R ₁ and R ₂ belong to the same object; otherwise, R ₁ and R ₂ does not belong to the same object. Optionally, the third threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto.

In the embodiment of the present invention, the area plane may be determined by determining that multiple feature areas belong to the same area. Specifically, in any two adjacent micro cell image first micro cell image and second micro cell image, any one of the first micro cell images is selected as a third feature region, and the second micro cell image is selected. Each feature region on the top is regarded as a suspected homologous region. Calculate the color error value E of the third feature region and each suspected homologous region according to the following formula (7):

Where p is the number of pixels in the third feature region; I _p represents the color value of the pixel in the third feature region, and I _p+d represents the pixel point in the suspected homologous region and the third feature region The color value of the corresponding pixel; E represents the sum of the differences of the color values of all the pixels in the third feature region and the suspected homologous region. Selecting a suspected homologous region in which all the suspected homologous regions satisfying the color error value E is less than or equal to the fourth threshold is the fourth feature region, and the fourth feature region and the third feature region are homologous region. If there is no fourth feature area, the same area determined as the third feature area is the third feature area itself. Optionally, the fourth threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto. Each feature region in each microcell image and the adjacent microcell image is sequentially determined by the above method, and the feature regions belonging to the homologous region are combined to obtain a region plane. Alternatively, the color value of the pixel may include an RGB value or an HSV value of the pixel, but the invention is not limited thereto.

Optionally, in the embodiment of the present invention, whether the multiple feature regions belong to the homologous region may also be determined by the following method. Any one of the plurality of micro cell images is used as the fifth feature region, and the central pixel of the fifth feature region is determined. In the fifth feature area In the micro cell image adjacent to the micro cell image, each pixel point on the same pole line as the central pixel point is sequentially selected, and the same size and shape as the fifth feature region are established with these pixel points as a center. The region is a suspected homologous region. According to the formula (7), the color error value E between the fifth feature region and each of the plurality of suspected homologous regions is sequentially calculated. In the region where the color error value E is less than or equal to the fifth threshold, the region where the color error value E is the smallest is the sixth feature region, and the sixth feature region is the homologous region of the fifth feature region. If the sixth feature area does not exist, the same area determined as the fifth feature area is the fifth feature area itself. Optionally, the fifth threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto. Alternatively, the color value of the pixel may include an RGB value or an HSV value of the pixel, but the invention is not limited thereto.

In the embodiment of the present invention, when the feature regions are merged by the foregoing method, the feature regions may be divided according to a certain direction, and the feature regions are merged in a certain order to obtain a region plane. For example, the microcell image may be sequentially divided into a plurality of feature regions in order from left to right and from top to bottom, and the fifth feature region is determined in the microcell image in which the feature region is divided, and the adjacent regions are not divided. A sixth feature area is determined in the microcell image of the feature area.

In S140, after determining a plurality of area planes, the area plane depth values of the respective area planes are calculated. Optionally, the depth value of the area plane may be determined by determining feature points in each area plane and calculating a depth value of each feature point. Specifically, the feature points in the plane of the area may be determined by the SIFT feature point search method, and the feature points may also be determined by the FAST feature point search method, and the present invention is not limited thereto.

In the embodiment of the present invention, after all the feature points in the area plane are determined, the depth values of the respective feature points are calculated. Specifically, a plurality of microcell images can be obtained by a light field camera, the microcell images constitute a two-dimensional microcell image array, and the depth value w _m ' of the mth feature point is determined according to the following formula (8):

As shown in FIG. 2, where t is the adjacent lens center interval of the light field camera; i is the distance from the microcell image array plane to the light field camera lens array plane; d _m is the parallax value of the mth feature point, m Representing any one of all feature points, the depth value of each feature point on each area plane is calculated by the formula (8).

In the embodiment of the present invention, the disparity value d _m in the formula can be calculated by a block matching algorithm.

Alternatively, the disparity value d _m can also be calculated more accurately by the method of color error value matching. Specifically, taking the mth feature point as an example, the original matching block is established centering on the mth feature point, and the size of the matching block can be set according to an empirical value, and the mth feature is determined according to the block matching algorithm. The pixel to be matched on the adjacent microcell image of the microcell image where the point is located, and the original disparity value of the mth feature point is calculated. And determining, according to the calculated original disparity value, a matching microcell image of the matching block that is farthest from the microcell image where the mth feature point is located, and matching the original matching block, and determining the matching microcell image The difference n between the number of microcell images between the microcell image in which the original matching block is located. And determining, according to the block matching algorithm, a matching block that matches the original matching block in the matched microcell image, and calculating a matching disparity value D of the mth feature point according to the original matching block and the matching block by using a block matching algorithm. Calculate the accuracy disparity value d _m of the mth feature point according to the following formula (9):

Where d _m represents the disparity value of the mth feature point, and the mth feature point is any feature point in any area plane. Similarly, the disparity value of each feature point can be calculated according to the above method.

In the embodiment of the present invention, after obtaining the depth values of the feature points in the area plane by using the foregoing method, the depth values of all the feature points in the area plane may be averaged, and the average depth value is used as the plane in the area. The depth value of all the pixels, but the present invention is not limited thereto.

In the embodiment of the present invention, optionally, when there is no feature point in a certain area plane, the area plane may be ignored, and the depth value is not calculated, but the present invention is not limited thereto.

In S150, after determining the depth value of each area plane, a three-dimensional image is generated according to the depth value. First, a three-dimensional coordinate system is established. The three-dimensional coordinate system includes an x-axis, a y-axis, and a z-axis. Alternatively, the backward direction of the generated three-dimensional scene can be set to the positive direction of the z-axis, and the generated three-dimensional scene is oriented. The right direction is set to the positive x-axis direction, and the direction of the generated three-dimensional scene is set to the positive direction of the y-axis. Alternatively, the x and y directions may correspond to the horizontal and vertical directions of the two-dimensional microcell image array of the original plurality of microcell images. According to the established three-dimensional coordinate system, the coordinate value P _j of any pixel in any micro cell image in the three-dimensional coordinate system is calculated by using the following formula (10):

Wherein, P _j represents a coordinate value of a j-th pixel point in the micro-cell image array corresponding to the three-dimensional coordinate system, and C _j represents a coordinate value of the j-th pixel point in the micro-mirror image array corresponding to the center of the micro-lens, X _j represents the coordinate value of the j-th pixel point in the micro-cell image array, w _j represents the depth value of the area of the region where the j-th pixel point is located, and i represents the micro-cell image array plane to the light The distance of the field camera lens array plane, where j is less than or equal to the number of all pixel points in the microcell image array.

It should be understood that, in an embodiment of the present invention, an adjacent microcell image or an adjacent feature region of a feature region of the microcell image may be the microcell image or a micro neighborhood within the 4 neighborhood or 8 neighborhoods of the feature region. The unit image or the feature area, the present invention is not limited thereto.

It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be taken to the embodiments of the present invention. The implementation process constitutes any limitation.

Therefore, in the method for generating a three-dimensional image according to an embodiment of the present invention, a plurality of micro-cell images are acquired by a light field camera, and a feature region is divided for each micro-cell image, by determining whether each feature region belongs to the same object, and/or each feature region. Whether it belongs to a homologous region, and the feature regions belonging to the same object and/or belonging to the homologous region are region planes, and the average depth value of the feature points in each region plane is calculated, and the average depth value is used as all the pixels in the region plane. The depth value is used to generate a three-dimensional stereo image, which avoids mismatching in the depth value extraction process, thereby more accurately and quickly extracting the depth value, thereby making the generated three-dimensional image more accurate and realistic, and the application scene range is wider.

A method of generating a three-dimensional image according to an embodiment of the present invention is described in detail above with reference to FIGS. 1 through 2, and an apparatus for generating a three-dimensional image according to an embodiment of the present invention will be described below with reference to FIG.

FIG. 3 shows a schematic block diagram of an apparatus for generating a three-dimensional image in accordance with an embodiment of the present invention. As shown in Figure 3, the device comprises:

The obtaining module 210 is configured to acquire a plurality of micro unit images;

The dividing module 220 is configured to divide a plurality of feature regions on each of the micro cell images, and a difference of color values of any two pixel points in each of the plurality of feature regions is less than or equal to a first threshold ;

The first determining module 230 is configured to determine, according to the plurality of feature regions, a plurality of region planes, wherein the feature regions included in each of the region planes belong to the same object or belong to the same region, and each of the plurality of feature regions The feature area belongs to only one of the plurality of area planes;

a second determining module 240, configured to determine an area plane depth value of each area plane;

The third determining module 250 is configured to obtain a three-dimensional image according to the regional plane depth value.

Therefore, the method for generating a three-dimensional image according to an embodiment of the present invention passes through a plurality of acquired micro cells The feature region is divided on the image, the merged feature region is the region plane, the depth value of the region plane is calculated, and the three-dimensional stereo image is generated according to the depth value, thereby avoiding the mismatch in the depth value extraction process, thereby more accurately extracting the depth value, and further Make the 3D stereo image more accurate and realistic, and the application scene range is wider.

In the embodiment of the present invention, a plurality of micro cell images may be acquired by a light field camera to form a micro cell image array, and the acquiring module 210 acquires the micro cell image array, and the micro cell image array includes a plurality of micro cell images. Specifically, in order to make the micro-cell image array more accurate when dividing the feature region, the acquisition module 210 may perform mapping processing on the light field image by the mean shift method. Specifically, each pixel in the light field image is mapped to a five-dimensional space to obtain a corresponding mapped pixel, and the coordinates of the five-dimensional space include: horizontal X-direction coordinates, vertical Y-direction coordinates, and red component intensity values. Coordinate, green component intensity value coordinate and blue component intensity value coordinate; for each mapped pixel obtained by mapping, obtaining an average color value of a maximum density region within a neighborhood of each mapped pixel; using the color value as the mapped pixel The new color value of the point, and then the color value of the original pixel point is re-determined from the five-dimensional space to obtain a new micro-cell image array. Alternatively, the size of the mapped pixel point neighborhood may be determined based on empirical values.

In the embodiment of the present invention, each micro cell image of the plurality of micro cell images may be divided into a plurality of feature regions by the dividing module 220, so that the pixel points included in each feature region satisfy the color values of any two pixel points. The difference is less than or equal to the first threshold. After each of the micro cell images divides the plurality of feature regions, the different feature regions do not overlap, and each pixel point belongs to only one of the plurality of feature regions. Alternatively, the color value of the pixel may include an RGB value or an HSV value of the pixel, but the invention is not limited thereto.

Alternatively, the feature area can be divided by the flooding method. The feature area divided by the flooding method is a continuous area, and the difference of the color values of any two pixel points belonging to the same feature area satisfies less than or equal to the first threshold. Optionally, the feature region is further divided by the Kmeans algorithm, and the obtained feature region includes a discontinuous region, and each of the feature regions meets a difference between the color values of any two pixels in the same feature region is less than or equal to the first A threshold, the first threshold may be set according to an empirical value.

In the embodiment of the present invention, according to the plurality of feature regions divided by the dividing module 220, the first The determination module 230 determines a plurality of area planes. Optionally, the plurality of feature regions belonging to the same object may be combined to obtain a region plane by determining whether the plurality of feature regions belong to the same object; and the plurality of feature regions may belong to the same region, and may belong to the same region. Multiple feature regions are combined to obtain a region plane; it is also possible to determine whether multiple feature regions belong to the same object, combine feature regions belonging to the same object, and determine whether multiple feature regions belong to the same region, and belong to the same region. Multiple feature areas of the area are also merged to finally obtain the area plane. Optionally, the feature area may be merged, the area plane may be obtained by combining the feature areas on the position, and the feature area may be merged by marking the feature areas of different positions as the same area plane.

In the embodiment of the present invention, the first determining module 430 may determine whether the plurality of feature regions belong to the same object by using the following method. In particular, in a micro unit on either image, determines two adjacent feature region respectively adjacent region R of the first region R ₁ and a first characteristic feature region _2. Assuming that the regions in the image have constant gray values and are contaminated by independent, additive, and zero-mean Gaussian noise, the gray values follow a normal distribution. The calculated feature region R ₁ and the adjacent feature region R ₂ respectively include m ₁ and m ₂ pixel points, and have the following two assumptions:

In general, the above parameters are unknown, but can be estimated using samples. For example, when the region contains n pixels, the gray value of each pixel is g _i , i=1, 2...n, obeying the normal distribution as shown in formula (1), that is, by formula (2) And (3) the correlation parameters σ ₀ , σ ₁ , σ ₂ in the embodiment of the present invention can be obtained. Therefore, in the case of H ₀ , the joint density

As shown in formula (4); in the case of H ₁ , joint density

As shown in formula (5). The ratio of the combined densities of H ₁ and H _{0 is} calculated by the formula (6), and the ratio is the likelihood ratio L.

Optionally, in the embodiment of the present invention, the first determining module 430 may also be a merged area by combining the feature area and the adjacent area of the feature area, by calculating a similarity between the merged area and the adjacent area of the feature area and the feature area. However, it is determined whether the feature area and the adjacent area of the feature area belong to the same object. Specifically, selecting any two adjacent feature regions in any one of the plurality of micro cell images is the second feature region R ₁ and the adjacent region R _{2 of the} second feature region, and R ₁ and R ₂ merged into a merged region R _3, according to the above equations (1) - (6) calculating the likelihood ratio L ₃₁ R ₃ and R ₁ are, respectively, R ₃ and R ₂ log likelihood ratio L _32, if L ₃₁ and / or If L _{32 is} less than or equal to the third threshold, it may be determined that R ₁ and R ₂ belong to the same object. Otherwise, it may be determined that R ₁ and R ₂ do not belong to the same object. Optionally, the third threshold may be set according to an empirical value. It can also be set according to image processing requirements, and the present invention is not limited thereto.

In the embodiment of the present invention, the first determining module 430 may determine the area plane by determining that multiple feature areas belong to the same area. Specifically, in any two adjacent micro cell image first micro cell image and second micro cell image, any one of the first micro cell images is selected as a third feature region, and the second micro cell image is selected. Each feature region on the top is regarded as a suspected homologous region. Calculating the color error value E of the third feature region and each of the suspected homology regions according to formula (7), where p is the number of pixel points in the third feature region; _Ip represents the pixel point in the third feature region Color value, I _p+d represents the color value of the pixel corresponding to the pixel in the third feature region in the suspected homologous region; E represents the color value of all the pixels in the third feature region and the suspected homologous region The sum of the differences. Selecting a suspected homologous region in which all the suspected homologous regions satisfying the color error value E is less than or equal to the fourth threshold is the fourth feature region, and the fourth feature region and the third feature region are homologous region. If there is no fourth feature area, the same area determined as the third feature area is the third feature area itself. Optionally, the fourth threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto. Each feature region in each microcell image and the adjacent microcell image is sequentially determined by the above method, and the feature regions belonging to the homologous region are combined to obtain a region plane. Alternatively, the color value of the pixel may include an RGB value or an HSV value of the pixel, but the invention is not limited thereto.

Optionally, in the embodiment of the present invention, the first determining module 430 may further determine whether the multiple feature regions belong to the same region by using the following method. Any one of the plurality of micro cell images is used as the fifth feature region, and the central pixel of the fifth feature region is determined. In the microcell image adjacent to the microcell image in which the fifth feature region is located, each pixel point on the same pole line as the central pixel point is sequentially selected, and the pixel points are established as the center and the fifth pixel. A plurality of regions having the same size and shape as the feature region are suspected homologous regions. According to the formula (7), the color error value E between the fifth feature region and each of the plurality of suspected homologous regions is sequentially calculated. In the region where the color error value E is less than or equal to the fifth threshold, the region where the color error value E is the smallest is the sixth feature region, and the sixth feature region is the homologous region of the fifth feature region. If the sixth feature area does not exist, the same area determined as the fifth feature area is the fifth feature area itself. Optionally, the fifth threshold may be set according to an empirical value, or may be set according to an image processing requirement, and the present invention is not limited thereto. Alternatively, the color value of the pixel may include an RGB value or an HSV value of the pixel, but the invention is not limited thereto.

In the embodiment of the present invention, after the first determining module 230 determines the plurality of area planes, the second determining module 240 determines the area plane depth values of the respective area planes. Optionally, the determining module 240 may determine the depth value of the area plane by determining feature points in each area plane and calculating a depth value of each feature point. Specifically, the feature points in the plane of the area may be determined by the SIFT feature point search method, and the feature points may also be determined by the FAST feature point search method, and the present invention is not limited thereto.

In the embodiment of the present invention, after the second determining module 240 determines all the feature points in the area plane, the depth value of each feature point is calculated. Specifically, a plurality of microcell images can be obtained by a light field camera, and the microcell images form a two-dimensional microcell image array, and the depth value w _m ' of the mth feature point is determined according to formula (8), as shown in FIG. 2, where t is the adjacent lens center interval of the light field camera; i is the distance from the microcell image array plane to the light field camera lens array plane; d _m is the disparity value of each feature point, and m represents all feature points Any one of the feature points, the depth value of each feature point on each area plane is calculated by the formula (8).

In the embodiment of the present invention, the disparity value d _m in the formula can be calculated by a block matching algorithm. Alternatively, the disparity value d _m can also be calculated more accurately by the method of color error value matching. Specifically, taking the mth feature point as an example, the original matching block is established centering on the mth feature point, and the size of the matching block can be set according to an empirical value, and the mth feature is determined according to the block matching algorithm. The pixel to be matched on the adjacent microcell image of the microcell image where the point is located, and the original disparity value of the mth feature point is calculated. And determining, according to the calculated original disparity value, a matching microcell image of the matching block that is farthest from the microcell image where the mth feature point is located, and matching the original matching block, and determining the matching microcell image The difference n between the number of microcell images between the microcell image in which the original matching block is located. And determining, according to the block matching algorithm, a matching block that matches the original matching block in the matched microcell image, and calculating a matching disparity value D of the mth feature point according to the original matching block and the matching block by using a block matching algorithm. Calculating the accuracy disparity value d _m of the mth feature point according to the following formula (9), where d _m represents the disparity value of the mth feature point, and the mth feature point is any one of any area plane Feature points, similarly, the disparity value of each feature point can be calculated according to the above method. .

In the embodiment of the present invention, after the second determining module 240 obtains the depth values of the feature points in the area plane by using the foregoing method, the depth values of all the feature points in the area plane may be averaged, and the average depth value is obtained. The depth value is taken as the pixel value of all the pixels in the plane of the area, but the present invention is not limited thereto.

In the embodiment of the present invention, after the second determining module 240 determines the depth values of the respective area planes, the third determining module 250 generates a three-dimensional image according to the depth values. First, a three-dimensional coordinate system is established. The three-dimensional coordinate system includes an x-axis, a y-axis, and a z-axis. Alternatively, the backward direction of the generated three-dimensional scene can be set to the positive direction of the z-axis, and the generated three-dimensional scene is oriented. The right direction is set to the positive x-axis direction, and the direction of the generated three-dimensional scene is set to the positive direction of the y-axis. Alternatively, the x and y directions may correspond to the horizontal and vertical directions of the two-dimensional microcell image array of the original plurality of microcell images. Calculating the coordinate value P _j of any pixel in any micro cell image in a three-dimensional coordinate system according to the established three-dimensional coordinate system, wherein C _j represents the j-th pixel point in the a coordinate value corresponding to the center of the microlens in the microcell image array, X _j represents a coordinate value of the jth pixel point in the microcell image array, and w _j represents the depth of the plane of the region where the jth pixel point is located The value i represents the distance from the microcell image array plane to the plane of the light field camera lens array, where j is less than or equal to the number of all pixel points in the microcell image array.

It should be understood that the apparatus 200 for generating a three-dimensional image according to an embodiment of the present invention may correspond to the execution of the present invention. The method 100 for generating a three-dimensional image in an embodiment of the invention, and the above-described and other operations and/or functions of the respective modules in the apparatus 200 for generating a three-dimensional image are respectively implemented in order to implement the respective processes of the respective methods in FIGS. 1 to 2. , will not repeat them here.

Therefore, the apparatus for generating a three-dimensional image according to an embodiment of the present invention acquires a plurality of microcell images by a light field camera, and divides a feature region for each microcell image, by determining whether each feature region belongs to the same object, and/or each feature region. Whether it belongs to a homologous region, and the feature regions belonging to the same object and/or belonging to the homologous region are region planes, and the average depth value of the feature points in each region plane is calculated, and the average depth value is used as all the pixels in the region plane. The depth value is used to generate a three-dimensional stereo image, which avoids mismatching in the depth value extraction process, thereby more accurately and quickly extracting the depth value, thereby making the generated three-dimensional image more accurate and realistic, and the application scene range is wider.

As shown in FIG. 4, an embodiment of the present invention further provides an apparatus 300 for generating a three-dimensional image, including a processor 310, a memory 320, and a bus system 330. The processor 310 and the memory 320 are connected by a bus system 330 for storing instructions for executing instructions stored by the memory 320. The memory 320 stores program code, and the processor 310 can call the program code stored in the memory 320 to perform the following operations:

Obtaining multiple microcell images;

Dividing, on each of the micro cell images, a plurality of feature regions, wherein a difference in color values of any two pixel points in each of the plurality of feature regions is less than or equal to a first threshold;

Determining, according to the plurality of feature regions, a plurality of region planes, wherein the feature regions included in each of the region planes belong to the same object or belong to the same region, and each of the plurality of feature regions belongs to only the plurality of regions An area plane in the plane;

Determining an area plane depth value of each of the area planes;

A three-dimensional image is obtained based on the planar depth value of the region.

Therefore, the apparatus for generating a three-dimensional image according to the embodiment of the present invention divides the feature area on the acquired plurality of micro unit images, merges the feature area into the area plane, calculates the depth value of the area plane, and generates a three-dimensional stereoscopic image according to the depth value. The mismatching in the depth value extraction process is avoided, so that the depth value can be extracted more accurately, thereby making the three-dimensional stereo image more accurate and realistic, and the application scene range is wider.

It should be understood that, in the embodiment of the present invention, the processor 310 may be a central processing unit ("CPU"), and the processor 310 may also be other general-purpose processors, digital signal processors (DSPs). , application specific integrated circuit (ASIC), off-the-shelf programmable gate array (FPGA) Or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.

The memory 320 can include read only memory and random access memory and provides instructions and data to the processor 310. A portion of the memory 320 may also include a non-volatile random access memory. For example, the memory 320 can also store information of the device type.

The bus system 330 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. However, for clarity of description, various buses are labeled as bus system 330 in the figure.

In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 310 or an instruction in a form of software. The steps of the method disclosed in the embodiments of the present invention may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor. The software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like. The storage medium is located in the memory 320, and the processor 310 reads the information in the memory 320 and combines the hardware to perform the steps of the above method. To avoid repetition, it will not be described in detail here.

Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to: determine a first feature region of the plurality of feature regions and a contiguous region of the first feature region; determine the first Determining, by the feature area, the first joint probability density of the adjacent feature area of the first feature area not belonging to the same object; determining a second joint probability density of the first feature area and the adjacent feature area of the first feature area belonging to the same object; When the ratio of the first joint probability density to the second joint probability density is less than or equal to the second threshold, determining that the first feature region and the adjacent feature region of the first feature region belong to the same region of the plurality of region planes In the plane, the feature area included in the same area plane belongs to the same object.

Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to: determine a second feature region of the plurality of feature regions and a contiguous region of the second feature region; determine a merge region a first likelihood ratio of the second feature region, a second likelihood ratio of the merged region and the adjacent feature region of the second feature region, the merge region including adjacency of the second feature region and the second feature region a region; when the first likelihood ratio and/or the second likelihood ratio is less than or equal to a third threshold, determining that the second feature region and the adjacent feature region of the second feature region belong to the plurality of region planes The same area plane, the feature area included in the same area plane belongs to the same object.

Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to: determine a third feature region in the first microcell image of the plurality of feature regions; determine the plurality of features a fourth feature region of the second microcell image in the region having the smallest color error value of the third feature region, the second microcell image being adjacent to the first microcell image, the fourth feature region and the first The color error value of the three feature regions is less than or equal to the fourth threshold; determining that the third feature region and the fourth feature region belong to the same region plane in the plurality of region planes, and the feature region included in the same region plane belongs to the same region .

Optionally, as an embodiment, the processor 310 may invoke the program code stored in the memory 320 to: determine a fifth feature region and the fifth feature region in the third microcell image of the plurality of feature regions. a central pixel; in the fourth microcell image, a plurality of regions having the same size and shape as the fifth feature region are determined centering on a pixel point on the same pole line as the central pixel point, the fourth micro a unit image adjacent to the third microcell image; determining, in the plurality of regions, a sixth feature region having a smallest color error value with the fifth feature region, and a color error value of the sixth feature region and the fifth feature region And less than or equal to the fifth threshold; determining that the fifth feature region and the sixth feature region belong to the same region plane in the plurality of region planes, and the feature region included in the same region plane belongs to the same region.

Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to perform the following operations: acquiring a light field image by using a light field camera; mapping each pixel point in the light field image to one by one The dimension space obtains corresponding mapped pixel points, and the coordinates of the five-dimensional space include: horizontal X direction coordinates, vertical Y direction coordinates, red component intensity value coordinates, green component intensity value coordinates, and blue component intensity value coordinates; The average color value of the highest density region in the dot neighborhood is determined as the color value of the mapped pixel; the plurality of microcell images are determined according to the mapped pixel determined by the color value.

Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to: determine at least one feature point in each area plane; determine a depth value of the at least one feature point; determine the An area plane depth value of each area plane, the area plane depth value being an average of depth values of the at least one feature point.

Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to: acquire the plurality of micro unit images by using a light field camera; and determine a center interval of adjacent lenses of the light field camera; Determining a distance from a plane of the plurality of microcell images to a plane of the light field camera lens array; determining a disparity value of the mth feature point; and calculating a depth value w _m ' of the mth feature point according to the following formula:

Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to perform an operation of: establishing an original matching block centering on the mth feature point; determining a micro cell image where the original matching block is located a block to be matched in an adjacent microcell image; determining an original disparity value of the mth feature point according to the original matching block and the to-be-matched block; determining the mth feature point according to the original disparity value The microcell image is located at the farthest distance to match the microcell image, and determines the difference in the number of images between the image of the microcell to be matched and the microcell image where the original matching block is located; according to the original matching block and the to-be-matched a matching block having the smallest color error value of the original matching block in the micro unit image, determining a matching disparity value of the mth feature point; and calculating an accurate disparity value d _{m of the mth} feature point according to the following formula:

Optionally, as an embodiment, the processor 310 may call the program code stored in the memory 320 to: acquire the plurality of micro unit images by using a light field camera; and establish a three-dimensional coordinate system, where the three-dimensional coordinate system includes an x-axis , y-axis and z-axis; generate a three-dimensional image in the three-dimensional coordinate system according to the following formula:

It should be understood that the apparatus 300 for generating a three-dimensional image according to an embodiment of the present invention may correspond to the apparatus 200 for generating a three-dimensional image in the embodiment of the present invention, and may correspond to performing an embodiment according to the present invention. The above-described and other operations and/or functions of the respective modules in the method 300 for generating the three-dimensional image, respectively, in order to implement the respective processes of the respective methods in FIGS. 1 to 2, for the sake of brevity, Let me repeat.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of cells is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or integrated. Go to another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated in In a unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

An integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, can be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent modification or can be easily conceived by those skilled in the art within the technical scope of the present disclosure. Such modifications or substitutions are intended to be included within the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

A method for generating a three-dimensional image, comprising:

Obtaining multiple microcell images;

Dividing a plurality of feature regions on each of the microcell images, wherein a difference in color values of any two pixel points in each of the plurality of feature regions is less than or equal to a first threshold;

Determining, according to the plurality of feature regions, a plurality of region planes, wherein the feature regions included in each of the region planes belong to the same object or belong to a homologous region, and each of the plurality of feature regions belongs to only Describe one of a plurality of area planes;

Determining an area plane depth value of each of the area planes;

A three-dimensional image is obtained based on the area plane depth value.
The method according to claim 1, wherein the determining the plurality of area planes according to the plurality of feature areas comprises:

Determining a first feature region of the plurality of feature regions and an adjacent region of the first feature region;

Determining a first joint probability density that the first feature region and the adjacent feature region of the first feature region do not belong to the same object;

Determining, by the second feature probability density, that the first feature region and the adjacent feature region of the first feature region belong to the same object;

Determining that the first feature region and the adjacent feature region of the first feature region belong to the plurality of regions when a ratio of the first joint probability density to the second joint probability density is less than or equal to a second threshold The same area plane in the plane, the feature area included in the same area plane belongs to the same object.
The method according to claim 1, wherein the determining the plurality of area planes according to the plurality of feature areas comprises:

Determining a second feature region of the plurality of feature regions and an adjacent region of the second feature region;

Determining a first likelihood ratio of the merged region and the second feature region, a second likelihood ratio of the merged region and the adjacent feature region of the second feature region, the merged region including the second feature region And an adjacent area of the second feature area;

Determining that the adjacent feature regions of the second feature region and the second feature region belong to the plurality of regions when the first likelihood ratio and/or the second likelihood ratio is less than or equal to a third threshold Flat The same area plane in the face, the feature area included in the same area plane belongs to the same object.
The method according to claim 1, wherein the determining the plurality of area planes according to the plurality of feature areas comprises:

Determining a third feature region of the first of the plurality of feature regions;

Determining, in a second microcell image of the plurality of feature regions, a fourth feature region having a smallest color error value with the third feature region, the second microcell image being adjacent to the first microcell image The color error value of the fourth feature area and the third feature area is less than or equal to a fourth threshold;

Determining that the third feature region and the fourth feature region belong to the same region plane of the plurality of region planes, and the feature region included in the same region plane belongs to a homologous region.
The method according to claim 1, wherein the determining the plurality of area planes according to the plurality of feature areas comprises:

Determining a fifth feature region of the third plurality of feature regions and a central pixel point of the fifth feature region;

In the fourth microcell image, a plurality of regions having the same size and shape as the fifth feature region are determined centering on a pixel point on the same pole line as the central pixel point, the fourth microcell image Adjacent to the third microcell image;

Determining, in the plurality of regions, a sixth feature region having a smallest color error value with the fifth feature region, wherein a color error value of the sixth feature region and the fifth feature region is less than or equal to a fifth threshold;

Determining that the fifth feature region and the sixth feature region belong to the same region plane of the plurality of region planes, and the feature region included in the same region plane belongs to a homologous region.
The method according to any one of claims 1 to 5, wherein the acquiring a plurality of microcell images comprises:

Acquiring a light field image through a light field camera;

Mapping each pixel point in the light field image to a five-dimensional space to obtain corresponding mapping pixel points, wherein the coordinates of the five-dimensional space include: horizontal X direction coordinates, vertical Y direction coordinates, and red component intensity value coordinates , green component intensity value coordinates and blue component intensity value coordinates;

Determining an average color value of the highest density region in the neighborhood of the mapped pixel point as a color value of the mapped pixel point;

The plurality of microcell images are determined based on the mapped pixel points that determine the color value.
The method according to any one of claims 1 to 5, wherein the determining the regional plane depth value of each of the area planes comprises:

Determining at least one feature point in each of the area planes;

Determining a depth value of the at least one feature point;

Determining an area plane depth value of each of the area planes, the area plane depth value being an average of depth values of the at least one feature point.
The method according to claim 7, wherein the acquiring a plurality of microcell images comprises:

Acquiring the plurality of microcell images using a light field camera;

Determining the depth value of the at least one feature point includes:

Determining a center interval of adjacent lenses of the light field camera;

Determining a distance from a plane in which the plurality of microcell images are located to a plane of the light field camera lens array;

Determining a disparity value of the mth feature point;

The depth value w m ' of the mth feature point is calculated according to the following formula:

Where t is the center spacing of adjacent lenses of the light field camera; i is the distance from the plane of the plurality of microcell images to the plane of the light field camera lens array; d m is the mth The disparity value of the feature point.
The method according to claim 8, wherein the determining the disparity value of the mth feature point comprises:

Establishing an original matching block centering on the mth feature point;

Determining a block to be matched in a microcell image adjacent to the microcell image in which the original matching block is located;

Determining an original disparity value of the mth feature point according to the original matching block and the to-be-matched block;

Determining, according to the original disparity value, a to-be-matched microcell image that is farthest from the microcell image in which the mth feature point is located, and determining the microcell to be matched with the original matching block The difference in the number of images between images;

Determining a matching disparity value of the mth feature point according to the original matching block and the matching block in the to-be-matched microcell image that has the smallest color error value of the original matching block;

Calculating the exact disparity value d m of the mth feature point according to the following formula:

Where D is the matching disparity value; n is the difference in the number of images.
The method according to any one of claims 1 to 9, wherein the acquiring a plurality of microcell images comprises:

Acquiring the plurality of microcell images using a light field camera;

The generating a three-dimensional image according to the regional plane depth value includes:

Establishing a three-dimensional coordinate system including an x-axis, a y-axis, and a z-axis;

A three-dimensional image is generated within the three-dimensional coordinate system according to the following formula:

Wherein, P j represents a coordinate value of the j-th pixel point of the plurality of micro-cell images corresponding to the three-dimensional coordinate system, and C j represents that the j-th pixel point corresponds to the micro-cell image a coordinate value of a lens center, X j represents a coordinate value corresponding to the j-th pixel point in the plurality of micro-cell images, and w j represents a depth of the region plane of a region plane where the j-th pixel point is located a value, i represents a distance from a plane in which the plurality of microcell images are located to a plane of the light field camera lens array, the j being less than or equal to the number of all pixel points in the plurality of microcell images.
An apparatus for generating a three-dimensional image, comprising:

An acquiring module, configured to acquire a plurality of micro unit images;

a dividing module, configured to divide a plurality of feature regions on each of the microcell images, wherein a difference in color values of any two pixel points in each of the plurality of feature regions is less than or equal to the first Threshold value

a first determining module, configured to determine, according to the plurality of feature regions, a plurality of region planes, wherein the feature regions included in each of the region planes belong to the same object or belong to a homologous region, and the plurality of feature regions Each feature area belongs to only one of the plurality of area planes;

a second determining module, configured to determine an area plane depth value of each area plane;

And a third determining module, configured to obtain a three-dimensional image according to the regional plane depth value.
The device according to claim 11, wherein the first determining module is specifically configured to:

Determining a first feature region of the plurality of feature regions and an adjacent region of the first feature region;

Determining a first joint probability density that the first feature region and the adjacent feature region of the first feature region do not belong to the same object;

Determining, by the second feature probability density, that the first feature region and the adjacent feature region of the first feature region belong to the same object;

Determining that the first feature region and the adjacent feature region of the first feature region belong to the plurality of regions when a ratio of the first joint probability density to the second joint probability density is less than or equal to a second threshold The same area plane in the plane, the feature area included in the same area plane belongs to the same object.
The device according to claim 11, wherein the first determining module is specifically configured to:

Determining a second feature region of the plurality of feature regions and an adjacent region of the second feature region;

Determining a first likelihood ratio of the merged region and the second feature region, a second likelihood ratio of the merged region and the adjacent feature region of the second feature region, the merged region including the second feature region And an adjacent area of the second feature area;

Determining that the adjacent feature regions of the second feature region and the second feature region belong to the plurality of regions when the first likelihood ratio and/or the second likelihood ratio is less than or equal to a third threshold The same area plane in the plane, the feature area included in the same area plane belongs to the same object.
The device according to claim 11, wherein the first determining module is specifically configured to:

Determining a third feature region of the first of the plurality of feature regions;

Determining, in a second microcell image of the plurality of feature regions, a fourth feature region having a smallest color error value with the third feature region, the second microcell image being adjacent to the first microcell image The color error value of the fourth feature area and the third feature area is less than or equal to a fourth threshold;

Determining that the third feature region and the fourth feature region belong to the same region plane of the plurality of region planes, and the feature region included in the same region plane belongs to a homologous region.
The device according to claim 11, wherein the first determining module is specifically configured to:

Determining a fifth feature region of the third plurality of feature regions and a central pixel point of the fifth feature region;

In the fourth microcell image, a plurality of regions having the same size and shape as the fifth feature region are determined centering on a pixel point on the same pole line as the central pixel point, the fourth microcell image Adjacent to the third microcell image;

Determining, in the plurality of regions, a sixth feature region having a smallest color error value with the fifth feature region, wherein a color error value of the sixth feature region and the fifth feature region is less than or equal to a fifth threshold;

Determining that the fifth feature region and the sixth feature region belong to the same region plane of the plurality of region planes, and the feature region included in the same region plane belongs to a homologous region.
The device according to any one of claims 11 to 15, wherein the acquisition module is specifically configured to:

Acquiring a light field image through a light field camera;

Mapping each pixel point in the light field image to a five-dimensional space to obtain corresponding mapping pixel points, wherein the coordinates of the five-dimensional space include: horizontal X direction coordinates, vertical Y direction coordinates, and red component intensity value coordinates , green component intensity value coordinates and blue component intensity value coordinates;

Determining an average color value of the highest density region in the neighborhood of the mapped pixel point as a color value of the mapped pixel point;

The plurality of microcell images are determined based on the mapped pixel points that determine the color value.
The device according to any one of claims 11 to 15, wherein the second determining module is specifically configured to:

Determining at least one feature point in each of the area planes;

Determining a depth value of the at least one feature point;

Determining an area plane depth value of each of the area planes, the area plane depth value being an average of depth values of the at least one feature point.
The device according to claim 17, wherein the obtaining module is specifically configured to:

Acquiring the plurality of microcell images using a light field camera;

The second determining module is specifically configured to:

Determining a center interval of adjacent lenses of the light field camera;

Determining a distance from a plane in which the plurality of microcell images are located to a plane of the light field camera lens array;

Determining a disparity value of the mth feature point;

The depth value w m ' of the mth feature point is calculated according to the following formula:

Where t is the center spacing of adjacent lenses of the light field camera; i is the distance from the plane of the microcell image to the plane of the light field camera lens array; d m is the mth The disparity value of the feature point.
The device according to claim 18, wherein the second determining module is specifically configured to:

Establishing an original matching block centering on the mth feature point;

Determining a block to be matched in a microcell image adjacent to the microcell image in which the original matching block is located;

Determining an original disparity value of the mth feature point according to the original matching block and the to-be-matched block;

Determining, according to the original disparity value, a to-be-matched microcell image that is farthest from the microcell image in which the mth feature point is located, and determining the microcell to be matched with the original matching block The difference in the number of images between images;

Determining a matching disparity value of the mth feature point according to the original matching block and the matching block in the to-be-matched microcell image that has the smallest color error value of the original matching block;

Calculating the exact disparity value d m of the mth matching block according to the following formula:

Where D is the matching disparity value; n is the difference in the number of images.
The device according to any one of claims 11 to 19, wherein the acquisition module is specifically configured to:

Acquiring the plurality of microcell images using a light field camera;

The third determining module is specifically configured to:

Establishing a three-dimensional coordinate system including an x-axis, a y-axis, and a z-axis;

A three-dimensional image is generated within the three-dimensional coordinate system according to the following formula:

Wherein, P j represents a coordinate value of the j-th pixel point of the plurality of micro-cell images corresponding to the three-dimensional coordinate system, and C j represents that the j-th pixel point corresponds to the micro-cell image a coordinate value of a lens center, X j represents a corresponding coordinate value of the j-th pixel point in the plurality of micro-cell images, and w j represents a depth of the region plane of a region plane where the j-th pixel point is located a value, i represents a distance from a plane in which the plurality of microcell images are located to a plane of the light field camera lens array, the j being less than or equal to the number of all pixel points in the plurality of microcell images.