JP4740481B2 - 3D image information generation apparatus, 3D image information generation method, and 3D image information generation program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional image information generation apparatus, a three-dimensional image information generation method, and a three-dimensional image information generation program.
[0002]
[Prior art]
Conventionally, when acquiring three-dimensional information based on a stereo image acquired by a stereo imaging system, it is recognized by corresponding point search which point on the other image corresponds to a point on one reference image. The distance information from the deviation (hereinafter referred to as “parallax”), the viewpoint positions of a plurality of imaging systems, and the distance from the line-of-sight direction to the corresponding point by the principle of triangulation is obtained. In this method, images from a plurality of viewpoints are acquired, and three-dimensional information is obtained from the difference in the positional relationship between the viewpoints and the appearance of the images.
[0003]
Here, the “stereo imaging system” means an imaging system from a plurality of viewpoints. Furthermore, the thing which acquires the image of a several viewpoint using a stereo converter etc. from one imaging device, and the thing using a some imaging device are included. The “stereo image obtained by the stereo imaging system” means an image from a plurality of viewpoints. This includes not only a plurality of images from a plurality of imaging devices but also a plurality of viewpoint images embedded in one image.
[0004]
For example, as shown in FIG. 8, stereo images 2-1 and 2-2 are acquired from a plurality of viewpoints (in this case, cameras 100 and 101), and one of them is used as a reference image 3 (here, stereo image 2). -1). Next, triangulation is performed based on the difference between the position of the point A on the subject 4 on the reference image 3 coordinate and the position on the stereo image 2-2 coordinate (hereinafter referred to as parallax), the viewpoint position, and the viewing direction. Based on this principle, the distance to the point A on the object 4 can be calculated.
[0005]
Conventionally, when three-dimensional information is acquired using a stereo image, it is recognized by corresponding point search which point on the image serving as one reference corresponds to which point on the other image. For example, JP-A-6-215111 discloses an example of such a method.
[0006]
FIG. 9 is a diagram for explaining a procedure for generating a three-dimensional image from an image obtained by the stereo imaging system.
[0007]
Acquire stereo images 2-1 and 2-2 captured by a stereo imaging system 1 composed of a plurality of cameras (two in this case) (at least one camera is a color camera for pasting textures). Store. Here, the stereo image 2-1 is set as the reference image 3 (step S1). Next, as pre-processing, in order to easily perform stereo matching, removal of lens distortion (lens distortion) and parallel projection (rectification) are performed to generate images 5-1 and 5-2 from which distortion is removed (Step S1). S2). Among these, the image from which the distortion of the reference image 3 is removed is referred to as a distortion removal reference image 6.
[0008]
Next, the color images 5-1 and 5-2 are converted into monochrome images 7-1 and 7-2, respectively (step S3). Next, stereo matching is performed on the monochrome images 7-1 and 7-2, and the distance image 7 is acquired and stored (step S4). Next, the distortion removal reference image 6 acquired in step S2-1 is pasted on the three-dimensional information of the distance image 7 as a texture (step S5).
[0009]
At this time, the processing time required for each process varies depending on the apparatus configuration, the lens distortion removal and rectification process (step S2), and the stereo matching (step S4) algorithms. According to the applicant's experiment, it has been found that the processing in step S2 and step S4 requires a considerable amount of time. In particular, when the processing in step S2 is performed on a color image, the processing time is three times that of a monochrome image. Cost. Therefore, in order to increase the speed of the system, the amount of calculation in step S2 is a very important issue.
[0010]
FIG. 10 is a diagram for explaining the rectification process. Rectification as used herein refers to rectification by projecting stereo images 2-1 and 2-2 obtained by imaging the subject 4 by the imaging system 1 including the cameras 100 and 101 to the same plane. To obtain the processed image 5.
[0011]
11A to 11D are diagrams for explaining lens distortion removal. Lens distortion removal is removal of image distortion that does not exist when a pinhole lens is assumed. First, when an original image (FIG. 11A) is picked up by a stereo image pickup system, images (FIGS. 11B and 11C) including distortion due to the lens of the stereo image pickup system are obtained. An original image is obtained by performing image deformation on the image by removing lens distortion (FIG. 11D).
[0012]
By performing these processes, the region where the corresponding points can exist when performing stereo matching is limited to a straight line (epipolar line) on the stereo image. This facilitates searching for corresponding points.
[0013]
There is a method of taking an image whose shape has an absolute value, determining parameters for image deformation (lens distortion removal or rectification), and deforming the image according to the parameters.
[0014]
[Problems to be solved by the invention]
However, image deformation processing such as rectification and lens distortion is an operation with a large amount of calculation, and has been an obstacle to high-speed processing.
[0015]
The present invention has been made paying attention to such problems, and the object of the present invention is to generate three-dimensional image information that can reduce the amount of calculation when performing stereo matching by reducing the number of image conversion processes. An apparatus, a three-dimensional image information generation method, and a three-dimensional image information generation program are provided.
[0016]
[Means for Solving the Problems]
To achieve the above object, the first invention obtains three-dimensional information of an imaging target based on an image obtained by imaging the imaging target, and generates three-dimensional image information of the imaging target. An apparatus including an input unit (stereo imaging system) that includes at least one color image and that captures an imaging target from a plurality of viewpoints, and an extraction unit that extracts luminance information of the color image (monochrome, Gray scale imaging means), pre-processing means for pre-processing (lens distortion processing, rectification processing) on the luminance information extracted by the extraction means, and three-dimensional information from the image processed by the pre-processing means. Three-dimensional information acquisition means (stereo matching means) to be acquired, and compensation processing means (reverse processing) for performing processing for compensating the effect of the preprocessing on the three-dimensional information. Distortion processing, reverse rectification processing, enlargement / reduction processing, parallel movement processing), and 3D image generation means for generating a 3D image by pasting the color image as a texture to the compensated 3D information It comprises.
[0017]
According to a second aspect of the present invention, in the three-dimensional image information generating apparatus according to the first aspect of the invention, the compensation processing means performs an inverse conversion process of the preprocessing.
[0018]
According to a third aspect of the present invention, in the three-dimensional image information generating apparatus according to the first aspect of the invention, the compensation processing means performs an enlargement / reduction process and / or a parallel movement process so as to cancel the preprocess result.
[0019]
In addition, the fourth invention is In a three-dimensional image information generation device having at least a stereo imaging system and a calculation device, A three-dimensional image information generation method for acquiring three-dimensional information of an imaging target based on an image obtained by imaging the imaging target and generating three-dimensional image information of the imaging target, In the stereo imaging system, An input step of inputting an image including at least one color image and imaging an imaging target from a plurality of viewpoints; In the computing device, An extraction step of extracting luminance information of the color image; In the computing device, A preprocessing step of performing preprocessing on the extracted luminance information; In the computing device, A three-dimensional information acquisition step of acquiring three-dimensional information from the image processed by the preprocessing; In the computing device, A compensation processing step for performing processing for compensating the effect of the preprocessing on the acquired three-dimensional information; In the computing device, A three-dimensional image generation step of generating a three-dimensional image by pasting the color image as a texture on the compensated three-dimensional information.
[0020]
According to a fifth aspect of the present invention, in the three-dimensional image information generation method according to the fourth aspect of the invention, the compensation process is an inverse conversion process of the preprocess.
[0021]
According to a sixth aspect of the present invention, in the three-dimensional image information generation method according to the fourth aspect of the invention, the compensation process is an enlargement / reduction process and / or a parallel movement process that cancels the preprocess result.
[0022]
The seventh invention is a three-dimensional image information generation program for obtaining three-dimensional information of an imaging target based on an image obtained by imaging the imaging target and generating the three-dimensional image information of the imaging target. The computer includes at least one color image, an input unit that inputs an image of the imaging target from a plurality of viewpoints, an extraction unit that extracts luminance information of the color image, and preprocesses the extracted luminance information Pre-processing means, said pre-processing means 3D information acquisition means for acquiring 3D information from the image processed by the above, compensation processing means for performing processing for compensating the effect of the preprocessing on the acquired 3D information, and the compensation processed 3D The color image is affixed to the information as a texture to function as a three-dimensional image generation unit that generates a three-dimensional image.
[0023]
The eighth invention is the three-dimensional image information generation program according to the seventh invention, wherein the compensation processing is performed. means Performs the inverse transformation process of the pre-process.
[0024]
According to a ninth aspect, in the three-dimensional image information generation program according to the seventh aspect, the compensation processing is performed. means Executes an enlargement / reduction process and / or a parallel movement process that cancels the preprocess result.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0026]
(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of a three-dimensional image information generating apparatus according to the present invention. A computing apparatus 12 having a storage device 10, a stereo imaging system 1 as input means connected thereto, and a display The apparatus 11 is comprised. The configuration of the stereo imaging system 1 is the same as that shown in FIG. 8, and the configuration shown in FIG. 10 is used for the rectification process, and the configuration shown in FIG. 11 is used for removing the lens distortion. 2 and 3 are flowcharts for explaining the procedure for generating the three-dimensional image information.
[0027]
The to-be-photographed object 4 is image | photographed with the stereo imaging system 1 comprised by the color camera 100 and the monochrome camera 101 shown in FIG. 8, and stereo image 2-1 and 2-2 ((A) of FIG. 3) are acquired and memorize | stored. Store in the device 10 (step S1). Here, a stereo image 2-1 photographed by the color camera 100 is set as a reference image 3. Next, the calculation device 12 converts the reference image 3 to monochrome and obtains a monochrome image 7 ((B) in FIG. 3) (step S3).
[0028]
Next, rectification processing and / or lens distortion processing is performed as preprocessing for the monochrome image 7 and the stereo image 2-2 (step S2 ′, FIG. 3C), and the stereo image 5-1 from which distortion has been removed. 5-2 is generated and stored in the storage device 10.
[0029]
Next, the calculation device 12 performs stereo matching processing on the stereo images 5-1 and 5-2 from which distortion is removed, and obtains a distance image 8 ((D) in FIG. 3) as three-dimensional information related to the subject 4. This is stored in the storage device 10 (step S4). Next, the calculation device 12 performs the inverse transformation process (compensation process) of the rectification and / or the lens distortion process performed in Step 2 ′ on the distance image 8 and performs the distance image 9 ((E) in FIG. 3). It is generated and stored in the storage device 10 (step S2 ″). The calculation device 12 pastes the reference image 3 acquired in step S1 as a texture on the three-dimensional information obtained from the distance image 9 (step S5). ((F) in FIG. 3).
[0030]
6A and 6B are diagrams showing the effects of the first embodiment as compared with the conventional example. In this example, a stereo imaging system including four color cameras and four monochrome cameras is assumed.
[0031]
6A and 6B, S2, S2 ′, S2 ″, S3, and S4 mean the steps shown in FIGS. 2 and 8. 3A is the time required to preprocess the color image, A Is the time required for preprocessing a monochrome image, B is the time required to convert a color image to a monochrome image, and C is the time required for stereo matching processing.
[0032]
The time required to execute steps S3 and S4 is the same between the present invention and the conventional example. Although not shown, the time required to execute steps S1 and S5 is the same between the present invention and the conventional example.
[0033]
On the other hand, in the case of four color cameras and four monochrome cameras, the time required to execute step S2 by the conventional method is 3A × 4 + A × 4 = 16A, but steps S2 ′ and S2 ″ are performed by the method of the present invention. The time required to execute is A × 4 + A × 4 + A = 9A.
[0034]
This will be generalized to x color cameras and y monochrome cameras, and the two will be compared.
[0035]
Number of color cameras: x
Number of monochrome cameras: y
Distortion removal processing time per monochrome image: A
Monochrome conversion processing time per color image: B
Stereo matching processing time: C
Then,
In the conventional example,
3Ax + Ay + Bx + C = A (3x + y) + Bx + C
In the case of the present invention
Bx + A (x + y) + C + A = A (x + y + 1) + Bx + C
And the difference is A (2x-1).
[0036]
As is apparent from this result, it can be seen that the difference between the present invention and the prior art increases as the number of color cameras increases and the distortion removal processing time increases.
[0037]
The values of A, B, and C vary greatly depending on the computing device and algorithm used. For example, in the experimental system (CPU: Pentium III-733 RAM: 256M) by the applicant,
4 color cameras (x = 4),
4 monochrome cameras (y = 4)
Monochrome image size (640 x 800 x 8)
Color image size (640 x 800 x 24)
Distortion removal processing time per monochrome image A = 2.5 seconds
Monochrome conversion processing time per color image B = 0.005 seconds
Stereo matching processing time C = 20 seconds
And
Conventional example
A (3x + y) + Bx + C = 2.5 × (12 + 4) + (0.005 × 4) + 20 = 60.02 seconds
In the present invention
A (x + y + 1) + Bx + C = 2.5 × (4 + 4 + 1) + (0.005 × 4) + 20 = 42.52 seconds
And 30% faster.
[0038]
In addition, naturally each deformation | transformation of embodiment of this invention can be variously modified and changed. For example, the number of cameras may be arbitrary as described above, and the texture may be synthesized not only from the reference image 3 but also from a plurality of other stereo images 2. Further, the stereo imaging system 1 and the computing device 12 do not need to be connected online, and may send images offline. Further, the distance image or the like generated in the middle of the process may be present as data, and is not necessarily present as an image.
[0039]
(Second Embodiment)
A second embodiment of the present invention will be described below with reference to FIGS. The configuration of the three-dimensional image information generation apparatus according to the second embodiment is shown in FIG. 1 and FIGS. 8, 10, and 11 as in the first embodiment.
[0040]
The object 4 photographed by the stereo imaging system 1 configured by the color camera 100 and the monochrome camera 101 shown in FIG. 8 is photographed to obtain stereo images 2-1 and 2-2 ((A) in FIG. 5). And stored in the storage device 10 (step S1). Here, a stereo image 2-1 photographed by the color camera 100 is set as a reference image 3. Next, the calculation device 9 converts the reference image 3 to monochrome and obtains a monochrome image 7 ((B) in FIG. 5) (step S3).
[0041]
Next, rectification processing and / or lens distortion processing is performed as preprocessing for the monochrome image 7 and the stereo image 2-2 (step S2 ′, FIG. 5C), and the stereo image 5-1, from which distortion has been removed, 5-2 is generated and stored in the storage device 10.
[0042]
Next, the calculation device 12 performs a stereo matching process on the stereo images 5-1 and 5-2, generates a distance image 8 ((D) in FIG. 5) as three-dimensional information related to the object 4 and stores it in the storage device 10. Store (step S4). Next, the calculation device 12 performs enlargement / reduction and parallel movement processing (compensation processing) on the distance image 8 with values stored in advance to generate a distance image 9 ((E) in FIG. 5) and store it in the storage device 10. Store (step S6). Next, the calculation device 9 pastes the reference image 3 acquired in step S1 as a texture on the three-dimensional shape information obtained from the distance image 9 (step S5) and displays the image ((F) in FIG. 5) on the display device 11. To do.
[0043]
Hereinafter, a method for determining the enlargement ratio and the amount of translation used in the enlargement / reduction and translation processing described in step S6 will be described. Normally, the lens distortion (distortion) increases as the distance from the optical axis increases. Distortion removal processing is indispensable for stereo matching even for images taken with a lens with little distortion, but it can be used practically without distortion removal processing for human viewing as an image. Yes (for home video shooting and viewing).
[0044]
Based on this idea, in the present embodiment, a method for reducing the amount of calculation by omitting distortion removal for a texture image to be pasted on already acquired three-dimensional information will be described with reference to FIG.
[0045]
1. An object having a known shape (a board or the like on which a certain pattern is described is preferable) is imaged in advance, and a distortion amount on the screen is acquired from the relationship between the image (with distortion) 13 and the known shape.
[0046]
2. A certain allowable amount e1 is set, and a region 15 having a distortion amount equal to or smaller than the allowable amount e1 is set as a processing target. In this case, a rectangular area is extracted.
[0047]
3. An image 14 obtained by removing distortion from the image 13 is prepared.
[0048]
4). A search is performed for an enlargement ratio and a translation amount of the image 13 in which the difference between the region 15 of the image 13 with distortion and the region 15 of the image 14 with distortion removed is minimized.
[0049]
The processing shown in FIGS. 4 and 5 is performed on the region 15 using the searched enlargement ratio and parallel movement amount.
[0050]
According to such a method, it is not necessary to remove complicated distortion. Depending on the size of the lens distortion and the value of the allowable amount e1, it is possible to perform the pasting without performing processing (magnification rate 1, translation 0). However, this method does not provide an accurate image, and it cannot be applied to an image taken away from the center of the image (in most cases, the optical axis).
[0051]
In addition, naturally each deformation | transformation of embodiment of this invention can be variously modified and changed.
[0052]
For example, the number of cameras may be arbitrary, and the texture may be synthesized not only from the reference image 3 but also from a plurality of other stereo images 2. Further, the stereo imaging system 1 and the computing device 12 do not need to be connected online, and may send images offline. Further, the distance image or the like generated in the middle of the process may be present as data, and is not necessarily present as an image.
[0053]
The processing time in the second embodiment described above is calculated as follows.
[0054]
Number of color cameras: x
Number of monochrome cameras: y
Distortion removal processing time per monochrome image: A
Monochrome conversion processing time per color image: B
Stereo matching processing time: C
Reduction and translation processing time per image: D
Then,
Bx + A (x + y) + C + D
Can be obtained. Here, the processing times of A and D are in the same order.
[0055]
(Appendix)
1. For images from multiple viewpoints obtained by a stereo imaging system
In a method of acquiring stereoscopic information of an object to be photographed by performing stereo matching,
After reducing the amount of information each image has, perform conversion processing on each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A reverse conversion process is performed on the acquired distance information with respect to the conversion process.
A method for generating three-dimensional image information from a stereo image, characterized by being applied to an original image.
[0056]
(Corresponding Embodiment of the Invention)
The first and second embodiments described above correspond to the embodiments relating to the present invention.
[0057]
(Function)
Of the plurality of images 2-1 and 2-2 of the subject 4 obtained by the stereo imaging system 1, one of the images is set as a reference image 3. (For example, left image if left and right stereo image) (FIG. 8)
Information amount reduction (monochrome color image, filter processing, etc.) is performed on the plurality of images 2-1 and 2-2. Thereafter, a plurality of images 5-1 and 5-2 are generated by transforming the images 2-1 and 2-2 with a reduced amount of information into a shape suitable for stereo matching. Stereo matching processing is performed on the images 5-1 and 5-2 to acquire distance information for the object to be photographed 4, and a distance image 8 is generated.
[0058]
A distance image 9 is generated by performing an inverse conversion process on the distance image 8 with respect to the conversion process.
[0059]
The distance image 9 is set as three-dimensional information, and the image 2-1 (reference image 3) is pasted as a texture (FIGS. 3 and 5).
[0060]
(effect)
It is not necessary to transform the image 2 into a shape suitable for performing stereo matching on the image 2 before the information amount is reduced, and the amount of calculation required because the image is deformed after the information amount is reduced. Is reduced.
[0061]
2. In a method for obtaining stereoscopic information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system including a color imaging system,
After the image captured by the color imaging system is converted to a gray image, a conversion process for correcting image distortion is performed on each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A reverse conversion process is performed on the acquired distance information with respect to the conversion process.
A method for generating three-dimensional image information from a stereo image, which is used as distance information for an original color image.
[0062]
(Corresponding Embodiment of the Invention)
The first embodiment corresponds to the embodiment relating to the present invention.
[0063]
(Function)
Among a plurality of images 2-1 and 2-2 (at least one is a color image) of the subject 4 obtained by the stereo imaging system 1, one color image (here, 2-1) among them. Is a reference image 3. (For example, left image if left and right stereo image) (FIG. 8)
Of the plurality of images 2-1 and 2-2, the color image 2-1 is converted into a monochrome image and converted to a monochrome image 7.
[0064]
Rectification and / or lens distortion processing is performed on the monochrome images 7 and 2-2 to generate a plurality of images 5-1 and 5-2. Stereo matching processing is performed on the plurality of images 5-1 and 5-2, distance information for the object 4 is acquired, and a distance image 8 is generated.
[0065]
A distance image 9 is generated by performing an inverse conversion process on the distance image 8 with respect to the conversion process (rectification and lens distortion).
[0066]
The distance image 9 is set as three-dimensional information, and the image 2-1 (reference image 3) having color information is pasted as a texture. (Figure 2)
(effect)
Since there is no need to perform rectification and / or lens distortion processing on the color image, the amount of calculation required is reduced.
[0067]
3. In a method for obtaining stereoscopic information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system including a color imaging system,
After converting images captured by the color imaging system into gray images, each image is subjected to conversion processing to correct image distortion,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
Perform scale conversion and / or translation on the acquired distance information.
A three-dimensional image information generation method characterized by being used as distance information with respect to an original color image.
[0068]
(Corresponding Embodiment of the Invention)
The embodiment relating to this invention corresponds to the second embodiment.
[0069]
(Function)
Among a plurality of images 2-1 and 2-2 (at least one is a color image) of the subject 4 obtained by the stereo imaging system 1, one (here, 2-1) color image among them. Is a reference image 3. (For example, left image if left and right stereo image) (FIG. 8)
Of the plurality of images 2-1 and 2-2, the color image 2-1 is converted into a monochrome image and converted to a monochrome image 7.
[0070]
Rectification and / or lens distortion processing is performed on the monochrome images 7 and 2-2 to generate a plurality of images 5-1 and 5-2.
[0071]
Stereo matching processing is performed on the plurality of images 5-1 and 5-2, distance information for the object 4 is acquired, and a distance image 8 is generated.
[0072]
The distance image 9 is generated by performing scale conversion and / or parallel movement processing on the distance image 8.
[0073]
The distance image 9 is set as three-dimensional information, and the image 2-1 (reference image 3) having color information is pasted as a texture.
[0074]
(effect)
Since there is no need to perform rectification and / or lens distortion processing on the color image, the amount of calculation required is reduced.
[0075]
4). For images from multiple viewpoints obtained by a stereo imaging system
In a device that acquires stereo information of an object by performing stereo matching,
After reducing the amount of information each image has, perform conversion processing on each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A three-dimensional image information generation device characterized in that the acquired distance information is converted and applied to the original image.
[0076]
5.
In an apparatus for acquiring three-dimensional information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system including a color imaging system,
After the image captured by the color imaging system is converted to a gray image, a conversion process for correcting image distortion is performed on each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A reverse conversion process is performed on the acquired distance information with respect to the conversion process.
A three-dimensional image information generation apparatus characterized by being used as distance information with respect to an original color image.
[0077]
6). At least one of the images from multiple viewpoints obtained by a stereo imaging system including a color imaging system
In a device that acquires stereo information of an object by performing stereo matching,
After converting the image captured by the color imaging system to a gray image,
Perform a conversion process to correct image distortion for each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
Perform scale conversion and / or translation on the acquired distance information.
A three-dimensional image information generation apparatus characterized by being used as distance information with respect to an original color image.
[0078]
7). In a program for acquiring three-dimensional information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system,
After reducing the amount of information each image has, perform conversion processing on each image,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A three-dimensional image information generation program characterized in that the acquired distance information is converted and applied to the original image.
[0079]
8). In a program for obtaining stereoscopic information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system including a color imaging system,
After converting images captured by the color imaging system into gray images, each image is subjected to conversion processing to correct image distortion,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
A reverse conversion process is performed on the acquired distance information with respect to the conversion process.
A three-dimensional image information generation program characterized by being used as distance information for an original color image.
[0080]
9. In a program for obtaining stereoscopic information of a subject by performing stereo matching on images from a plurality of viewpoints obtained by a stereo imaging system including a color imaging system,
After converting images captured by the color imaging system into gray images, each image is subjected to conversion processing to correct image distortion,
After that, distance information is acquired by performing stereo matching processing for multiple images after conversion,
Perform scale conversion and / or translation on the acquired distance information.
A three-dimensional image information generation program characterized by being used as distance information for an original color image.
[0081]
【The invention's effect】
According to the present invention, it is possible to reduce the amount of calculation when performing stereo matching by reducing the number of image conversion processes.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a three-dimensional image information generation apparatus of the present invention.
FIG. 2 is a flowchart for explaining a procedure of generating three-dimensional image information in the first embodiment of the present invention.
FIG. 3 is a diagram for describing a procedure for generating three-dimensional image information according to the first embodiment of the present invention.
FIG. 4 is a flowchart for explaining a procedure of generating three-dimensional image information according to the second embodiment of the present invention.
FIG. 5 is a diagram for explaining a procedure for generating three-dimensional image information in a second embodiment of the present invention.
FIG. 6 is a diagram showing the effect of the first embodiment of the present invention in comparison with a conventional example.
FIG. 7 is a diagram for explaining a method of reducing a calculation amount.
FIG. 8 is a diagram illustrating a configuration example of a stereo imaging system.
FIG. 9 is a diagram for explaining a procedure for generating a three-dimensional image from an image obtained by a stereo imaging system.
FIG. 10 is a diagram for explaining rectification processing;
FIG. 11 is a diagram for explaining lens distortion removal;
[Explanation of symbols]
1 Stereo imaging system
2-1, 2-2 Stereo image
3 reference images
4 Subject
10 Storage device
11 Display device
12 Calculator
13 Image with distortion
14 Distortion-removed image
15 areas

Claims (9)

A three-dimensional image information generation device that acquires three-dimensional information of an imaging target based on an image obtained by imaging the imaging target and generates three-dimensional image information of the imaging target,
An input unit that includes at least one color image and inputs an image of an imaging target from a plurality of viewpoints;
Extracting means for extracting luminance information of the color image;
Pre-processing means for pre-processing the luminance information extracted by the extraction means;
Three-dimensional information acquisition means for acquiring three-dimensional information from the image processed by the preprocessing means;
Compensation processing means for performing processing for compensating the effect of the preprocessing on the three-dimensional information;
3D image generation means for generating a 3D image by pasting the color image as a texture on the compensated 3D information;
A three-dimensional image information generating apparatus comprising:   The three-dimensional image information generation apparatus according to claim 1, wherein the compensation processing unit performs an inverse transformation process of the preprocessing.   The three-dimensional image information generation apparatus according to claim 1, wherein the compensation processing unit performs enlargement / reduction processing and / or parallel movement processing so as to cancel the preprocessing result. A three-dimensional image information generation device having at least a stereo imaging system and a calculation device, acquires three-dimensional information of an imaging target based on an image obtained by imaging the imaging target, and generates three-dimensional image information of the imaging target An information generation method,
An input step of inputting an image including at least one color image in the stereo imaging system and imaging an imaging target from a plurality of viewpoints;
An extraction step of extracting luminance information of the color image in the computing device ;
A preprocessing step of performing preprocessing on the extracted luminance information in the computing device ;
A three-dimensional information acquisition step of acquiring three-dimensional information from the image processed by the preprocessing in the computing device ;
A compensation processing step for performing processing for compensating the effect of the preprocessing on the acquired three-dimensional information in the calculation device ;
A three-dimensional image generation step of generating a three-dimensional image by pasting the color image as a texture on the compensated three-dimensional information in the calculation device ;
A three-dimensional image information generating method comprising:   The three-dimensional image information generation method according to claim 4, wherein the compensation process is an inverse transform process of the preprocess.   5. The three-dimensional image information generation method according to claim 4, wherein the compensation process is an enlargement / reduction process and / or a parallel movement process that cancels the preprocess result. In order to acquire the three-dimensional information of the imaging target based on the image obtained by imaging the imaging target and generate the three-dimensional image information of the imaging target,
An input unit that includes at least one color image and inputs an image of an imaging target from a plurality of viewpoints;
Extraction means for extracting luminance information of the color image;
Preprocessing means for performing preprocessing on the extracted luminance information;
Three-dimensional information acquisition means for acquiring three-dimensional information from the processed image by the preprocessing means,
Compensation processing means for performing processing for compensating the effect of the preprocessing on the acquired three-dimensional information; and
3D image generation means for generating a 3D image by pasting the color image as a texture on the compensated 3D information;
Program for generating three-dimensional image information. The three-dimensional image information generation program according to claim 7, wherein the compensation processing unit executes an inverse transformation process of the preprocessing. 8. The three-dimensional image information generation program according to claim 7, wherein the compensation processing means executes enlargement / reduction processing and / or parallel movement processing for canceling the preprocessing result.

Images