WO2004098204A1

WO2004098204A1 - Stereoscopic photographing method and apparatus

Info

Publication number: WO2004098204A1
Application number: PCT/JP2003/005617
Authority: WO
Inventors: Seijiro Tomita
Original assignee: Seijiro Tomita
Priority date: 2003-05-02
Filing date: 2003-05-02
Publication date: 2004-11-11
Also published as: AU2003231376A1; JPWO2004098204A1

Abstract

A stereoscopic photographing method and apparatus comprising at least two imaging units performing main scanning in the horizontal direction and subscanning in the vertical direction and photographing an identical object using these imaging units in order to output such an image as the horizontal axes of two imaging units are aligned at the scanning line level. Video information on a predetermined scanning line is compared between two videos photographed by two imaging units and the output image from the imaging unit is corrected based on the comparison results to be suitable for stereoscopic vision.

Description

Specification

3D image capturing method and device

The present invention relates to a three-dimensional image capturing method and apparatus, and particularly to a three-dimensional image capturing method that can easily output two images that can be correctly recognized in three dimensions from an image capturing device when capturing a three-dimensional image with two image capturing devices. And equipment.

Technical background.

When capturing a stereoscopic image, it is common practice to arrange two video cameras, which are imaging devices, at a predetermined distance from each other, and to capture two images with parallax.

In such a case, it is necessary that the optical axes of the two cameras face the same horizontal height position, and that the horizontal scanning of the imaging element is arranged at the same position with respect to the same horizontal axis. desirable.

Usually, the two cameras are fixed to the base as a single unit, so they are fixed to the horizontal base as a single unit.However, if the base for fixing the two cameras is tilted, people can view this image. It has a great sense of discomfort when viewed with both eyes. In other words, as shown in Fig. 8, let us consider a case where images are taken by two cameras 1L and 1R. Suppose that a person A is at a relatively short distance (for example, distance F = 5 m) as a subject in an image, and a mountain B is at a long distance (for example, several km away). In such a case, if both cameras 1L and 1R are slightly inclined, as shown in Fig. 4, the displacement of person A on the images 41L and 41R can be reduced to several tens of lines. However, the image of the distant mountain B is enlarged and the difference δ becomes several hundred lines. When a stereoscopic image is played back in such a state, the viewer can recognize the images AL and AR of the person as the same thing in their heads and see them stereoscopically, but see two images BL and BR of the same image for mountain B. It cannot be recognized as an image and will be perceived as a double image. In addition, as shown in Fig. 7, when an image of only the near view without a distant view is taken, a difference of several scanning lines occurs in the image of the person of interest, AL and AR, as shown in Fig. 5 in the image. However, the observer who is paying attention to this person A is greatly conscious and becomes an obstacle to comfortable stereoscopic vision.

As a method of measuring the inclination angle in such a stereoscopic image photographing apparatus, use of a level or the like can be considered. However, even with such a level, it is actually difficult to meet the horizontal standard with an accuracy of one scanning line.

It is also conceivable to irradiate the laser light horizontally and adjust the horizontal base axis of the camera based on the laser light, but this is not preferable because the laser light is photographed.

Therefore, an object of the present invention is to provide a stereoscopic image photographing method and apparatus capable of easily measuring the inclination of a camera in order to align the horizontal base axes of two cameras at a scanning line level.

Disclosure of the invention

The present invention described in claim 1 includes a three-dimensional image capturing apparatus that includes at least two image capturing apparatuses that perform main scanning in the horizontal direction and sub-scans in the vertical direction, and that captures the same object with these image capturing apparatuses. A stereoscopic video photographing method, comprising comparing video information of scanning lines defined by video captured by two imaging devices, and detecting the positions of the two imaging devices based on the comparison result. You. ADVANTAGE OF THE INVENTION According to this invention, the inclination of two imaging devices can be measured accurately and quickly by electronically processing the data of the scanning line obtained from an imaging device, without using a mechanical device such as a level.

According to a second aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, a scanning line to be compared is selected from a central portion of the image. ADVANTAGE OF THE INVENTION According to this invention, the inclination of an imaging device can be measured accurately based on the scanning line of the location with the largest amount of information.

According to a third aspect of the present invention, in the stereoscopic image capturing method according to the first aspect, an absolute value of a difference between scanning line information is calculated, and an inclination of the two cameras is measured. Things. According to the present invention, a simple operation And an accurate inclination can be obtained.

According to a fourth aspect of the present invention, there is provided the stereoscopic image capturing method according to the first aspect, wherein an added value of a plurality of scanning lines is used as video information of a scanning line of each imaging unit. is there. According to the present invention, accurate measurement can be performed by canceling noise generated in an image.

According to a fifth aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, a target portion to be compared is extracted by performing image processing on scanning line information of each imaging unit. Things. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion. According to a sixth aspect of the present invention, there is provided the stereoscopic video imaging method according to any one of the first to fifth aspects, wherein the imaging is performed using an imaging device that captures an area larger than a display range, and the image to be output The tilt is corrected by outputting the range of the image as an image captured in the specific range of the image sensor. According to the present invention, the inclination can be corrected electronically one operation line at a time precisely and quickly.

The present invention described in claim 7 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and a stereoscopic image obtained by photographing the same object with these imaging devices. Assuming that the photographing device is a technical premise, the tilt angle measuring device in the stereoscopic video photographing device that measures the inclination of the plurality of image capturing devices extracts a predetermined scanning line from the images photographed by the two photographing means. Scanning line extracting means, comparing means for comparing the image information of the extracted scanning lines, and inclination angle calculating means for measuring the inclination of the photographing device based on the comparison result. . ADVANTAGE OF THE INVENTION According to this invention, the data of the scanning line obtained from a camera can be processed electronically, without using a mechanical device, such as a level, and can measure the inclination angle of an imaging device accurately and quickly.

According to the present invention described in claim 8, in the body image photographing device according to claim 7, the scanning line extracting means selects a scanning line from a central portion of the image. It is characterized by the following. According to the present invention, it is possible to accurately measure the inclination of two cameras based on the scanning line at the location having the largest amount of information.

The present invention described in claim 9 is the stereoscopic image photographing device according to claim 7, wherein the scanning line comparing means calculates an absolute value of the difference between the scanning line information, and the inclination angle calculating means calculates the difference of the difference. The feature is to measure the tilt angle of two cameras on the basis of. According to the present invention, an accurate inclination value can be obtained by a simple calculation process.

According to a tenth aspect of the present invention, in the stereoscopic video imaging apparatus according to the seventh aspect, the scanning line extracting unit extracts a plurality of scanning lines, and the scanning line comparing unit extracts the sum value. It is characterized in that it is compared as image information of scanning lines of each photographing means. According to the present invention, accurate measurement can be performed by canceling out noise generated in an image.

According to the present invention described in claim 11, in the tilt angle measuring device in the stereoscopic video imaging device described in claim 7, the scanning line extracting means performs image processing on the scanning line information of each imaging means. It is characterized by selecting a target location to be compared by using. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.

According to a third aspect of the present invention, there is provided the stereoscopic image photographing device according to any one of the seventh to eleventh aspects, wherein each imaging unit captures an area larger than a display range. And an output image correction means for correcting an image taken in a specific range of the image sensor to be an output image. ADVANTAGE OF THE INVENTION According to this invention, the correction | amendment of inclination can be performed electronically precisely and promptly one operation line at a time.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a stereoscopic video photographing apparatus according to the present invention. FIG. 2 is a flowchart showing the stereoscopic image capturing method shown in FIG. is there.

FIG. 3 is a diagram illustrating an example of a rough used for determining a horizontal direction in the stereoscopic video imaging method illustrated in FIG.

FIG. 4 is a diagram illustrating an image captured by the stereoscopic video capturing device.

FIG. 5 is a diagram showing a correction state of a captured image of the stereoscopic video imaging device shown in FIG.

FIG. 6 is a schematic diagram showing an image when two cameras are kept horizontal.

FIG. 7 is a schematic diagram showing an image when two cameras are tilted from horizontal by an angle.

FIG. 8 is a diagram showing an arrangement relationship of a stereoscopic image photographing device to which the present invention is applied.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a stereoscopic video photographing apparatus according to the present invention, FIG. 2 is a flowchart showing a stereoscopic video photographing method shown in FIG. 1, and FIG. 3 is a horizontal direction of the stereoscopic video photographing apparatus shown in FIG. Fig. 4 is a diagram showing an example of a graph used to determine the image, Fig. 4 is a diagram showing an image including a distant view and a near view taken by the stereoscopic image photographing device, and Fig. 5 is a photographing including a distant view and a near view of the stereoscopic image photographing device shown in Fig. Diagram showing the image correction state, Fig. 6 is a schematic diagram showing the image when the two cameras are kept horizontal, and Fig. 7 shows the image when the two cameras are tilted from horizontal by an angle. FIG. 8 is a schematic diagram showing a positional relationship of a stereoscopic image photographing apparatus to which the present invention is applied.

In this example, as shown in FIG. 1, the tilt angle measuring device in the stereoscopic image photographing device is composed of scanning line extracting means 2 connected to left and right cameras 1 L and 1 R as photographing means, and scanning line comparing means 3. And inclination angle calculation means 4 and output image correction means.

First, based on Figs. 1 to 4 and Fig. 8, A description will be given of a stereoscopic image photographing apparatus suitable for the case.

The cameras 1L and 1R are fixed to the base, shoot the subject, output video with parallax, and output video data for displaying stereoscopic video (Fig. 4, Fig. 2, S1).

When the cameras 1L and 1R are horizontally on the same horizontal base axis, correct left and right images without vertical displacement are obtained, and the subject A and the mountain B are also at the same horizontal position. As shown in FIG. 4, if the imaging device is tilted, the positions of the subjects A and B are shifted vertically in the left and right images. In this case, the shift amount δ between the left and right images AL and AR of the person A in the foreground is small (for example, several tens of lines), and the shift amount BL and BR for the left and right images of the distant mountain Β is large (for example, several hundred lines). .

The scanning line extracting means 2 extracts the image extraction ranges 41 L and 41 R (for example, scanning line numbers 1 to N) (S 2). This is, for example, about 1/3 of the vertical dimension at the top of the image. If there is parallax between the images of both cameras 1L and 1R and there is a shift in the left and right direction, determine the extraction range by detecting the subject.

Next, the scanning line comparison means, for example, first focuses on the scanning line 42 L (the n-th scanning line from the top) in the scanning range 41 L, and sequentially outputs the luminance data of the scanning line in the horizontal direction. Acquire and store (these luminance data are referred to as An). Then, scan lines of the search range 41R are acquired from above, and their luminance data (Bm (m = l to N)) are acquired.

Next, the scanning line comparison means 3 sequentially calculates the absolute value of the difference between the luminance data (| A n -Bm I (m = l to N) (S 3). In this graph, the absolute value of the difference between the luminance signals is represented by a curve, and the minimum value is i An — B pi It can be seen that it can be obtained. It can be seen that the n-th scanning line of the left image 41 L coincides with the p-th scanning line of the right image 41 R. Note that this minimum value is not always 0 due to the effects of noise and the like.

The tilt angle calculating means 4 can calculate the angle 0 from the distance between the cameras 1L and 1R and the dimensions of the scanning line (S4). Then, the output image correcting means 5 corrects the output image (S5). In this correction, if a correction corresponding to is performed, a correction for this is performed (S4). This correction is larger than the actual output image 52 L, 52 R (for example, 640 horizontal pixels X 480 vertical pixels) as the image pickup devices 51 L and 51 R of the photographing device ( For example, by using 100 pixels horizontally and 600 pixels vertically, for example, the right output image 52R is moved or rotated up, down, left, and right.

Also, in order to perform the tilt detection and the correction at the same time, once the image is loaded into the memory, and the brightness of the output images 52 L and 52 R is compared for each line, for example, the right output image May be moved.

In this way, the camera tilt can be measured accurately and quickly. The horizontal position of the camera can be adjusted based on this value.

The scanning line extracting means extracts a plurality of scanning lines, for example, three lines, and the scanning line comparing means can use the added value as video information of the scanning lines of each camera. In this way, accurate measurements can be made by canceling out noise in the video.

In addition, the brightness is extracted using the same number of scanning lines, the camera brightness is changed while comparing the two brightness data, and at the point where the brightness difference becomes minimum, the force camera matches the horizontal reference. It can be.

Next, based on Figs. 2 and 3, and Figs. 6 to 8, A description will be given of a stereoscopic image photographing apparatus that is suitable for the case.

As in the previous example, the cameras 1L and 1R are fixed to the base, photograph the subject A as a subject, output an image with a parallax amount, and output image data for displaying a stereoscopic image (FIG. 2). , S 1).

When the cameras 1 L and 1 R are horizontally on the same horizontal base axis, correct right and left images 20 L and 2 OR are obtained as shown in FIG. 6, and the subject 30 is also at the same horizontal position. . As shown in FIG. 7, if the camera is tilted by 0 with respect to the horizontal base axis GL, the position of the person A shifts vertically in the left and right images. Each figure does not take into account the horizontal shift due to the parallax between the two cameras.

As shown in FIG. 7, the scanning line extracting means 2 extracts the image extraction ranges 11 L and 11 R (for example, scanning line numbers 1 to N) (S 2). This is, for example, about one-third the size of the top, bottom, left and right in the center of the video. If there is a parallax between the images of both cameras 1L and 1R and there is a deviation in the left and right direction, determine the extraction range by detecting the subject.

Next, as shown in FIG. 7, the scanning line comparison unit 3 focuses on the scanning line 12 L (the n-th scanning line from the top) in the scanning range 11 L, and calculates the luminance data of the scanning line. Are sequentially acquired in the horizontal direction and stored (these luminance data are referred to as An).

Next, the scanning line of the search range 11R is sequentially acquired by scanning the upper and lower n-th lines (Δn: for example, 5) from the top, and their luminance and luminance data (B m (πι = η ± Δη) )).

Next, the scanning line comparison means 3 sequentially calculates the absolute value of the difference between the luminance data (| A η -B m |) (S 3) This is graphed as shown in FIG. In this graph, the absolute value of the difference It is shown by the line, and it can be seen that the minimum value is obtained by IA n-B p I. Also, it can be seen that the n-th scanning line of the left image 11L and the p-th scanning line of the right image 11R coincide. Note that this minimum value is not always 0 due to the influence of noise and the like.

The tilt angle calculating means 4 calculates the angle 0 from the distance between the cameras 1 and 1R and the size of the scanning line. If the correction for the parallax has been performed, the correction for this is performed (S4).

Further, the scanning line extracting means extracts a plurality of, for example, three rows of scanning lines, and the scanning line comparing means 3 can use the added value as image information of the scanning line of each force lens. In this way, accurate measurements can be made by canceling the noise generated in the video.

In addition, the luminance is extracted using the same number of scanning lines, and the two luminance data are compared, and the inclination of the camera is changed. At the point where the luminance difference is minimized, it is determined that the camera matches the horizontal reference. can do.

In this case, a range in which no tilt occurs can be output by using a tilt driving device for the camera or using an image sensor larger than the actual image capturing range as in the above-described example.

Industrial availability

The present invention described in claim 1 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and captures the same object using these imaging devices. A three-dimensional video imaging method comprising: comparing video information of a predetermined scanning line of video captured by two imaging devices; and detecting a position of the two imaging devices based on the comparison result. is there. According to the present invention, scanning line data obtained from an imaging device can be processed electronically accurately and quickly without using a mechanical device such as a level. The inclination of two imaging devices can be measured.

According to a second aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, a scanning line to be compared is selected from a central portion of the image. ADVANTAGE OF THE INVENTION According to this invention, the inclination of an imaging device can be measured accurately based on the trajectory of the location with the largest amount of information.

According to a third aspect of the present invention, in the stereoscopic image capturing method according to the first aspect, an absolute value of a difference between scanning line information is calculated, and a tilt of two cameras is measured. It is. According to the present invention, an accurate inclination can be obtained by a simple calculation.

According to a fourth aspect of the present invention, there is provided the stereoscopic image photographing method according to the first aspect, wherein an added value of the plurality of scanning lines is used as image information of a scanning line of each imaging unit. It is. According to the present invention, accurate measurement can be performed by canceling noise generated in an image.

According to a fifth aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, a target portion to be compared is extracted by performing image processing on scanning line information of each imaging unit. Things. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.

According to a sixth aspect of the present invention, in the stereoscopic video imaging method according to any one of the first to fifth aspects, the imaging is performed using an imaging element that captures a range larger than a display range. The tilt is corrected by outputting a range of an image to be output as an image captured in a specific range of the image sensor. ADVANTAGE OF THE INVENTION According to this invention, correction | amendment of inclination can be performed electronically one operation line precisely and promptly.

The present invention described in claim 7 provides at least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction, and a stereoscopic image obtained by photographing the same object with these imaging devices. In the tilt angle measuring device in the stereoscopic video capturing device that measures the tilt of the plurality of image capturing devices, a predetermined scanning line is extracted from the video captured by the two capturing units. Stereoscopic image photographing comprising: scanning line extracting means; comparing means for comparing image information of the extracted scanning lines; and inclination angle calculating means for measuring the inclination of the photographing device based on the comparison result. Device. ADVANTAGE OF THE INVENTION According to this invention, the inclination angle of an imaging device can be measured accurately and quickly by electronically processing the data of the scanning line obtained from a camera, without using a mechanical device, such as a level.

The present invention described in claim 8 is characterized in that, in the stereoscopic image photographing device according to claim 7, the scanning line extracting means selects a scanning line from a central portion of the image. According to the present invention, it is possible to accurately measure the inclination of two cameras based on the scanning line at the location having the largest amount of information.

According to a tenth aspect of the present invention, in the stereoscopic video imaging apparatus according to the seventh aspect, the scanning line extracting unit extracts a plurality of scanning lines, and the scanning line comparing unit calculates the added value. The comparison is made as image information of the scanning lines of the photographing means. According to the present invention, accurate measurement can be performed by canceling noise generated in an image.

According to the present invention as set forth in claim 11, in the tilt angle measuring device in the stereoscopic imaging apparatus according to claim 7, the scanning line extracting means compares the scanning line information of each imaging means by performing image processing. It is characterized by selecting the target place to be. According to the present invention, even if an image is displaced due to parallax, the inclination can be reliably measured by extracting a target image portion and comparing the scanning lines of the portion.

According to a third aspect of the present invention, there is provided the stereoscopic image photographing device according to any one of the seventh to eleventh aspects, wherein each imaging unit captures an area larger than a display range. The range of the output image is An output image correcting means for correcting an image captured in a specific range of the image sensor to be an output image is provided. According to the present invention, inclination correction can be performed electronically precisely and quickly for each operation line.

Claims

Claims: At least two imaging devices that perform main scanning in the horizontal direction and sub-scanning in the vertical direction are provided, and these imaging devices photograph the same subject and output a stereoscopic image display image. A stereoscopic video imaging method, comprising: comparing video information of a predetermined scanning line among videos captured by two imaging devices; and detecting a tilt of the imaging device based on the comparison result. 3. The method of claim 1, wherein the scan line to be compared is selected from a central portion of each image. 2. The stereoscopic video imaging method according to claim 1, wherein an inclination of the imaging device is detected by calculating an absolute value of a difference between the pieces of scanning line information and obtaining a minimum value of the difference. 3. The stereoscopic video imaging method according to claim 1, wherein an added value of a plurality of scanning lines is used as video information of scanning lines of each imaging device. 3. The stereoscopic video imaging method according to claim 1, wherein a portion to be compared is extracted by performing video processing on the scanning line information of each imaging device. The imaging is performed using an image sensor that captures a range larger than the display range, and the inclination of the image is output by outputting the range of the image to be captured in a specific range of the image sensor. The stereoscopic video imaging method according to any one of claims 1 to 5. A stereoscopic image capturing device that has at least two image capturing devices that perform main scanning in the horizontal direction and sub-runs in the vertical direction, captures the same subject with these image capturing devices, and outputs images for displaying a stereoscopic image. Scanning line extracting means for extracting a predetermined scanning line of the video imaged by the two imaging devices; comparing means for comparing the video information of the extracted scanning lines; and determining the inclination of the imaging device based on the comparison result. A stereoscopic video photographing device comprising: a tilt angle calculating means for detecting. 8. The stereoscopic image photographing apparatus according to claim 7, wherein the scanning line extracting means selects a scanning line from a central portion of each image. The scanning line comparison means calculates the absolute value of the difference between the scanning line information, and the inclination angle calculation means obtains the minimum value of the difference to measure the inclination angles of the two imaging devices. 8. The stereoscopic image photographing device according to claim 7. 9. The stereoscopic image according to claim 7, wherein the scanning line extracting unit extracts a plurality of scanning lines, and the scanning line comparing unit compares the added values as image information of the scanning lines of each imaging device. Shooting method.

9. The stereoscopic video photographing apparatus according to claim 7, wherein the scanning line extracting means selects a target portion to be compared by performing video processing on the scanning line information of each imaging device.

2. Each imaging device includes an image sensor that captures an area larger than the display range, and an output image correction unit that corrects a range of an output image to an image captured in a specific range of the image sensor as an output image. The stereoscopic image photographing device according to any one of claims 7 to 11, characterized in that the device is provided.