JP2001174220A - Device and method for generating image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain the simplification of a device constitution and the efficiency of a work, at the time of generating a parallax picture from a photographic image. SOLUTION: Subjects 11 and 12 are photographed by one camera 13 via lens array 38, in which plural lens 35-37 are arrayed, and the photographic image is divided into areas corresponding to each lens 35-37 so that plural parallax images can be generated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image generating apparatus and an image generating method, and more particularly to a parallax image generating apparatus and a parallax image generating method for generating a parallax image to be input to a device for measuring the depth of an object.

[0002]

2. Description of the Related Art Conventionally, a distance (depth) from a camera to a subject is measured by measuring a difference in a position in the image where the subject is photographed using a plurality of images photographed from different positions. There is a depth measuring device.

A plurality of images (referred to as parallax images) input to the depth measuring device have been generated by arranging and photographing a plurality of cameras.

Here, a conventional method of photographing a subject will be described. The apparatus shown in FIG. 7 is called a stereo camera, and includes a left camera 93 and a right camera 94.
It is assumed that the optical axes 95 and 96 of the cameras 93 and 94 are parallel.

FIGS. 8 (a) to 8 (c) show cameras 93 and 94.
3 is a photographed image of a subject photographed by.

FIG. 8A shows an image 81 of the objects P91 and Q92 taken by the left camera 93.

FIG. 8B shows an image 82 of the objects P91 and Q92 taken by the right camera 94.

FIG. 8C shows the images 81 and 82 superimposed and used to measure the distance (depth) from the camera to the subject.

In FIG. 7, a distance 83 based on the difference between the positions where the object P91 is photographed by the left and right cameras 93 and 94 and a distance based on the difference between the positions where the object Q92 is photographed by the left and right cameras 93 and 94. 84, the distance 8 due to the difference in the position of the closer object Q92
4 is larger than the distance 83 due to the difference in the position of the object P91.

Therefore, the left and right cameras 93, 9
The depth of the object can be measured by comparing the distances 83 and 84 due to the difference in the position photographed in Step 4.

As other conventional examples, there have been proposed methods in which the optical axes are not parallel, examples in which the number of cameras is more than two, examples in which cameras are arranged not only horizontally but also vertically. However, all have the common feature that a plurality of cameras are arranged to shoot an image.

[0012]

However, in the above conventional example, when photographing a subject, photographing with a plurality of cameras arranged side by side, particularly when measuring depth using photographed images, each camera has It is necessary to have the same characteristics.

This is because if the characteristics of the cameras are not uniform, the same object is photographed with different brightness by each camera. Objects having different luminances are recognized as different objects, so that the depth of the object cannot be measured.

Assume that the object P71 is hidden behind the object Q72 as viewed from the right camera 74 as shown in FIG.

At this time, an image 61 captured by the left camera 73 is as shown in FIG. 10A, and an image 62 captured by the right camera 74 is as shown in FIG. 10B.

In the image 62, since the object P71 is not photographed, the difference in the position at which the object P71 is photographed is indicated by the image 61.
Cannot be measured between the image 62 and the image 62. Therefore,
The depth of the object P71 cannot be measured. This disadvantage can be improved by increasing the number of cameras used.

For example, as shown by a broken line in FIG.
By arranging another camera 75, an image 63 shown in FIG.
By measuring the difference in the position at which the object P71 is photographed between and, the depth can be measured.

However, as the number of cameras increases, the task of matching the characteristics of each camera becomes more complicated.

It is an object of the present invention to provide an image generating apparatus and an image generating method capable of simplifying the apparatus configuration and increasing work efficiency when generating a parallax image from a captured image. .

[0020]

SUMMARY OF THE INVENTION The present invention is an apparatus for photographing a subject from different positions to generate an image, comprising: one camera for photographing at least one subject; A lens array is provided between the lines of sight connecting the cameras, and a plurality of lenses are arranged in an array in a direction intersecting the line of sight, and images are taken by the one camera via each lens of the lens array. An image generating apparatus, comprising: Is configured.

The present invention is an image generating method for generating an image by photographing an object from different positions, wherein at least one object is photographed through a lens array having a plurality of lenses arranged in an array. A photographing step of photographing by inputting to one camera, and dividing each part of the photographed image photographed by the one camera into a plurality of parallax images by dividing the area corresponding to each lens position of the plurality of lenses. And an area dividing step of forming an image.

Here, the area dividing means includes means for recognizing a lens position of each lens in the lens array, means for recognizing a photographing area included in each lens at the recognized lens position, Means for creating a plurality of parallax images based on the photographing area corresponding to each recognized lens.

The lens array may be a lens array in which concave lenses are arranged in an array, or a lens array in which convex lenses are arranged in an array.

[0024]

Embodiments of the present invention will be described below in detail with reference to the drawings.

[First Example] A first embodiment of the present invention will be described with reference to FIGS.

(System Configuration) First, the configuration of the image generating apparatus according to the present invention will be described.

The present apparatus comprises: a lens array in which a plurality of lenses arranged on the object side are arranged in an array; a camera including an optical camera or a television camera; And a device.

More specifically, as shown in FIG. 1, a concave lens array 18 arranged on the two objects 11 and 12 as subjects, a television camera 13, and an area connected to the output side of the television camera 13. And a dividing device 14.

The concave lens array 18 includes concave lenses 15, 1
6, 17 are arranged in a horizontal line. The number of lenses is not particularly limited.

The television camera 13 has a concave lens array 18
Objects 11 and 12 are photographed through.

The area dividing device 14 divides the area of the photographed image 21 according to the position of the concave lens at which each part of the photographed image 21 photographed by the camera 13 has passed.

If the number of lenses constituting the lens array 18 is two or more, the depth can be measured in any arrangement. In this case, the region dividing device 14
The depth between objects is measured by connecting a depth measuring device 50 at the subsequent stage.

The lenses constituting the lens array 18 are not limited to concave lenses, but may be convex lenses,
A general spherical lens or aspherical lens, or a gradient index optical fiber having optical characteristics equivalent to those of these lenses may be used.

Further, as the area dividing device 14, a wipe device used for producing a television program may be used. In this case, the wiping device may be either an analog type or a digital type.

(System Operation) Next, the operation of the present apparatus will be described.

Here, a case where two objects P11 and Q12 are photographed and their parallax images 22, 23 and 24 are obtained will be described.

First, the television camera 13 photographs the objects 11 and 12 through the concave lens array 18.

At this time, since the light is refracted by the concave lenses 15 to 17, the television camera 1
There are a plurality of optical paths to 3 and a plurality of images of the objects 11 and 12 are taken.

FIG. 2 shows a photographed image 21 photographed by the television camera 13.

In the photographed image 21, three circles indicate the outer shapes of the concave lenses 15 to 17. Since the side surfaces of the concave lenses 15 to 17 do not transmit light, when the lens array 18 is photographed by the television camera 13, the outer shape of the lens appears. By painting the sides of each of the lenses 15 to 17 black, a clearer lens outline can be photographed by the television camera 13.

Then, in an area divided by three circles indicating the outer shape, an image of a subject passing through the corresponding lenses 15 to 17 is photographed. The photographed image 2 thus photographed
1 is input to the area dividing device 14. In this region dividing device 14, each part of the photographed image 21
-17 are divided according to whether or not the image has passed through.

(Region Division) A region division method for creating a parallax image by cutting out a portion passing through each lens from the input photographed image 21 will be described.

The concave lens 15 constituting the lens array 18
Can be easily visually recognized as described above.

Assuming that the positional relationship between the lens array 18 and the television camera 13 is fixed, the outer shape of the concave lens 15 is always photographed at the same position in the photographed image 21. Therefore, after assembling the present apparatus, it is sufficient to select only the portion surrounded by the outer shape of the concave lens 15 in the photographed image 21 and output the photographed image 21 while watching the photographed image 21 photographed by the television camera 13. However, this selection adjustment can be performed not only by manual operation but also by automatic recognition by a program, and only needs to be performed once if the positional relationship does not change after assembling the apparatus.

By cutting out the photographed image 21 in this way, the parallax image 22 can be cut out. The same applies to a method of cutting out only a portion that has passed through the other concave lenses 16 and 17.

In other words, first, the lens position of each concave lens 15 in the lens array 18 is recognized, and the photographed image 21 included in each lens at the recognized lens position is recognized.
Recognize the shooting area. After that, the photographing area is divided in accordance with the recognized lens position, thereby obtaining FIG.
The parallax images 22 to 24 shown in FIG.

FIG. 3 shows three parallax images 22 to 24 obtained by area division. The parallax images 22 to 24 obtained in this way are the images 61 to 63 (FIGS. 10A to 10C) which are photographed by arranging three television cameras side by side.
).

Thus, by inputting the parallax images 22 to 24 to the depth measuring device 50, the depth between the object 11 and the object 12 can be measured by the same method as in the conventional case described above.

As described above, a plurality of images 11 and 12 are photographed from different positions by the television camera 13 via the lens array 18 having the concave lenses 15 to 17, and the photographed images 21 are input to the area dividing device 14. The parallax images 22 to 24 can be created by performing the region division processing by using the above-described method. This eliminates the need to arrange a plurality of cameras in parallel as in the related art. Can be eliminated.

The created parallax images 22 to 24
Is input to the depth measurement device, and the distance (depth) from the camera to the subject can be measured by measuring the difference in the position in the image where the subject is photographed.

[Second Example] A second embodiment of the present invention will be described with reference to FIGS. Note that the same parts as those in the above-described first example are denoted by the same reference numerals, and description thereof will be omitted.

In this embodiment, a convex lens array 38 is formed by using convex lenses 35, 36, and 37 instead of the concave lenses 15, 16, and 17. Other configurations are the first
Is the same as the example.

Here, two objects 11 (P), 12
The case where (Q) is photographed and a parallax image is created from the captured image will be described.

The television camera 13 has a convex lens array 38.
The objects 11 and 12 are photographed via. At this time, since the light is refracted by each of the convex lenses 35 to 37, the object 1
There are a plurality of optical paths from the light sources 1 and 12 to the television camera 13, and a plurality of images of the objects 11 and 12 as subjects are photographed.

Unlike the first example, the light is projected on the convex lenses 35-3.
7, an inverted image of the object is photographed.

While paying attention to this, the TV camera 1
3, the photographed image 41 as shown in FIG.
Is obtained.

In the photographed image 41 shown in FIG. 5, three circles indicate the outer shapes of the convex lenses 35 to 37.

When the lens array 38 is photographed by the television camera 13 as in the case of the concave lenses 15 to 17 of the first example described above, the outer shape of the lens appears. In addition, each lens 35-3
By painting the side surface of black 7 black, a clearer lens outline can be photographed by the television camera 13. Then, in a region divided by three circles indicating the outer shape, a photographed image 41 is obtained as an image of a subject passing through the corresponding lenses 35 to 37.

The photographed image 41 thus photographed
Is input to the area dividing device 14. In this region dividing device 14, each part of the photographed image 41 is determined
Depending on whether the image has passed through 37, the captured image 41 is divided into regions, and parallax images 42 to 44 as shown in FIG. 6 are obtained.

This area division processing can be performed in the same manner as in the first example described above. That is, after assembling the device, while observing the captured image captured by the television camera 13, only the portion of the parallax image 41 surrounded by the outer shape of each of the lenses 35 to 37 is selected and adjusted so as to be output. Good.

Images 42 to 4 output from the area dividing device 14
4 is equivalent to 180 degrees of each of the images 22 to 24 output in the first example.

Also in this example, the depth measuring device 5
Using 0, it is possible to measure the difference between the positions where the objects 11 and 12 are photographed, and to measure the depth between the objects. In this case, the captured image 41 is rotated by 180 degrees, but can be measured without any inconvenience.

In each of the above examples, a spherical or aspherical lens can be used as the concave lens or the convex lens, and an optical element such as a gradient index optical fiber having the same optical characteristics as these can be used. Good.

The region dividing device 14 is manually adjusted while viewing the photographed images 21 and 41. If the shape and size of the concave lenses 15 to 17 or the convex lenses 35 to 37 used for the lens arrays 18 and 38 are the same, figures having the same shape are arranged in the captured image. In the case of the above-described example, since the circular lenses are used, circles of the same size are arranged in the captured images 21 and 41. If a circle having a predetermined size is automatically recognized by a pattern matching method, it is not necessary to manually recognize the outer shape of the lens.

For the purpose of calibrating the apparatus, a white subject having a sufficiently large area may be photographed. In this case, in the above-described example, the television camera 13 photographs only the outer shape of the lens. If this image is used for automatic recognition of the lens outline by a manual or pattern matching method, only the lens outline is photographed in the image, so the recognition work is easy and the accuracy is improved. I do.

[0066]

As described above, according to the present invention,
An image of a subject is shot by a single camera through a lens array in which a plurality of lenses are arranged in an array, and the shot image is divided into regions corresponding to each lens to generate a plurality of parallax images. Therefore, it is not necessary to use a plurality of cameras, and the work of matching the characteristics of the cameras is not required, whereby the work efficiency can be improved and the device configuration can be simplified.

According to the present invention, the depth of a subject can be measured by inputting a plurality of parallax images generated using the apparatus according to the present invention to a depth measuring device. Unlike the conventional case, it is not necessary to make the characteristics of a plurality of cameras uniform, and an inexpensive measuring device can be manufactured.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image generation device according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating a captured image captured by a camera.

FIG. 3 is an explanatory diagram illustrating a parallax image generated from the captured image in FIG. 2;

FIG. 4 is a block diagram illustrating a configuration of an image generation device according to a second embodiment of the present invention.

FIG. 5 is an explanatory diagram illustrating a captured image captured by a camera.

6 is an explanatory diagram illustrating a parallax image generated from the captured image in FIG.

FIG. 7 is an explanatory diagram showing a camera arrangement in a stereo camera which is a conventional parallax image generation device.

FIG. 8 is an explanatory diagram showing a photographed image of a subject photographed by the camera of FIG. 7;

FIG. 9 is an explanatory diagram showing a camera arrangement in the case of photographing an object at a hidden position in the conventional parallax image generation device.

FIG. 10 is an explanatory diagram illustrating a captured image of a subject captured by the camera in FIG. 9;

[Explanation of symbols]

 11, 12 Object 13 Camera 14 Region dividing device 15-17 Concave lens 18 Concave lens array 21 Captured image 22-24 Parallax image 35-37 Convex lens 38 Convex lens array 41 Captured image 42-44 Parallax image 50 Depth measuring device

 ──────────────────────────────────────────────────の Continuing on the front page (72) Inventor Tomoyuki Mishina 1-10-11 Kinuta, Setagaya-ku, Tokyo Inside the Japan Broadcasting Corporation Broadcasting Research Institute (72) Inventor Mitsuho Yamada 1-1-10 Kinuta, Setagaya-ku, Tokyo 11 Japan Broadcasting Corporation Broadcasting Research Institute (72) Inventor Fumio Okano 1-10-11 Kinuta, Setagaya-ku, Tokyo F-term in Japan Broadcasting Corporation Broadcasting Research Institute 2F065 AA06 DD06 FF01 FF04 JJ03 JJ19 JJ26 LL02 LL10 QQ00 QQ38 UU07 2F112 AC06 BA20 CA12 DA04 FA38 5B047 BC05 CB23

Claims (6)

[Claims] 1. An apparatus for photographing a subject from different positions to generate an image, comprising: a camera for photographing at least one subject; and a line of sight connecting the subject and the one camera. A lens array in which a plurality of lenses are arranged in an array in a direction intersecting with the line of sight, and a photographed image photographed by the one camera via each lens of the lens array is input; An image generating apparatus comprising: an area dividing unit that divides each part of a captured image into areas corresponding to each lens position of the plurality of lenses to form a plurality of parallax images. 2. The apparatus according to claim 2, wherein the area dividing unit includes: a unit configured to recognize a lens position of each lens in the lens array; a unit configured to recognize an imaging region included in each lens at the recognized lens position; 2. An image generating apparatus according to claim 1, further comprising: means for generating a plurality of parallax images based on the photographing areas corresponding to the respective lenses. 3. The image generating apparatus according to claim 1, wherein the lens array is a lens array in which concave lenses are arranged in an array. 4. The image generating apparatus according to claim 1, wherein the lens array is a lens array in which convex lenses are arranged in an array. 5. An image generating method for generating an image by photographing a subject from different positions, wherein at least one subject is connected to one lens via a lens array having a plurality of lenses arranged in an array. A plurality of parallax images are formed by dividing each part of the photographed image photographed by the one camera into regions corresponding to the respective lens positions of the plurality of lenses. An image generation method, comprising: 6. The area dividing step includes: a step of recognizing a lens position of each lens in the lens array; a step of recognizing a photographing area included in each lens at the recognized lens position; 6. A method according to claim 5, further comprising the step of: generating a plurality of parallax images based on the photographing areas corresponding to the respective lenses.