JP3321941B2 - Image synthesis device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a method for synthesizing a plurality of
The present invention relates to an image synthesizing apparatus for forming a single image.

[0002]

2. Description of the Related Art FIG.
FIG. 1 is a configuration diagram of a conventional wide-angle image capturing apparatus disclosed in Japanese Patent Application Laid-Open Publication No. HEI 10-115,036. In the figure, 101 is a prism, 102 is a taking lens, 104 is a film, and 105 is a reel.

Next, the operation will be described. Shooting lens 1
The prism 101 provided in front of 02 is arranged so that its second surface b and the surface of the image frame are parallel to each other, and is rotatably provided. When the release is performed in the state shown in FIG. 10, a subject at a position shifted to the left from the normal is photographed. When the first frame is photographed, the prism 101 is rotated by 180 degrees by the motor while the parallelism between the second surface b and the surface of the image frame is maintained, and the second frame is photographed. In this case, the subject at the position shifted to the right from the normal is photographed, so that the subject at the position shifted to the right considerably from the first frame in the second frame is shot. As a result, an angle of view equivalent to that of an ultra-wide-angle lens can be obtained in the horizontal direction of the screen simply by using an ordinary wide-angle lens as a photographing lens. High price problems can be avoided.

[0004]

Since the conventional image synthesizing apparatus is constructed as described above, it is assumed that the distance between the camera and the object can be regarded as constant regardless of the angle of view, for example, when the object is sufficiently far away. However, when the image is a plane and the distance from the camera is short, the right and left images have different perspectives and the image near the combining point is unnatural.

Also, when the camera is installed at an angle to the object plane, there is another problem that the image near the combining point is unnatural.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is an image synthesizing apparatus which synthesizes two images and obtains an ultra-wide-angle image similar to that seen from a single viewpoint. The purpose is to provide.

[0007]

According to a first aspect of the present invention, there is provided an image synthesizing apparatus, comprising: one or more imaging devices for obtaining images from a plurality of directions; and a screen address of a plurality of images obtained by the imaging devices. An address generation circuit for generating
A screen address generated from the address generation circuit
By changing only the vertical address of Chi images, and an address conversion circuit which composite image obtained is deformed images equivalent shape and imaging region when viewed from a single viewpoint,
A memory for synthesizing a plurality of images transformed by the address conversion circuit is provided.

According to a second aspect of the present invention, in the image synthesizing apparatus, the address conversion circuit outputs a multiplier for multiplying a vertical address by inputting a horizontal address of the image; And a multiplier for multiplying the output of the table by a vertical address.

[0009]

According to the first aspect of the present invention, there is provided an image synthesizing apparatus comprising: one or more image capturing apparatuses for obtaining images from a plurality of directions;
An address generating circuit for generating a screen address of the plurality of images obtained by the imaging device, using the address conversion circuit for transforming the image to convert the said screen addresses, images Chi sac image obtained by the imaging device The shape of the region to be imaged is adjusted only in the vertical direction, and the transformed plurality of images are synthesized by the memory.

According to a second aspect of the present invention, there is provided an image synthesizing apparatus, comprising: a look-up table which outputs a multiplier for multiplying an input of a horizontal address of an image by a vertical address with respect to an input of a horizontal address of the image. , constituted by a multiplier, changing only the vertical address of the image.

[0011]

[Embodiment 1] FIG. 1 is a block circuit diagram showing a configuration of an image synthesizing apparatus according to one embodiment. In the figure, reference numeral 1 denotes a first having an image pickup device such as a CCD image pickup device for photographing an area on the left side from the center.
2, a second camera for photographing an area on the right side, 3 a synchronous drive circuit for driving the first camera 1 and the second camera 2 in synchronization, 4 a video signal of the first camera 1 Is a first A / D converter for converting the video signal of the second camera 2 into a digital signal, and 6 is a first low-pass filter (hereinafter, referred to as a first low-pass filter). 7 is a second LPF, 8 is an address generation circuit, 9 is an address conversion circuit, 10 is a video signal selection circuit, 11 is a camera selection signal, and 12 is a random access memory (hereinafter, referred to as “RAM”). ), 13 are D / A
The converter 14 is an output terminal.

FIG. 2 shows the object plane as the ground plane, and the area to be photographed on the ground plane photographed by each camera when the first and second cameras are installed at the same depression angle from the horizontal plane. FIG. 3 is a diagram showing a state in which the imaging area is formed on the image sensor. In FIG. 2, reference numeral 15 denotes an area photographed by the camera 1 when the first camera 1 is set to swing to the left from the center line by a certain angle, and 16 denotes an area set to swing to the right by the same angle as the camera 1. This is an area captured by the second camera 2.
In addition, a, b, c, and d in 15 are four corners of the photographing area, and l, m, and p, q, r, and s are straight lines drawn in a grid pattern on the subject surface. Reference numeral 17 denotes a state in which the region 15 photographed by the first camera 1 and the grid lines in the region are imaged on the image sensor, and the four corners a, b, and 4 of the photographing region are shown.
Since c and d are imaged at the four corners A, B, C and D on the image sensor, they are deformed as shown in FIG. Straight lines L and M and straight lines P and
Q, R, and S correspond to the straight lines l, m and the straight lines p, q, r, s in 15, respectively. Numeral 18 indicates a state in which an image of the area 16 captured by the second camera 2 is formed on the image sensor. That is, if two images are combined as they are, a straight line cannot be correctly reproduced in the combined portion as shown in FIG. FIG. 4 is a diagram showing an example of the output of the image synthesizing device according to the present embodiment, and 19 is a display frame.

Next, the operation will be described. The first camera 1 and the second camera 2 are driven by the synchronous drive circuit 3 in the same phase. Video signals output from the first camera 1 and the second camera 2 are output from a first A / D converter 4,
The signal is converted into a digital signal for each pixel by the second A / D converter 5, and the band is limited by the first LPF 6 and the second LPF 7 so that aliasing noise does not appear in the subsequent processing.

The address generating circuit 8 generates horizontal and vertical addresses (x, y) on the image sensor in the scanning order of the screen based on the image sensor driving clock from the synchronous drive circuit 3 and the horizontal synchronizing signal and the vertical synchronizing signal. , A camera selection signal 11 for alternately switching the outputs of the first camera 1 and the second camera 2 for each clock. The camera selection signal 11 is, for example, “Hi” when selecting an image from the first camera 1.
gh ', a signal of' Low 'when an image from the second camera 2 is selected. The address conversion circuit 9 is a read-only memory (hereinafter, referred to as “ROM”) that outputs an address for writing a video signal to the RAM 12 in response to the address on the image sensor obtained from the address generation circuit 8 and the input of the camera selection signal 11. It is composed of

The ROM in the address conversion circuit 9 stores, as data, values corresponding to coordinates on a shooting area corresponding to a screen address on the image sensor. The coordinates (X1, Y1) of a point on the imaging region 15 that forms an image at one point (x1, y1) on the image sensor of the first camera 1 are the points (x1, y1) on the image sensor.
Since the straight line connecting the camera and the principal point of the lens intersects the object plane, the angle of the camera 1 with respect to the object plane, the distance from the object plane and the swing angle from the center to the left, the focal length of the lens, and the size of the image sensor It can be obtained from it. In the same manner, a point (X2, Y2) on the photographing area 16 which forms an image on the point (x2, y2) on the image sensor of the second camera 2 can be obtained. As a result, points on the photographing area corresponding to all the pixels on the image sensor are obtained, and obtained (X1, Y1), (X2, Y
2) All or a part of the area is reduced to an appropriate size so that it can be displayed on one screen, and data to be a write address of a video signal to the RAM 12 is determined and stored in the ROM.

The video signals from the first camera 1 and the second camera 2 are input to a video signal selection circuit 10 and output to the RAM 12 alternately for each clock, that is, for each pixel in accordance with the camera selection signal 11. . When the camera selection signal 11 is “High”, a video signal from the first camera 1 is output. At this time, the address conversion circuit 9 supplies the camera selection signal 11 indicating the first camera and the screen address corresponding to the pixel. Is entered. In accordance with the camera selection signal 11, the address conversion circuit 9 outputs data corresponding to each screen address. The output from the address conversion circuit 9 is used as a write address to the RAM 12, and the video signal is written to a designated address on the RAM 12 according to the output. The data is sequentially read from the RAM 12 in the scanning order, converted into an analog signal by the D / A converter 13, and then output from the output terminal 14 to a display, a recorder, or the like. As a result, the lattice-like pattern on the object surface is directly displayed on the display or the like as shown in FIG.

Embodiment 2 FIG. FIG. 5 is a block circuit diagram showing the configuration of the image synthesizing apparatus according to one embodiment. In the figure, reference numeral 31 denotes a camera having a lens and an image pickup device such as a CCD image pickup device; 32, a principal point position of the lens of the camera 31; 33, a camera rotation drive circuit for driving the camera 31 to rotate around the principal point position 32; And 34, an image sensor driving circuit for driving the image sensor of the camera 31, 35 an A / D converter for converting the video signal of the camera 31 into a digital signal, 36 an LPF, 37 an address generating circuit, and 38 an address converter. Circuit, 39 is camera position information, 40 is RAM, 41 is D / A converter, 42
Is an output terminal.

Next, the operation will be described. The camera 31 is rotationally driven left and right by a camera rotation drive circuit 33 about an axis orthogonal to the bottom surface of the camera and passing through the principal point 32 of the lens. The camera rotation drive circuit 33 is a camera 3
When 1 is at a predetermined shooting position, camera position information 39 indicating that fact is output. When the camera 31 rotates a predetermined angle to the left from the center and has a depression angle similar to that of the first embodiment, and the photographing area becomes a portion 15 in FIG. The photographing area when rotated by the same angle to the right is the part 16 in FIG. This portion forms an image on the image sensor as in FIG. 3 as in the first embodiment. In this case, the trajectory drawn by the optical drive of the camera 31 in the photographing area by the rotation drive is a straight line.

The image sensor of the camera 31 is driven by an image sensor driver circuit 34 and outputs a video signal. The output signal is converted into a digital signal for each pixel by an A / D converter 35, and the band is limited by an LPF 36 so that aliasing noise does not appear in subsequent processing.

In the address generating circuit 37, the synchronous driving circuit 34
Then, an address (x, y) on the image sensor is generated in the scanning order of the screen based on the image sensor driving clock, horizontal synchronizing signal and vertical synchronizing signal. The address conversion circuit 38 includes an address on the image sensor obtained from the address generation circuit 37 and camera position information 39 obtained from the camera rotation drive circuit 33.
The ROM is configured to output an address for writing a video signal to the RAM 40 in response to the input of.

One point (x, y) on the image sensor of the camera 31
The point (X, Y) on the photographing area 15 that forms an image on
Can be obtained in the same way as Here, assuming that the horizontal angle of view of the camera 31 is α degrees and the vertical angle of view is β degrees, an image obtained when the camera is rotated left and an image obtained when the camera is rotated right are obtained by synthesis. The resulting image is a horizontally long image having a horizontal angle of view of about 2α degrees while maintaining the vertical angle of view. If a virtual camera using a lens having a horizontal angle of view of 2α degrees and a vertical angle of view of β degrees and having no distortion is installed facing the center in the same state as the camera 31, the area in FIG. 7
An image is formed on the image sensor as shown in FIG. Here, the straight lines p, q, r, s and the straight lines l, m on FIG. 6 are straight lines drawn in a grid pattern on the subject plane, similarly to FIG. Also, the straight lines p ′, q ′, r ′, s ′ and the straight lines l ′, m ′ in FIG. 7 are formed by connecting the straight lines p, q, r, s and the straight lines l, m on the image sensor of the virtual camera. It is an image. A point (U, U) on the imaging region of FIG.
V) can be determined on a one-to-one basis as in the first embodiment.
The photographing area of the virtual camera (FIG. 6) such that the lattice-shaped straight line coincides with the photographing area obtained in the first embodiment (FIG. 2).
Correspond. Thereby, the point (x, y) on the image sensor of the camera 31 corresponds to the coordinates (X, Y) on the shooting area.
The value of the coordinates (U, V) on the area photographed by the virtual camera having the angle of view of the sailor 2α degrees is determined. Further, the coordinates (U, V)
Is converted into coordinates (u, v) on the image sensor of the virtual camera corresponding to this one-to-one. R in the address conversion circuit 38
In OM, the screen address (x,
The data is stored in advance so that the coordinates (u, v) of the virtual camera determined above are output in response to the input of y) and the camera position information 39.

In the RAM 40, the corresponding video signal output from the LPF 35 is written to the address output from the address conversion circuit 38. The data is sequentially read from the RAM 40 in the scanning order, converted into an analog signal by the D / A converter 41, and then output from the output terminal 42 to a display, a recorder, or the like. As a result, as shown in FIG. 7, the lattice-like pattern on the subject surface is corrected in the horizontal direction on the display or the like, and the image is displayed with the perspective in the depth direction remaining.

Embodiment 3 FIG. FIG. 8 is a block circuit diagram showing a configuration of the image synthesizing apparatus according to one embodiment. In the figure, 1 to 7, 10 to 14
Is the same as in the first embodiment. Reference numeral 51 denotes an address generation circuit, reference numeral 52 denotes an address conversion look-up table (hereinafter, referred to as “LUT”), reference numeral 53 denotes a multiplication circuit. The address conversion LUT 52 and the multiplier 53 form an address conversion circuit 54. FIG. 9 is a diagram illustrating an example of an output of the image synthesizing apparatus according to the present embodiment.

Next, the operation will be described. Camera 1, 2
Since the flow of the video signal from the video signal selection circuit 10 to the video signal selection circuit 10 and the operation of the synchronous drive circuit 3 are the same as in the first embodiment, the description is omitted. In the address generation circuit 51, the horizontal address (x) on the image sensor in the scanning order of the screen based on the image sensor drive clock from the synchronous drive circuit 3 and the horizontal synchronization signal,
It generates a vertical address (y) and a camera selection signal 11 for alternately switching the outputs of the first camera 1 and the second camera 2 for each clock. Of these, the horizontal address is
When inputting to the address conversion LUT 52, for example, in the case of an output signal from the first camera 1 for photographing the left side, the left end of the image becomes 0 and the address increases as going to the right, and the output from the second camera 2 In the case of a signal, conversion is performed so that the right end of the image becomes 0 and the address increases as going to the left, so that the address becomes symmetrical when viewed from the center of the composite image. Further, the vertical address (y) is represented by the amount of displacement in the vertical direction from the center line with the vertical center line of the image sensor as 0, and + above the center and − below the center.
Is converted to the value of LUT 52 for address conversion
Is the horizontal address from the address generation circuit 51,
In response to the input of the signal to determine the sign of the vertical address,
Outputs the multiplier to multiply the vertical address.

The multiplication circuit 53 multiplies the vertical address output from the address generation circuit 51 by a multiplier output from the address conversion LUT 52 to generate a Y address for writing to the RAM 12. As a result, L is adjusted so that the vertical length of the converted image becomes equal to the ratio in FIG.
The multiplier of the UT 52 is set. The RAM 12 stores the video signal from the video signal selection circuit 10 at a location specified by the X address from the address generation circuit 51 and the Y address from the multiplication circuit 53. The data is sequentially read from the RAM 12 in the scanning order, converted into an analog signal by the D / A converter 13, and then output from the output terminal 14 to a display, a recorder, or the like. As a result, as shown in FIG. 9, a lattice-like pattern on the object surface is displayed on a display or the like while keeping the perspective in the depth direction.

[0026]

As described above, according to the first aspect of the present invention, the screen address generated by the address generating circuit for deforming the image so that the linear portion is reproduced in the obtained composite image. since an address conversion circuit for changing only the vertical address of Chi images, it is possible to obtain a composite image having the same sense of depth and when viewed from one viewpoint approximately in a small circuit scale.

According to the second aspect of the present invention, the address conversion circuit outputs a multiplier for multiplying a vertical address by inputting a horizontal address of an image, and an output of the lookup table. And a multiplier that multiplies the vertical address by a vertical address, so that it is possible to obtain a synthesized image having a sense of depth equivalent to that when viewed from one viewpoint approximately with a small circuit scale.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram showing a configuration of an image synthesizing apparatus according to one embodiment.

FIG. 2 is a diagram showing an area on a subject surface photographed by the imaging device in FIG. 1;

FIG. 3 is a diagram showing an image obtained on an image sensor in FIG. 1;

FIG. 4 is a diagram illustrating an example of an output of the image synthesizing device in FIG. 1;

FIG. 5 is a block circuit diagram showing a configuration of an image synthesizing apparatus according to one embodiment.

FIG. 6 is a diagram showing an area on a subject plane photographed by a virtual camera having a horizontal angle of view of 2α degrees.

FIG. 7 is a diagram illustrating an example of an output of the image synthesizing device in FIG. 5;

FIG. 8 is a block circuit diagram showing a configuration of an image synthesizing apparatus according to one embodiment.

FIG. 9 is a diagram illustrating an example of an output of the image synthesizing device in FIG. 8;

FIG. 10 is a cross-sectional view of a conventional wide-angle imaging device.

[Explanation of Signs] 1, 2, 31 Imaging device 8, 37, 51 Address generation circuit 9, 38, 54 Address conversion circuit 12, 40 RAM 32 Main point of lens 33 Camera rotation drive circuit 52 Address conversion LUT 53 Multiplier

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-5-110926 (JP, A) JP-A-5-14751 (JP, A) JP-A-7-79379 (JP, A) JP-A-4-110 275783 (JP, A) JP-A-3-226075 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 5/262-5/265 H04N 5/225-5/232 G06T 1/00 280 G06T 3/00

Claims (2)

(57) [Claims] 1. One or more imaging devices that obtain images from a plurality of directions, an address generation circuit that generates screen addresses of a plurality of images obtained by the imaging device, and an image signal that is generated from the address generation circuit. Screen address
By changing only the vertical address of Chi images, and an address conversion circuit which composite image obtained is deformed images equivalent shape and imaging region when viewed from a single viewpoint, in the address conversion circuit An image synthesizing device, comprising: a memory for synthesizing a plurality of deformed images. 2. The address conversion circuit according to claim 1, further comprising: a look-up table for outputting a multiplier for multiplying a vertical address by an input of a horizontal address of the image; and multiplying an output of the look-up table by a vertical address. 2. The image synthesizing apparatus according to claim 1, comprising a multiplier.