JP4545503B2 - Image generating apparatus and method - Google Patents

Description

  The present invention relates to an image generation apparatus and method, and more particularly to an image generation apparatus and method suitable for generating and displaying a virtual viewpoint image based on images captured by a plurality of imaging means.

  In order to display a specific area on a monitor screen for monitoring or other purposes, images obtained by photographing a wide area with multiple cameras can be divided and displayed on a single screen, or images taken every time can be displayed. The display is switched sequentially. In addition, by mounting a camera on the vehicle, using the camera directed to the rear of the vehicle, the area that the driver cannot see directly or indirectly is photographed and displayed on the monitor provided in the driver's seat, contributing to safe driving I try to let them.

  However, because these monitoring devices display images on a camera-by-camera basis, the number of installations increases when shooting a wide area, and the number of installations decreases when a wide-angle camera is used, but the accuracy of the image displayed on the monitor is displayed roughly. The image is difficult to see and the monitoring function is degraded. For this reason, a technique has been proposed in which images from a plurality of cameras are combined and displayed as one image.

At the time of image generation, distance data such as obstacles existing around the vehicle is measured, and image generation is performed using the data as data for displaying the obstacle on the image.
For example, Patent Document 1 proposes a technique for capturing the surroundings of a vehicle with a plurality of cameras installed on the vehicle and displaying a composite image from an arbitrary viewpoint. In this method, the video output of each camera is coordinate-transformed on a two-dimensional plane viewed from the virtual viewpoint in units of pixels and expanded, and a plurality of camera images are combined into one image viewed from the virtual viewpoint and displayed on the monitor screen. It is what I did. As a result, the driver can instantly grasp what kind of object is present around the entire periphery of the vehicle from one virtual viewpoint image. At this time, there is disclosed a method of displaying a plane on a space model based on a distance between a vehicle and an obstacle existing around the vehicle using a distance sensor.

Also, the subject is imaged with a stereo camera, the correlation between a pair of images is obtained, and the distance is obtained from the parallax for the same object using the camera parameters such as the installation position and focal length of the stereo camera to determine the distance for image processing. We are using. (For example, cited document 2)
Japanese Patent No. 3286306 Japanese Patent Laid-Open No. 5-114099

  However, since the generated image of Patent Document 1 is displayed with reduced image quality of the three-dimensional image in order to speed up the composite image processing, there is a problem that the generated image is not clear and the visibility of the display image is poor.

  Also, it is known that stereo captured images have a short base line length, and the distance measurement accuracy decreases at long distances, and if the base line length is long, the subject at a short distance is too large to be displayed on the screen. ing.

  Accordingly, in order to solve the above-described problems of the prior art, the present invention generates a viewpoint-converted image with a high degree of recognition using a short-distance image or a long-distance image in generating a viewpoint-converted image from images from a plurality of imaging means. It is an object of the present invention to provide an image generation apparatus and method capable of performing the same.

An image generation apparatus according to the present invention is an image generation apparatus that generates a viewpoint-converted image from a virtual viewpoint based on image information supplied from a plurality of imaging units, and a viewpoint selection that selects an arbitrary virtual viewpoint to be displayed Means, distance measurement means for generating a distance image corresponding to the image information based on input images from the plurality of imaging means, space model generation means for generating a space model using the distance image, and A spatial reconstruction unit that maps input images from a plurality of imaging units to the generated spatial model to generate spatial data, and generates a viewpoint-converted image viewed from a desired virtual viewpoint with reference to the spatial data to a viewpoint conversion unit, wherein the at least a plurality of predetermined ones of the plurality of the imaging means is constituted by a stereo camera unit, between the respective stereo camera unit Location distance is set longer than the baseline length of the stereo camera unit alone, the space model generation means, spatial model of short distance from the virtual viewpoint is selected using a distance image based on the short baseline length imaging by the stereo camera unit alone The long-distance spatial model is characterized in that the spatial model is generated using a distance image based on a long baseline length imaging by a stereo camera configured by a combination of the pair of stereo camera units.

In this case, it is preferable to provide switching means for switching between imaging with a short baseline length by the stereo camera unit and imaging with a long baseline length by a stereo camera configured by a combination of a pair of stereo camera units. .

The image generation method according to the present invention is an image generation method for generating a viewpoint-converted image from a virtual viewpoint based on image information supplied from a plurality of imaging units, and selects an arbitrary virtual viewpoint to be displayed. , At least a predetermined plurality of the plurality of imaging means is configured by a stereo camera unit, and an arrangement distance between the stereo camera units is set to be longer than a baseline length in the stereo camera unit alone, and the selected virtual viewpoint The short distance spatial model uses a distance image based on the short baseline length imaging by the stereo camera unit alone, and the long distance spatial model has a long baseline length by a stereo camera configured by a combination of the pair of stereo camera units. The spatial model is generated using a distance image based on imaging, and input images from the plurality of imaging means To generate spatial data by mapping the generated space model, with reference to the spatial data, it is characterized by generating a viewpoint conversion image seen from a desired virtual viewpoint.

  According to the image generating apparatus and method having the above-described configuration, the short baseline length imaging by a single stereo camera unit and the long baseline length imaging by a stereo camera configured by a combination of a pair of stereo camera units are enabled. Images by distance imaging and long-distance imaging can be generated. Based on this image, it is possible to obtain distance information obtained by calculating the distance of the subject on the image, a spatial model for short-distance imaging, and a spatial model for long-distance imaging. For this reason, a spatial model with high recognition degree can be generated by selectively using these at an arbitrary distance. Therefore, it is possible to generate a viewpoint conversion image with a high degree of recognition.

Embodiments of an image generation apparatus and method according to the present invention will be described below in detail with reference to the accompanying drawings.
FIG. 1 is a system block diagram illustrating a configuration of an image generation apparatus according to the present embodiment. The basic configuration of this system includes a plurality of imaging means, an image generation apparatus for processing image data acquired by the imaging means, and reproducing and displaying them as a composite image viewed from a virtual viewpoint different from the camera viewpoint. The viewpoint conversion composite image generation / display device 10 is configured as follows.

  The basic processing in the viewpoint conversion composite image generation / display apparatus 10 is to input an image photographed from the viewpoint of each imaging means, set a three-dimensional space where an imaging means placement object such as a vehicle is placed, and this 3 The dimensional space is specified by an arbitrarily set origin (virtual viewpoint), and the pixel of the image data is coordinated and corresponded to the three-dimensional space viewed from the specified virtual viewpoint, and on the image plane viewed from the virtual viewpoint. A process of rearranging the pixels is performed. As a result, an image obtained by rearranging the pixels of the image data obtained from the camera viewpoint in the three-dimensional space defined by the virtual viewpoint to obtain a composite image from a desired viewpoint other than the camera viewpoint is created. It can be output and displayed.

  FIG. 2 shows a schematic configuration diagram of the stereo camera unit 12. As shown in the figure, a pair of left and right light receiving portions 42L and 42R each having a light receiving lens group disposed on the left and right sides of the front portion of the casing 40 by a distance L1 (base line length) are formed so that the subject 44 can be photographed in stereo. Yes. Imaging elements 46L and 46R that receive imaging light input through the light receiving parts 42L and 42R as imaging signals are arranged on the left and right sides of the back surface of the casing 40. Infrared cut filters / low pass filters 48L, 48R are arranged on the optical path between the light receiving parts 42L, 42R and the image sensors 46L, 46R. The infrared cut filter cuts harmful infrared light, and the low pass filter is used to prevent moire.

  Since the captured image with this configuration has almost the same distortion distribution between the left and right cameras, a single stereo camera can be used when using one of the two stereo camera units as a long baseline image. The same image correction algorithm for stereo images as that for short-distance stereo processing on the unit, particularly the image distortion correction parameter in rectification, does not change significantly. Therefore, the combination of the long base lines by the stereo camera units 12A and 12B can be arbitrarily selected from four combinations of 12AL-12BL, 12AR-12BR, 12AR-12BL, and 12AL-12BR.

  An embodiment in which a plurality of stereo camera units 12 are mounted on a vehicle will be described with reference to FIG. A plurality of stereo camera units 12 as imaging means are provided at the front and rear portions of the vehicle 50 as the imaging means arranged object. In the illustrated example, the front camera group 12F (12FR, 12FL) is mounted on the front portion of the vehicle 50. Further, a rear camera group 12R (12RR, 12RL) as an imaging unit is also provided at the rear part of the vehicle. In this embodiment, an image selection device 30 (30F, 30R) for capturing image data from the front and rear camera groups 12F, 12R is provided, and an image from the viewpoint conversion image generation / display device 10 provided near the driver's seat is provided. In response to the selection command, a necessary image is selected and returned to the viewpoint converted image generation / display device 10 as image information. Data transmission / reception may be performed through an in-vehicle LAN line.

  FIG. 4 is an example of an embodiment applied to an indoor monitoring apparatus, and the virtual viewpoint is configured to switch the imaging means at a short distance or a long distance. That is, two stereo camera units 12 as imaging means are installed in parallel on one wall surface of the monitoring target room 40. In this case, as described above, the stereo camera units 12A and 12B are installed such that the arrangement distance is longer than the base line length of the stereo camera unit 12 alone.

  As shown in FIG. 1A, when a person existing at a short distance from the camera is imaged, short base length imaging is performed by a single stereo camera unit 12. The baseline length of the stereo camera unit 12 is set to the optimum baseline length for imaging a short distance. For this reason, since two fields of view overlap in a short distance from the camera and stereo imaging is possible, distance image data can be obtained.

  On the other hand, as shown in FIG. 2B, when imaging a subject at a long distance from the camera installed on the wall surface, a combination of a pair of stereo camera units 12, that is, a pair of stereo camera units 12 12AR-12BR. Or long base line length imaging by the combination of 12AL-12BL is performed. Since the above-described captured image by the stereo camera unit 12 alone has the base length set to short base length imaging, the short-range captured image has good accuracy. However, in the case of a long-distance subject, the image becomes rough, and accurate ranging image data cannot be obtained by the ranging device.

  Therefore, if the pair of stereo cameras 12 is combined (for example, right-right or left-left of a stereo camera), the base line length can be increased. For this reason, it becomes possible to perform distance measurement based on imaging of an area effective for distance measurement at a long distance, which is difficult with short baseline length imaging. Accordingly, the accuracy of the captured image of the subject at a long distance is improved, and accurate ranging image data obtained by the ranging device can be obtained.

  Image data photographed by each stereo camera unit 12 is packet-transmitted to the image selection device 30. Since the image data to be acquired from each stereo camera unit 12 is determined by the set virtual viewpoint, an image selection device 30 is provided to acquire image data corresponding to the set virtual viewpoint. An image data packet corresponding to the set virtual viewpoint is selected from the image data packet input from the buffer device attached to an arbitrary stereo camera unit 12 by the image selection device 30 and used for the subsequent image composition processing. The

  Further, the imaging means switches the plurality of stereo camera units 12 by the image selection device 30. The image selection device 30 serving as switching means switches between imaging with a short baseline length by the stereo camera unit 12 and imaging with a long baseline length by a stereo camera configured by a combination of a pair of stereo camera units. The image selection device 30 performs switching control of the short and long baselines based on a virtual viewpoint of a viewpoint selection device 36 described later. The photographed image data of the imaging unit is temporarily stored in the photographed image data storage device 32.

In the present embodiment, the distance measuring device 13 serving as a distance measuring unit may use both distance measurement by stereo imaging and distance measurement by a radar such as a laser radar or a millimeter wave radar.
In the distance measurement by stereo imaging, the same subject is photographed from a plurality of different viewpoints, the correspondence of the same point of the subject in these images is obtained, and the distance to the subject is calculated by the principle of triangulation. More specifically, the entire right image of the image captured by the stereo imaging means is divided into small areas to determine a range for performing stereo distance measurement, and then the position of the image that is the same as the left image is detected. Then, the position difference between these images is calculated, and the distance to the object may be calculated from the relationship between the left and right camera mounting positions. A distance image is generated based on distance information obtained by stereo distance measurement between two or more images captured by the stereo camera.

  FIG. 5 shows a stereo ranging area by the stereo camera unit. FIG. 2A shows a stereo distance measuring area by a stereo camera unit having a short base length. When the baseline length is shortened due to the principle of triangulation, the distance resolution is reduced, so that it is difficult to measure the distance of a distant object, and a distance measurement limit distance of a practical short baseline length is defined. In the figure, reference numeral 50 denotes a distance measurement limit distance by the stereo camera unit 12 having a short baseline length. In addition, the overlapping 52 of the imaging areas of the single cameras within the distance limit distance indicates a stereo distance measurement area by the stereo camera unit 12 having a short base length. FIG. 2B shows a stereo distance measurement area having a long base line length between the stereo camera units 12A and 12B. When the baseline length becomes long, an object at a short distance does not enter the imaging range common to the two viewpoints, and parallax cannot be measured. Therefore, it is impossible to measure a short distance area. As shown in the figure, in the case of the long baseline length between the pair of stereo camera units 12AL-12BL, 54 indicates the distance limit distance, and 56 indicates the stereo distance measurement area by the stereo camera unit having the long baseline length.

  FIG. 3C shows a stereo distance measuring area between the stereo camera units 12A and 12B. In the figure, reference numerals 58a and 58b denote stereo imaging areas by the stereo camera units 12A and 12B having a short baseline length. In the figure, reference numerals 60a and 60b denote stereo imaging areas by the stereo camera unit pair 12AL-12BL and 12AR-12BR having a long base line length. Further, a ranging area 62 using a laser radar may be overlapped. With this configuration, a distance measurement region for generating a spatial model is defined.

  In the above example, a single base image on the same side of the stereo camera unit is used as a long baseline length image to form a stereo image, and stereo distance measurement is performed. (-12BL) one-side image may be used for stereo ranging.

  Further, the calibration device 18 of FIG. 1 is for the imaging unit arranged in the three-dimensional real world, the mounting position of the imaging unit, the mounting angle, the lens distortion correction value, the focal length of the lens, etc. in the three-dimensional real world. The camera parameters representing the camera characteristics are determined and specified. Camera parameters obtained by the calibration are temporarily stored in the calibration data storage device 17 as calibration data.

  The spatial model generation device 15 can generate short-distance and long-distance spatial models based on the image information of the short baseline length imaging and the long baseline length imaging by the stereo camera unit 12 and the distance image data by the distance measuring device 13. .

  The spatial reconstruction device 14 calculates the correspondence between each pixel constituting the image obtained from the imaging means and the points in the three-dimensional coordinate system, generates spatial data, and temporarily stores it in the spatial data storage device 24. To do. The calculation is performed for all the pixels of the image obtained from each imaging means.

  The viewpoint conversion device 19 serving as a viewpoint conversion unit can convert an image in a three-dimensional space into an image viewed from an arbitrary viewpoint position, and can specify an arbitrarily set viewpoint. That is, it designates from which position in the three-dimensional coordinate system, at which angle, and at what magnification, the image is to be viewed. Further, the image from the viewpoint is reproduced from the spatial data and displayed on the display device 20.

  Note that the viewpoint conversion image generation / display device 10 is provided with an imaging device arrangement object model storage device 34 that stores and stores the vehicle model so that the vehicle model can be displayed at the same time when the space is reconstructed. Yes. In addition, a viewpoint selecting device 36 is provided, and image data corresponding to a preset virtual viewpoint set in advance is stored in the virtual viewpoint data storage device 38, so that an immediate response is made when the viewpoint selecting process is performed. The image is transmitted to the viewpoint conversion device 19 and a converted image corresponding to the selected virtual viewpoint is displayed.

An image generation method using the viewpoint conversion composite image generation / display apparatus 10 configured as described above will be described with reference to FIG. FIG. 6 is a flowchart showing the processing procedure of the image generation method according to the present invention. Hereinafter, an image generation method of the viewpoint conversion image will be described.
(S102) The viewpoint selecting device 36 selects an arbitrary virtual viewpoint to be displayed.
(S104) Images are captured separately for a long distance or a short distance corresponding to the selected virtual viewpoint.
(S106) A plurality of long baseline length stereo pairs are selected by combining the baseline lengths of the 12 pairs of stereo camera units.
(S108) Long baseline length imaging is performed by combining the baseline lengths of the 12 pairs of stereo camera units.
(S110) Calibration used for stereo matching is performed in advance by the calibration device 18, and calibration data such as baseline length, internal and external camera parameters corresponding to the selected stereo camera unit 12 is generated and selected.
(S112) Stereo matching of the picked-up image selected based on the obtained calibration data is performed. In other words, a predetermined window is cut out from the left and right images viewed in stereo, and the normalized correlation value of the window image is calculated while scanning on the epipolar line to search for corresponding points, and the parallax between the pixels of the left and right images is calculated. calculate. The distance is calculated from the parallax based on the calibration data, and the obtained distance data is used as the long baseline long distance image data.
(S114) Short baseline length imaging is performed by the stereo camera unit 12 serving as a short baseline length image pair.
(S116) The short baseline length imaging of the stereo camera unit 12 is performed.
(S118) Calibration used for stereo matching is performed in advance by the calibration device 18, and calibration data such as baseline length, internal and external camera parameters corresponding to the selected stereo camera unit 12 is generated and selected.
(S120) Stereo matching of the picked-up image selected based on the obtained calibration data is performed. In other words, a predetermined window is cut out from the left and right images viewed in stereo, and the normalized correlation value of the window image is calculated while scanning on the epipolar line to search for corresponding points, and the parallax between the pixels of the left and right images is calculated. calculate. The distance is calculated from the parallax based on the calibration data, and the obtained distance data is used as the short baseline long distance image data.
(S144) Regardless of the distance of the obstacle, in generating the spatial model, a short-distance spatial model is created using a short baseline length range image by the stereo camera unit 12, and the long-distance spatial model is the stereo camera unit 12. It is created using a distance image of a long baseline length by a stereo camera composed of a combination of pairs.

The spatial model generation device 15 serving as a spatial model generation unit receives the distance image data of the long baseline length, the distance image data of the short baseline length, and the distance image data from the distance measuring device by the input unit. By using these selectively at a desired distance, a more detailed spatial model is generated.
(S146) The actual image data corresponding to the space model is mapped to the space model in the three-dimensional space according to the calibration data. Spatial data subjected to texture mapping is generated.
(S148) With reference to the spatial data created by the spatial reconstruction device 14, the viewpoint conversion image viewed from the desired virtual viewpoint is generated by the viewpoint conversion device 19.
(S150) The generated viewpoint-converted image data is displayed on the display device 20.

Incidentally, a flowchart showing generation of a distance image by stereo matching according to the present embodiment is shown in FIG.
The right camera and the left camera of the captured image of the stereo camera unit 12 are selected, and a stereo left image and a stereo right image are captured. (S200), (S204)
(S202) The obtained stereo left image.
(S206) The obtained stereo right image.
(S208) The obtained rectification calibration data. The calibration device 18 performs calibration used for rectification in advance, and generates calibration data such as baseline length, internal and external camera parameters according to the right camera or left camera of the selected stereo camera unit 12.
(S210) Next, the left and right stereo images are corrected for distortion aberration of the stereo left and right images based on the calibration data for rectification so that the corresponding points of the left and right images are on the same line on the epipolar line. A rectification process for geometrically transforming the image is performed.
(S212) The obtained stereo left image after rectification.
(S214) The obtained stereo right image after rectification.
(S216) The left and right images after this rectification are subjected to stereo matching processing, corresponding point search is performed, and parallax is calculated. Thereby, a map of the amount of parallax at each point on the image is generated, and this becomes parallax data.
(S218) The obtained parallax data.
(S220) The obtained stereo distance calibration data. Calibration for use in stereo distance measurement is performed in advance by the calibration device 18 to generate calibration data 17 such as baseline length, internal and external camera parameters corresponding to the selected stereo camera unit 12.
(S222) By the stereo distance calibration, the parallax amount is converted into a distance from the reference point, and distance image data is generated.
(S224) The obtained distance image data.

  By performing the processing as described above, the distance image data can be calculated from the images of the plurality of stereo camera units 12. The obtained distance image data is used for generating a spatial model.

  According to such an image generation device 10, an image generation device that generates a viewpoint-converted image from a virtual viewpoint based on image information supplied from a plurality of imaging units, and at least of the plurality of imaging units. The predetermined plurality are configured by a stereo camera unit, and an arrangement distance between the stereo camera units is set to be longer than a baseline length in the stereo camera unit alone, and a short baseline length imaging by the stereo camera unit alone and the stereo camera Since a long base line length image can be captured by a stereo camera composed of a pair of units, distance measurement of a subject on a captured image at a short distance and a long distance can be performed. For this reason, a detailed spatial model can be generated by generating a spatial model using the distance image data and selectively using the spatial model at an arbitrary distance. Therefore, the accuracy of the viewpoint conversion image can be increased.

  Further, by providing a switching means for switching between imaging with a short baseline length by the stereo camera unit and imaging with a long baseline length by a stereo camera configured by a combination of a pair of stereo camera units, A short baseline length spatial model and a long baseline length spatial model can be generated. Therefore, the accuracy of the viewpoint conversion image can be improved.

Further, if the generated virtual viewpoint image is displayed on the monitor screen of the vehicle, it becomes possible to recognize the vehicle surrounding situation in detail, and the safety can be greatly improved.
In each of the above-described examples, the plurality of imaging devices may be used so as to configure a so-called trinocular stereo camera or may be used so as to configure a four-eye stereo camera. In this way, it is known that using a three-lens or four-eye stereo camera provides more reliable and stable processing results in three-dimensional reconstruction processing (Fumiaki Tomita: published by the Information Processing Society of Japan) Information processing "Volume 42 No.4" High-function 3D visual system "). In particular, when a plurality of cameras are arranged in two base length directions, a so-called multi-baseline stereo camera can be realized, and more accurate stereo distance measurement can be realized.

  In addition, although the target which installs imaging means, such as a camera in a predetermined form, shows an example mounted on a vehicle, the imaging device placement object is, for example, indoors such as a pedestrian, a street, a store or a residence, an office, etc. Even when attached, the same image generation can be performed. With this configuration, it is possible to adapt to a monitoring camera or a wearable computer that is attached to a person and performs video-based information acquisition.

It is a block diagram of an image generation device concerning this embodiment. It is a figure which shows the structure schematic of a stereo camera unit. It is a figure which shows the structure outline which mounted multiple stereo camera units in the vehicle. It is a figure which shows the stereo structure which has arrange | positioned the stereo camera unit. It is a figure which shows the stereo ranging area by a stereo camera unit. It is a flowchart of the image generation method which concerns on this embodiment. It is a flowchart which shows the production | generation of the distance image by the stereo matching which concerns on this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ......... Generation / display apparatus of a viewpoint conversion synthetic | combination image, 12 ......... Stereo camera unit, 13 ......... Distance measuring apparatus, 14 ......... Space reconstruction apparatus, 15 ......... Space model generation apparatus, 17 ... …… Calibration data storage device 18 …… Calibration device 19 ...... View point conversion device 20 ...... Display device 22 ...... Space model storage device 24 ...... Spatial data storage device 26 ......... viewpoint-converted image data storage device, 28 ......... distance image data storage device, 30 ......... image selection device, 32 ......... actual image data storage device, 34 ......... imaging device arrangement object model storage device, 36 ......... Viewpoint selection device, 38 ......... Virtual viewpoint data storage device, 40 ......... Monitoring room, 42 ......... Light receiving unit, 44 ......... Subject, 46 ...... Image sensor, 48 ......... Infrared fill Low-pass filter, 50 ......... vehicle.

Claims (3)

An image generation device that generates a viewpoint-converted image from a virtual viewpoint based on image information supplied from a plurality of imaging means,
Viewpoint selecting means for selecting an arbitrary virtual viewpoint to be displayed;
Ranging means for generating a distance image corresponding to the image information based on input images from the plurality of imaging means;
Spatial model generation means for generating a spatial model using the distance image;
Spatial reconstruction means for mapping the input images from the plurality of imaging means to the generated spatial model to generate spatial data;
Viewpoint conversion means for generating a viewpoint conversion image viewed from a desired virtual viewpoint with reference to the spatial data,
At least a predetermined plurality of each of the plurality of imaging means is configured by a stereo camera unit, and an arrangement distance between the stereo camera units is set to be longer than a base line length in the stereo camera unit alone,
The spatial model generation means uses a distance image based on a short baseline length imaging by the stereo camera unit alone as a short distance spatial model from the selected virtual viewpoint , and a long distance spatial model is a combination of the pair of stereo camera units. An image generation apparatus, characterized in that the spatial model is generated using a distance image based on a long baseline length imaging by a stereo camera configured by:   A switching means for switching between imaging with a short baseline length by the stereo camera unit and imaging with a long baseline length by a stereo camera constituted by a combination of a pair of stereo camera units is provided. The image generation apparatus according to claim 1. An image generation method for generating a viewpoint-converted image from a virtual viewpoint based on image information supplied from a plurality of imaging means,
Select any virtual viewpoint to display,
At least a predetermined plurality of the plurality of imaging means is configured by a stereo camera unit, and an arrangement distance between the stereo camera units is set to be longer than a base line length in the stereo camera unit alone,
The short distance spatial model from the selected virtual viewpoint uses a distance image based on the short baseline length imaging by the stereo camera unit alone, and the long distance spatial model is composed of a combination of the stereo camera unit pairs. Using the distance image based on long baseline length imaging with a stereo camera, the spatial model is generated,
Mapping input images from the plurality of imaging means to the generated spatial model to generate spatial data;
An image generation method, characterized in that a viewpoint conversion image viewed from a desired virtual viewpoint is generated with reference to the spatial data .

Images