JP6278771B2 - Projection position determination device and projection position determination program - Google Patents

Description

  The present invention relates to an image combining technique for generating a panoramic image by combining a plurality of images obtained by a plurality of photographing devices.

  The panorama image generation is a process of obtaining a single image by projecting and superimposing camera images taken by a plurality of cameras (imaging devices) on an imaging surface. Panorama image generation is a technology that uses multiple cameras to generate images with an angle of view that cannot be captured with ordinary cameras and images from a free viewpoint. Used.

In panorama image generation, a camera image is projected onto a virtual projection surface of various shapes such as a plane and a curved surface, and a panorama image is generated by cutting out a portion corresponding to the line-of-sight direction.
There are various algorithms for generating a panoramic image using a virtual projection plane. The simplest algorithm is to synthesize all input images by projectively transforming the camera image from the imaging surface to the virtual projection surface, and then projectively transform the portion that corresponds to the line-of-sight direction to a plane to obtain a panoramic image in the line-of-sight direction There is something to get. Further, if the ray tracing method is used, only the image in the line-of-sight direction can be obtained by back calculation, and a panoramic image in the line-of-sight direction can be obtained with a smaller calculation amount and memory capacity.

  When projecting a camera image in panoramic image generation (or when performing back calculation from the projection surface by the ray tracing method), it is necessary to set how the camera image is projected onto the projection surface. It should be noted that, hereinafter, general settings regarding the positional relationship between the projection plane and the camera imaging plane, such as the projection position and projection angle of the camera image, are simply referred to as “projection position”. In the panorama image generation, the camera image is converted into an output image based on the setting of the projection position P and the line-of-sight direction.

In order to prevent unnatural distortion and double images from occurring in the overlapped part of each camera image in the panorama image, the projection position of the camera image is set to match the relationship between the projection plane and the actual camera orientation as much as possible. Need to be done.
However, it is difficult to accurately measure the camera mounting angle and the optical axis direction, and it is actually necessary to change the setting value and the camera mounting angle while checking the composite image. Therefore, when there are a large number of cameras or when it is difficult to change the camera mounting angle, it takes time to determine the projection position and to fine-tune the camera mounting angle to solve the distortion and double image of the overlapped part. .

  In Patent Document 1, a new wide-angle camera is installed so as to include the entire field of view of each camera image to be panorama synthesized, and the projection position of each camera image is determined from the degree of coincidence between the wide-angle camera image and each camera image.

JP 2009-124665 A

However, the method described in Patent Document 1 requires a new wide-angle camera, which increases the cost of the apparatus. Further, only images within a range that can be covered by the field of view of the wide-angle camera can be synthesized. For example, when generating a panoramic image of the entire circumference with the projection surface as a spherical surface, a panoramic camera that can cover the entire circumference with one device is required.
It is an object of the present invention to set an appropriate projection position without using a new wide-angle camera and to generate a highly accurate panoramic image.

The projection position determining apparatus according to the present invention is
A projection position determination device that determines a projection position for generating a panoramic image by combining a plurality of images obtained by a plurality of imaging devices,
A projection position setting unit that sets a plurality of projection positions indicating positions where an image obtained by the imaging apparatus is projected with respect to at least one of the plurality of imaging apparatuses;
An evaluation value calculation unit that calculates an evaluation value indicating a degree of coincidence of each image in a panoramic image obtained by combining the plurality of images for each projection position set by the projection position setting unit; ,
And a projection position determining unit that determines a projection position based on the evaluation value calculated by the evaluation value calculating unit.

  The projection position determination apparatus according to the present invention calculates an evaluation value when a panoramic image is generated based on a plurality of projection positions, and determines the projection position based on the evaluation value. Thereby, an appropriate projection position can be set without using a new wide-angle camera, and a highly accurate panoramic image can be obtained.

The figure which shows the usage example of the panoramic image production | generation in the driving | operation control assistance of an aircraft. The conceptual diagram of panorama image generation. 1 is a configuration diagram of a panoramic image generation apparatus 100 (projection position determination apparatus) according to Embodiment 1. FIG. 5 is a flowchart showing the operation of the panoramic image generation apparatus 100. The figure which shows the intermediate | middle output (projection image of each camera image) and final output (panoramic image) in a panoramic image generation. Explanatory drawing of the calculation method of an evaluation value. Explanatory drawing of the search method of a projection position. Explanatory drawing of the process which specifies optimal projection position relationship. Explanatory drawing of the process which determines the optimal projection position _Pbest of each camera. Explanatory drawing of the process which determines the optimal projection position _Pbest of each camera in the case of considering the optimal projection position relationship of a surrounding camera. The figure which shows the difference in the process of Embodiment 1 and Embodiment 2. FIG. The figure for demonstrating the difference between the panorama image generation process for a display, and the panorama image generation process for a search. The figure which shows the difference in the process of Embodiment 2 and Embodiment 3. FIG. FIG. 3 is a diagram illustrating an example of a hardware configuration of a panoramic image generation apparatus 100 shown in the first to third embodiments.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating a usage example of panoramic image generation in assisting in driving and driving an aircraft.
In FIG. 1, six cameras are attached around the aircraft body, and panoramic images in the line-of-sight direction are generated from the six camera images and output to the display device. The line-of-sight direction is an arbitrary direction and does not necessarily coincide with the direction of the line of sight of someone. For example, any direction that the passenger wants to see may be used, or a fixed direction may be used. Of course, you may make it correspond with someone's gaze direction.
The lower part of FIG. 1 shows an image of a panoramic image obtained by synthesizing camera images obtained by three cameras (cameras 1 to 3) among the six cameras. Adjacent camera images partially overlap.

FIG. 2 is a conceptual diagram of panoramic image generation.
In panorama image generation, a camera image is projected onto a virtual projection plane of various shapes such as a plane and a curved surface, and a panorama image is generated by cutting out a portion corresponding to the line-of-sight direction. FIG. 2 shows a case where the projection surface is a spherical surface.
In the overlapping portion where the adjacent camera images partially overlap, a single object may appear to be plural or a straight line may be bent due to a shift in the projection position of each camera image.
Therefore, in order to obtain a highly accurate panoramic image, it is necessary to set an appropriate projection position for each camera.

FIG. 3 is a configuration diagram of the panoramic image generation apparatus 100 (projection position determination apparatus) according to the first embodiment.
The panorama image generation apparatus 100 includes a panorama image generation unit 110 (composite image generation unit), a projection position setting unit 120, a projection position evaluation unit 130 (evaluation value calculation unit), and a projection position determination unit 140.

The panorama image generation unit 110 generates a panorama image by inputting a camera image (hereinafter referred to as an input image) captured by each camera, a line-of-sight direction, and a projection position of each camera image set by the projection position setting unit 120. To do.
The panoramic image generation unit 110 includes a camera-by-camera image projection unit 111 and an inter-camera image synthesis unit 112. The per-camera image projecting unit 111 projects an input image for each camera based on the line-of-sight direction and the projection position, and generates a projected image of each camera image serving as an intermediate output. The inter-camera image combining unit 112 combines the projection images generated by the camera-by-camera image projecting unit 111 to generate a panoramic image.

  The projection position setting unit 120 sets the projection position for each camera based on the initial projection position of each camera and the search parameter.

  The projection position evaluation unit 130 calculates an evaluation value indicating the degree of coincidence of the images of the overlapped portion where the images in the projection images of the camera images generated by the camera-by-camera image projection unit 111 overlap.

  The projection position determination unit 140 determines the optimal projection position of each camera based on the evaluation value calculated by the projection position evaluation unit 130 and the projection position set by the projection position setting unit 120.

FIG. 4 is a flowchart showing the operation of the panoramic image generation apparatus 100.
(S1: Initial projection position setting process)
The projection position setting unit 120 sets an initial projection position _Pinit of each camera according to a user input or the like. The initial projection position _Pinit is the projection position at the start of processing as the name suggests, and sets the positional relationship between the approximate projection plane and the camera imaging plane. In addition, the projection position setting unit 120 sets search parameters according to user input and the like. The search parameter is a parameter for setting how to shift the projection position of the camera image, that is, its range, step size, and the like.

Note that the projection position and parameters of the camera image are set using different values depending on how the projection position of the panoramic image generation apparatus 100 is shifted (= optimum projection position search method) and the projection position setting method. Since the panorama image generation apparatus 100 can be used in combination with any search method and setting method, the search method and the setting method are not particularly defined. For example, the panoramic image generation apparatus 100 may be combined with a full search method or a gradient method. Further, the projection position of the camera image may be set by values such as yaw, roll, pitch, and origin position, or the optical axis direction may be represented by a matrix.
In the following, for simplification of description, as a simple example, three cameras are used, the full search method is used as the search method, and the operation when the yaw, roll, and pitch steps are set as the search parameters will be described. .

(S2: Panorama image generation process)
The panorama image generation unit 110 receives the input image from each camera, the line-of-sight direction, and the projection position, generates a panorama image, and outputs a projection image of each camera image that is an intermediate output of the process of generating the panorama image. .

FIG. 5 is a diagram illustrating an intermediate output (projected image of each camera image) and a final output (panoramic image) in panoramic image generation.
In the panorama image generation unit 110, first, the camera-by-camera image projection unit 111 projects an input image from each camera based on the line-of-sight direction and the projection position currently set for the camera, and outputs an intermediate output. A projection image of each camera image is generated. Then, the inter-camera image synthesis unit 112 synthesizes the projection image generated by the camera-by-camera image projection unit 111 for each pixel to generate a panoramic image.

(S3: Evaluation value calculation process)
The projection position evaluation unit 130 receives the projection image of each camera image as an input, and calculates an evaluation value indicating the degree of coincidence between the images for the overlapped portion where the projection images overlap.

FIG. 6 is an explanatory diagram of an evaluation value calculation method.
The projection position evaluation unit 130 cuts out at least a part of the image showing the same range from the overlapped portion of two adjacent projection images, and determines the degree of coincidence of the cut out images between two projection images (between cameras). (Evaluation value). Any kind of evaluation value may be used as long as the degree of coincidence between images can be uniquely calculated and superiority and inferiority can be compared. As the simplest example, the sum of absolute differences or the sum of squares of differences may be calculated for the pixel values of the clipped image.
In FIG. 6, as an example, a case where the image to be cut out is a rectangle is shown. However, an image to be cut out may be an arbitrary shape instead of a rectangle, or may be the entire overlapping portion. Further, the user may be able to set the shape and range of the image to be cut out.

(S4: Search end determination process)
The projection position setting unit 120 determines whether or not the search for the projection position is completed for the range set in S1. If the search is not completed (NO in S4), the process proceeds to S5. If the search is completed (YES in S4), the process proceeds to S6.

(S5: Projection position search process)
The projection position setting unit 120 changes the projection position P of the target camera image according to the parameter set in S1 every time image synthesis is performed with the initial projection position as a starting point in S1. Then, the process returns to S2.

FIG. 7 is an explanatory diagram of a projection position search method.
FIG. 7 shows a case where the projection position of the camera 2 is moved in the horizontal (= yaw) direction. When the pitch value or the roll value is changed, it moves in the vertical direction or the rotation direction. In FIG. 7, only the projection position of the camera 2 is moved, but the movement is not limited to the projection position of the camera 2. Any camera may be moved during the search, or a plurality of cameras may be moved simultaneously or one camera may be moved at a time.

(S6: Optimal projection position determination process)
First, the projection position determination unit 140 compares the evaluation values between the cameras for each projection position, thereby specifying the projection position relationship (optimum projection position relationship Pd _best ) that gives the _best overlay between the cameras. . That is, the positional relationship between two cameras (when the projection position is represented by yaw, roll, pitch, determinant, three-dimensional coordinate information of the origin, etc., the difference between the projection positions simply represents the positional relationship between the two cameras) The best evaluation value is given when it becomes.

FIG. 8 is an explanatory diagram of processing for specifying the optimum projection position relationship.
By repeating the processing from S2 to S5, the evaluation value between the two cameras is calculated every time the projection position is shifted. The projection position determination unit 140 identifies a case where the most excellent evaluation value (an evaluation value having a high degree of coincidence between two projection images) is obtained from the calculated evaluation value. Then, the projection position determination unit 140 identifies the optimum projection position relationship Pd _best of the two cameras when identified.
For example, if only the projection position of the camera 2 is shifted, the evaluation value between the projection image of the camera 1 and the projection image of the camera 2 for each projection position of the camera 2 is calculated. Therefore, the projection position determination unit 140 specifies the projection position of the camera 2 when the evaluation value between the projection image of the camera 1 and the projection image of the camera 2 is the best value. The projection position determination unit 140, the projection position ^P _{2 1Best} of the identified camera 2, the optimum projection position relationship _Pd best _(1-2) between the camera 1 and the camera 2 _{(= -Pd best (2-1 ))} via the use of an initial projection position ^p _{1 init} camera _{1, Pd best (1-2) (} = -Pd best (2-1)) = identified by _{P 2 1best} ^-p _{1 init.}

The projection position determination unit 140 further determines an optimum projection position P _best for each camera from the optimum projection position relationship Pd _best for each camera.

FIG. 9 is an explanatory diagram of processing for determining the optimum projection position P _best of each camera.
First, the projection position determination unit 140 determines a reference camera in advance according to a user input or the like. The reference camera is a camera that serves as a reference for determining the optimum projection position of all cameras, and is a camera for setting the origin. The projection position determination unit 140 determines the optimum projection position of another camera using the initial projection position of the reference camera as the origin. That is, in the reference camera, the optimum projection position P _best = the initial projection position P _init . Therefore, when setting the initial projection position of the reference camera, it is necessary to measure the optical axis direction and the mounting angle, and to set an accurate value compared to other cameras.
The determination of the optimum projection position of a certain camera A is most simply obtained by adding the optimum projection position relationship Pd _best between the camera A and the reference camera to the initial projection position _Pinit of the reference camera.
FIG. 9 shows a case where the reference camera is the camera 1 and the optimal projection position P ² _best of the camera 2 is calculated. Since the camera 1 is a reference camera, the optimal projection position P ¹ _{best of the} camera 1 = the initial projection position P ¹ _init of the camera 1. The optimal projection position P ² _best of the camera ² is obtained by adding the optimal projection position relationship Pd _{best (1-2)} between the camera 1 and the camera 2 to the initial projection position P ¹ _init of the camera 1. , P ² _best = P ¹ _init + Pd _{best (1-2)} .

As shown in FIG. 10, it is possible to determine the optimum projection position P _best in consideration of the optimum projection position relationship with the surrounding cameras as well as the reference camera.
FIG. 10 shows a case where the optimum projection position P ² _best of the camera 2 is calculated in consideration of not only the optimum projection position relation with the camera 1 but also the optimum projection position relation with the camera 3. Here, (1) the optimal projection position of the camera 2 calculated from the above-described optimal projection position relationship between the camera 1 and the camera 2 (A) P ² _best = P ¹ _init + Pd _{best (1-2)} , (2 ) Calculated using the optimum projection position relationship Pd _{best (1-3)} between the camera ₁ and the camera 3 and the optimum projection position relationship Pd _{best (3-2)} between the camera 3 and the camera 2 Optimal projection position of camera 2 (B) P ² _best = P ¹ _init + Pd _{best (1-3)} + Pd _{best (3-2)} average (average of (1) and (2)) The projection position is P ² _best . Although the average is used here, a weighted average or the like may be used instead of the average.

  Even when the number of cameras is large and a certain camera A is not in contact with the reference camera, all the cameras can be obtained by adding the optimum projection position relationship via a camera close to the reference camera in the same manner as the method shown in FIG. Can be determined.

(S7: Optimal projection position setting process)
The projection position setting unit 120 sets the optimum projection position for each camera determined in S6 as the projection position of each camera image.

  As described above, the panoramic image generation apparatus 100 according to Embodiment 1 performs the evaluation on the projection position between the cameras while shifting the projection position, and determines the optimum projection position. Therefore, if only the projection plane and the projection position of the reference camera image are accurately measured and set, the other cameras can obtain a projection position in which a double image or the like does not occur only by setting an approximate value.

  In the above description, the case where the projection position of the camera image is changed for each frame of the image has been described. However, the projection position of the camera image may be changed every several frames.

Embodiment 2. FIG.
In the first embodiment, the panorama image is generated while shifting the projection position, and the optimum projection position is determined by calculating the evaluation value for the intermediate output projection image obtained in the process of panorama image generation. However, if the projection position is shifted in order to search for the optimum projection position, the projection position of the panorama image that is the final output of the panorama image generation unit 110 is naturally shifted little by little. When a panoramic image is displayed in real time, the panoramic image to be displayed is changed by changing the projection position for each frame.

In the second embodiment, display panorama image generation and search panorama image generation are made independent, and scheduling of operations is performed. Thereby, when the panorama image is displayed in real time, it is possible to prevent the displayed panorama image from being changed by changing the projection position for each frame.
In the second embodiment, only parts different from the first embodiment will be described.

FIG. 11 is a diagram illustrating a difference in processing between the first embodiment and the second embodiment.
In the first embodiment, the panorama image generation process for display (reproduction process) and the panorama image generation process for optimum projection position search are common. Therefore, when searching for the optimum projection position, the projection position for display (for real-time display) is also shifted at the same time.
That is, as shown in FIG. 11, in the first frame, a common panoramic image generation process is performed for display and optimum projection position search using the projection position P ₀₀ , and in the second frame, the projection position P ₀₀ using the projection position P ₀₁ which is shifted slightly from the common panoramic image generation process is performed for the search display and the optimum projection position, 3 in the frame, using the projection position P ₀₂ which is shifted slightly from the projection position P ₀₁ Thus, a common panoramic image generation process is performed for display and for searching for the optimum projection position.
In order to match the cycle to the frame rate, there is a stop period (vertical blanking period) in which the processing device in the panorama image generation device 100 stops between the panorama image generation processing of each frame.

In the second embodiment, the generation of the panoramic image for display and the generation of the panoramic image for searching are made independent, and the panoramic image for searching is generated during the stop period of the processing apparatus that exists to match the cycle with the frame rate. As a result, in the real-time display, the optimum projection position is searched while the projection position is fixed.
That is, as shown in FIG. 11, 1 in frame, together with the panoramic image generation processing for display by using the projection position P ₀₀ is performed, the projection image generation process for the optimum projection position search using the projection position P ₀₀ In the second frame, display panorama image generation processing is performed using the projection position P _00, and projection image generation processing for optimal projection position search is performed using the projection position P _01. the frame, together with the panoramic image generation processing for display by using the projection position P ₀₀ is performed, the projection image generation processing of the optimum projection position search is performed using the projection position P _02.
In particular, the projection image generation process for searching for the optimal projection position is performed during the stop period in which the processing apparatus is stopped, which is between the panoramic image generation processes of the respective frames in the process in the first embodiment.

That is, the projection position setting unit 120 sets the projection position P _init for display (P _init is an example, and the optimum projection position or the like in the previous search may be output), and the frame for the search. Projection positions P _{00 to} P _XX that are slightly shifted every time are set.

FIG. 12 is a diagram for explaining a difference between a display panorama image generation process and a search panorama image generation process.
In the display panorama image generation processing, the image-by-camera image projection unit 111 needs to perform processing on the entire input image, and the inter-camera image synthesis unit 112 needs to perform image synthesis. However, in the panoramic image generation process for search, it is only necessary to perform processing for a part of images (images cut out in S3 in FIG. 4) for the camera-by-camera image projection unit 111 to calculate the evaluation value. The combining unit 112 does not need to operate. Therefore, the panorama image generation process for search can be performed in a shorter time than the panorama image generation process for display.

  The projection position evaluation unit 130 and the optimum projection position determination unit 140 also operate only during the search period.

  As described above, in the panorama image generation device 100 according to Embodiment 2, when the panorama image is displayed in real time, the panorama image to be displayed is changed by changing the projection position for each frame. Can be prevented.

  In the above description, the case where one projection position is searched in the stop period for one frame has been described. However, it is also possible to search for a plurality of projection positions depending on the length of the stop period.

Embodiment 3 FIG.
In the second embodiment, evaluation values are calculated for one (or several) projection positions per frame. However, when the time change of the image is large, depending on the projection position, there are a projection position where the evaluation value is calculated for an image whose evaluation value is likely to be small, and a projection position where the evaluation value is calculated for an image which is likely to be large Therefore, if the superiority and inferiority of the evaluation values are compared as they are, there is a possibility that the evaluation is inappropriate.

In the third embodiment, an evaluation value is also calculated for a projection image generated for display, and this value is used as an index of the feature amount of each frame. This increases the validity of the evaluation.
In the third embodiment, only parts different from the second embodiment will be described.

FIG. 13 is a diagram illustrating a difference in processing between the second embodiment and the third embodiment.
The third embodiment is different from the second embodiment in that the projection position evaluation unit 130 also calculates an evaluation value for the projection image generated for display and uses the evaluation value as an index of the feature amount of the frame. Different.

Only the operation of the projection position evaluation unit 130 is different from the second embodiment.
The projection position evaluation unit 130 operates in both the display period and the search period, and calculates evaluation values for both the projection image generated for display and the projection image generated for search. . Further, the projection position evaluation unit 130 divides the evaluation value of the projection image generated for search by the evaluation value of the projection image generated for display, thereby outputting the true value to be output to the optimum projection position determination unit 140. An evaluation value is calculated.

  As described above, the panoramic image generation apparatus 100 according to Embodiment 3 can evaluate the projection position in consideration of the feature amount difference between frames, and can obtain a projection position with fewer double images and the like.

FIG. 14 is a diagram illustrating an example of a hardware configuration of the panoramic image generation apparatus 100 described in the first to third embodiments.
The panorama image generating apparatus 100 is a computer. Each element of the panoramic image generation apparatus 100 can be realized by a program.
As a hardware configuration of the panoramic image generation apparatus 100, an arithmetic device 901, an external storage device 902, a main storage device 903, a communication device 904, and an input / output device 905 are connected to a bus.

  The arithmetic device 901 is a CPU (Central Processing Unit) that executes a program. The external storage device 902 is, for example, a ROM (Read Only Memory), a flash memory, a hard disk device, or the like. The main storage device 903 is, for example, a RAM (Random Access Memory). The communication device 904 is, for example, a communication board. The input / output device 905 is, for example, a mouse, a keyboard, a display device, or the like.

The program is normally stored in the external storage device 902, and is loaded into the main storage device 903 and sequentially read into the arithmetic device 901 and executed.
The program is a program that realizes the functions described as the panoramic image generation unit 110, the projection position setting unit 120, the projection position evaluation unit 130, and the projection position determination unit 140.
Furthermore, an operating system (OS) is also stored in the external storage device 902. At least a part of the OS is loaded into the main storage device 903, and the arithmetic device 901 executes the above program while executing the OS.
Further, in the description of the first to third embodiments, the information described that the panorama image generation unit 110, the projection position setting unit 120, the projection position evaluation unit 130, and the projection position determination unit 140 generate, set, calculate, specify, and determine Is stored in the main storage device 903 as a file.

  The configuration in FIG. 14 is merely an example of the hardware configuration of the panorama image generation device 100, and the hardware configuration of the panorama image generation device 100 is not limited to the configuration described in FIG. There may be.

  100 panorama image generation device, 110 panorama image generation unit, 111 image-per-camera projection unit, 112 inter-camera image synthesis unit, 120 projection position setting unit, 130 projection position evaluation unit, 140 projection position determination unit

Claims (7)

A projection position determination device that determines a projection position for generating a panoramic image by combining a plurality of images obtained by a plurality of imaging devices,
A projection position setting unit that sets a plurality of projection positions indicating positions where an image obtained by the imaging apparatus is projected with respect to at least one of the plurality of imaging apparatuses;
An evaluation value calculation unit that calculates an evaluation value indicating a degree of coincidence of each image in a panoramic image obtained by combining the plurality of images for each projection position set by the projection position setting unit; ,
A projection position determination unit that determines a projection position based on the evaluation value calculated by the evaluation value calculation unit ;
The projection position setting unit sets a projection position between a reproduction projection position and an evaluation projection position,
The projection position determining device further includes:
Each of the plurality of images is projected onto the reproduction projection position, and a reproduction process for generating a panoramic image obtained by synthesizing the plurality of images is executed by a processing device, and the plurality of panoramic images generated by the reproduction process are continuously performed. Each of the plurality of images is projected by the processing device to the evaluation projection position during a stop period of the processing device between two playback processes for generating two consecutive panoramic images. Composite image generator
With
The evaluation value calculation unit calculates the evaluation value by the processing device based on a projection image projected on the evaluation projection position by the composite image generation unit during the stop period. Projection position determination device. The projection position determining unit determines a projection position relationship between two imaging devices based on the evaluation value, and the projection position relationship is based on a projection position of a reference imaging device as a reference among the plurality of imaging devices. The projection position determination apparatus according to claim 1, wherein the projection position of another imaging apparatus is determined based on the above. The projection position determination unit is configured to perform the imaging apparatus A based on the projection position relationship between an image obtained by the imaging apparatus A among the other imaging apparatuses and a plurality of imaging apparatuses on which the obtained images overlap. The projection position determination apparatus according to claim 2, wherein the projection position is determined. The composite image generation unit, only a portion of the overlapping portion of each of the plurality of images by projecting the evaluation projection position, claim 1, wherein the generating a composite image of only said portion 4. The projection position determination device according to any one of items 3 to 3 . The projection position setting unit sets a different projection position for each stop period as the evaluation projection position,
The projection position determination unit, based on the evaluation value calculated for a plurality of said stop period, the projection position determining apparatus according to any one of claims 1, characterized in that to determine the projected position to the 4 . The evaluation value calculation unit calculates an evaluation value α based on the projection image projected on the evaluation projection position by the composite image generation unit, and based on the projection image projected on the reproduction projection position by the composite image generation unit. The projection position determination apparatus according to claim 5 , wherein an evaluation value β is calculated, and the evaluation value is calculated based on a correlation between the evaluation value α and the evaluation value β. A projection position determination program for determining a projection position for generating a panoramic image by combining a plurality of images obtained by a plurality of imaging devices,
A projection position setting process for setting a plurality of projection positions indicating positions for projecting an image obtained by the imaging apparatus for at least one of the plurality of imaging apparatuses;
An evaluation value calculation process for calculating an evaluation value indicating a degree of coincidence of each image in a panoramic image obtained by combining the plurality of images for each projection position set in the projection position setting process, ,
Based on the evaluation value calculated in the evaluation value calculation process, causing the computer to execute a projection position determination process for determining a projection position ,
In the projection position setting process, a projection position between the reproduction projection position and the evaluation projection position is set,
The projection position determination program further includes:
Each of the plurality of images is projected onto the reproduction projection position, and a reproduction process for generating a panoramic image obtained by synthesizing the plurality of images is executed by a processing device, and the plurality of panoramic images generated by the reproduction process are continuously performed. Each of the plurality of images is projected by the processing device to the evaluation projection position during a stop period of the processing device between two playback processes for generating two consecutive panoramic images. Composite image generation processing
To the computer,
In the evaluation value calculation process, the evaluation value is calculated by the processing device based on the projection image projected on the evaluation projection position in the composite image generation process in the stop period. Projection position determination program.

Images