JP5521599B2 - Video composition apparatus and video composition program - Google Patents

Description

  The present invention relates to a technique for combining and displaying a bird's-eye view video.

  2. Description of the Related Art Conventionally, there is a system that synthesizes a single bird's-eye view image including the periphery of a vehicle from a plurality of camera images mounted on one vehicle and displays it on a display installed in a driver's seat of the vehicle.

  Typically, this is a system that synthesizes a single bird's-eye view image that includes the surroundings of the vehicle from images captured by the four cameras mounted on the front, rear, left, and right of the passenger car and displays them on an in-vehicle display (for example, patents). Reference 1). In this prior art, for example, four predetermined cameras are fixedly used as the vehicle-mounted camera used for synthesizing the overhead view video.

  For example, in the case of a long vehicle such as a large truck or bus, when the bird's-eye view image including the entire vehicle and its surroundings is displayed on one screen, the bird's-eye view image is reduced and the image becomes smaller, and the vehicle periphery appears smaller. Therefore, it is difficult to check the situation around the vehicle on the screen.

  As a countermeasure against the difficulty of confirming the vehicle surrounding situation, it is conceivable to display a part of the bird's-eye view video by automatically zooming up based on a user instruction or based on vehicle information or the like (for example, see Patent Document 2). ). However, in that case, if the number of cameras used is small, the resolution of the video is lowered particularly in places far from the cameras.

  Therefore, as a countermeasure against a further reduction in the resolution of the video, it is conceivable to increase the number of cameras so that the target area where the overhead video is to be displayed can be entirely covered with the video taken from a close distance.

JP 2003-235036 A Japanese Patent No. 3300334 (International Application No. PCT / JP00 / 02474)

  However, in the above-described conventional technology, there is a problem that when the number of camera videos used for synthesizing one bird's-eye view video increases, the circuit scale necessary for video synthesis increases and the amount of calculation processing increases.

  Therefore, in the present invention, in an apparatus that synthesizes a single bird's-eye view image from a plurality of camera images, an optimum image is automatically displayed so that the resolution of the image does not decrease even when the number of cameras used increases. The purpose is to do so.

  In order to achieve the above object, a video synthesizing apparatus disclosed below is a video synthesizing apparatus that synthesizes one overhead image from videos of a plurality of cameras, an input information receiving unit that receives input information, and the input A display range determination unit that determines a display range of the overhead view video based on the input information received by the information reception unit, and a composition of the overhead view video of the display range based on the display range of the overhead view video determined by the display range determination unit A camera selection deciding unit that selects and decides a predetermined number of cameras that shoot the video to be used, and decides which camera video is assigned to the overlapping portion of the predetermined number of camera images determined by the camera selection deciding unit And a predetermined number of cameras determined by the camera selection determination unit based on the allocation of the camera video allocation determination unit and the camera video allocation determination unit The using the video, and a video synthesizing unit which displays by combining the bird's-eye-view video.

  According to the above configuration, even if the number of cameras to be used increases in an apparatus that synthesizes one bird's-eye view image from a plurality of camera images, by efficiently assigning the camera images used for the bird's-eye view image composition, In order to prevent the resolution of the image from decreasing, an optimum overhead view video can be automatically synthesized and output to the display device.

System configuration of the present invention The relationship between the bird's-eye view image area, the vehicle, and the camera, and the field of view Overhead video area divided into meshes Example of a table that defines the fitness map of each camera for each display mode Overall processing flowchart of the present invention Flow chart of camera selection decision processing Flow chart of camera allocation decision processing Example of display range and camera assignment Example of display range and camera assignment

[Embodiment]
FIG. 1 is a block diagram showing the overall configuration of a video composition device 100 according to an embodiment of the present invention. In FIG. 1, the video composition device 100 includes an input information reception unit 110, a display range determination unit 120, a camera selection determination unit 130, a fitness map storage unit 140, a camera video allocation determination unit 150, an input video selection unit 160, and a video. A synthesis unit 170 is provided. Here, as an example, a case where the video composition device 100 is incorporated in an in-vehicle system will be described.

  The input information receiving unit 110 receives vehicle information and input information from the user. The vehicle information is, for example, information indicating a state such as a straight turn, a right turn, a left turn, and a reverse turn of the vehicle sent from the vehicle control unit. The input information is information such as a display range of a bird's-eye view video specified by the user via an input button, a microphone, or the like.

  The display range determination unit 120 is a range to be displayed as a bird's-eye view video (hereinafter referred to as a bird's-eye view video display range) based on information such as vehicle information received by the input information receiving unit 110 and user instruction input information, and a display The mode is determined according to a predetermined procedure. The display mode is, for example, a plurality of display formats set in advance in order to display an appropriate bird's-eye view video image in various cases. For example, even in a mode in which the vehicle is switched according to the state of going straight, turning right, turning left, reverse, etc. The mode may be switched according to the designation.

  The camera selection determination unit 130 selects and determines a set of cameras used for video composition based on the display range of the overhead view video determined by the display range determination unit 120 and the display mode.

  The camera image assignment determination unit 150 refers to the fitness map of each camera from the fitness map storage unit 140 based on the display range and the display mode of the overhead image determined by the display range determination unit 120, and displays each of the overhead video images. In order to synthesize the parts, an assignment of which camera image to use is determined for the overlapping portion of the camera image selected by the camera selection determination unit 130.

  The input video selection unit 160 selects a video of a set of cameras determined to be used for video synthesis by the camera selection determination unit 130 from the images of the cameras C1 to Cm connected to the video synthesis device 100, and the video synthesis unit To 170.

  The video synthesis unit 170 synthesizes the overhead video using the camera video received from the input video selection unit 160 based on the determined display range of the overhead video, display mode, camera selection information, and camera video allocation information. The data is displayed on the display unit 180 connected to the synthesis apparatus 100. The configuration of the video composition device 100 is not limited to the example shown in FIG. For example, the cameras C1 to Cm and the display unit 180 may be configured to be included in the video composition device 100. The fitness map storage unit 140 may be provided in an external storage accessible from the video composition device 100.

  In the processes of the camera selection determination unit 130 and the camera image allocation determination unit 150, the fitness level map of each camera is read from the fitness level map storage unit 140 and used. The fitness map is an example of camera fitness data, and can be configured as a map in which the degree of which camera should be used is defined by the fitness value for each part of the overhead view video to be synthesized. Here, the display range of the bird's-eye view image to be synthesized is defined around the vehicle, for example, as shown in FIG. Note that the field of view of the camera at this time is, for example, as shown in FIG. In the example shown in FIG. 2B, the field of view of each camera partially overlaps the field of view of the adjacent camera. Therefore, the video of each camera has an area partially overlapping with the video of the adjacent camera.

  The display range determining unit determines the display mode based on the input information, and the camera suitability data preferably has data for each display mode. The camera selection determination unit 130 can select and determine a camera using the camera suitability data corresponding to the display mode determined by the display range determination unit. In addition, the camera video assignment determination unit 150 can assign the overhead image to the composition using the camera suitability data corresponding to the display mode determined by the display range determination unit 120.

  Thus, by using the camera suitability data, it is possible to select a camera suitable for the synthesis of the overhead view video and to assign a camera video suitable for the synthesis of the overhead view video.

  The fitness data is, for example, based on the position, direction, viewing angle, etc. of the camera, the resolution, distortion, blind spot, etc. when combining each part of the overhead view video to be synthesized using the video of the camera. It is determined in advance and stored so as to meet the requirements for satisfying the purpose. Note that the value of the fitness data may be automatically calculated from the position, direction, viewing angle, etc. of the camera, or may be manually tunable.

  FIG. 3 divides the region where the bird's-eye view image is synthesized into a mesh shape, defines each divided portion as a segment, and defines a segment name (S11, S12,... S1n, S21, S22,...) For each segment. , S2n, ..., Sm1, ... Smn).

  A definition example of the fitness map at this time is shown in FIG. For each camera (here, C1 to C8), a table defining a fitness map is linked. (However, some drawings are omitted.) Each table defines a fitness map for each display mode (M1, M3,...). Specifically, fitness values for each segment (S11, S12,..., Smn) are defined. The higher the fitness value, the more suitable it is to use the video of the camera to display the segment in the display mode.

  For example, in the example of FIG. 4, the fitness value of the camera C2 is (80, 80, 70, 60, 50, 40) for each segment (S11, S12,..., Smn) in the straight traveling mode (M1). ...) is defined. The fitness value of the camera C3 is defined as (50, 70, 100, 100, 100, 100 ...) for each segment (S11, S12, ..., Smn) in the straight-ahead mode (M1). Has been. The fitness value of the camera C4 is defined as (20, 30, 40, 50, 60, 70...) For each segment (S11, S12,..., Smn) in the straight traveling mode (M1). Yes. Further, the fitness value of the camera C8 is defined as (0, 0, 0, 0, 0, 0 ...) for each segment (S11, S12, ..., Smn) in the straight traveling mode (M1). Has been. That is, in the example of FIG. 4, in the straight mode (M1), the segment S11 is the camera C2, the segment S12 is the camera C2, the segment S13 is the camera C3, the segment S14 is the camera C3, the segment S15 is the camera C3, and the segment S16 is the camera C3. It is shown that it is suitable to use the video. Note that the fitness map in FIG. 4 is an example.

  The camera suitability data has data for each display mode, and when the intermediate third display mode for switching from the first display mode to the second display mode is determined, the suitability of the first display mode is determined. New fitness data may be created using an average value of a value weighted to the value and a value weighted to the fitness value of the second display mode as a fitness value of the intermediate third display mode. As a result, even an intermediate mode can be handled.

  That is, FIG. 4 is an example in which a fitness map is defined for each display mode. However, when the fitness map is actually used, a new fitness map is weighted and synthesized. A goodness-of-fit map may be created and used. For example, when an intermediate third mode for switching from the first mode to the second mode is determined by the display range determination unit 120, a value weighted to the fitness value of the first mode and the second mode A new fitness map may be created and used by using an average value of the weighted fitness values as the fitness value of the intermediate third mode.

  As a specific example, in the case of a mode (intermediate mode) in the process of switching from the straight-ahead mode to the left-turn mode, for example, a value obtained by weighting the fitness value of the straight-ahead mode (M1) with a weight of 0.6 and the left-turn mode (M3) An average value of the fitness value obtained by weighting 0.4 may be used as an intermediate mode fitness value, and an intermediate mode fitness map may be created and used.

  Hereinafter, the operation of the video composition apparatus 100 according to the embodiment of the present invention will be described with reference to the flowchart of FIG.

  First, the display range determination unit 120 determines the display range and the display mode of the overhead view video according to a predetermined procedure based on information such as vehicle information received by the input information reception unit 110 and user instruction input information. (Step S501). For example, the input information accepting unit 110 performs straight, left, right, reverse as vehicle information based on operation information such as turn signals, shifts, steering wheels, accelerators, and brakes by a driver and information from a vehicle control system, a car navigation system, and the like. For example, the display range determination unit 120 can determine a display mode corresponding to the direction (any one of a straight-ahead mode, a left-turn mode, a right-turn mode, and a reverse mode). .

  Data that associates the vehicle information with the display mode is recorded in advance, and the display range determination unit 120 can determine the display mode using this data. For example, a plurality of display range candidates are prepared in advance, and for each display range candidate, the display range is recorded in advance by associating data that matches the conditions that the vehicle information must satisfy in order to select the display range. The determination unit 120 can determine the display range using this data. Furthermore, the display range determination unit 120 can arbitrarily change (move, enlarge, reduce, etc.) the display range based on a user instruction input received by the input information receiving unit 110 through a touch panel operation on the screen. It may be configured.

  Next, the camera selection determination unit 130 selects and determines a predetermined number of camera sets used for video composition based on the display range and the display mode of the overhead view video determined by the display range determination unit 120 (step S502). ). Details of the camera selection determination process will be described later.

  Next, the camera image assignment determining unit 150 refers to the fitness map of each camera from the fitness map storage unit 140 based on the display range and the display mode of the overhead image determined by the display range determining unit 120, and performs a bird's-eye view. In order to synthesize the video, an assignment of which camera video to use is determined for the overlapping portion of the video of the camera selected by the camera selection determination unit 130 (step S503). Details of the camera video assignment determination process will be described later.

  Next, the input video selection unit 160 selects a video of a set of cameras determined to be used for video synthesis by the camera selection determination unit 130 from the images of the cameras C1 to Cm connected to the video synthesis device 100. It is passed to the video composition unit 170 (step S504).

  Next, the video synthesis unit 170 synthesizes the overhead video using the camera video received from the input video selection unit 160 based on the determined overhead view video display range, display mode, camera selection information, and camera video allocation information. Then, it is transferred to the display unit 190 connected to the video composition apparatus 100 and displayed (step S505). The overhead view video is displayed on, for example, a display included in the in-vehicle system.

  Next, the video composition unit 170 determines whether or not to continue the process (step S506), and if so, returns to step S501 and repeats the process.

By repeating the above processing, an input video from an appropriate camera is selected and input according to the user's instruction input and the display range of the overhead view video determined based on the vehicle state input information and the display mode. A bird's-eye view video is synthesized and displayed based on the video.
(Camera selection decision process)
In the present embodiment, the camera selection determination unit 130 stores camera fitness data in which a fitness value, which is a degree suitable for use of a camera image, is determined for each camera for each part of the overhead view video. The data storage unit can be accessed. The camera selection determining unit 130 reads out the fitness value of each part in the display range of the overhead view video from the camera fitness data storage unit, calculates the fitness value in the display range for each camera, and displays the calculated display It is possible to select and determine a predetermined number of cameras in order from the camera having the highest fitness value in the range as the camera used for the synthesis of the overhead view video.

  Thereby, since the camera is selected using the addition value of the fitness values, it is possible to more optimally select the camera used for the synthesis of the overhead view video.

  Next, details of the camera selection determination process (step S502) for determining the selection of a camera group used for the composition of the overhead view video will be described with reference to the flowchart of FIG.

  In advance, a buffer for temporarily recording each “fitness value in the display range” is prepared for all cameras.

  First, the camera selection determination unit 130 selects one camera as a selection candidate (step S601).

  Next, the camera selection determination unit 130 prepares a fitness map for the selected camera (step S602).

  Next, the camera selection determination unit 130 adds all the fitness values for the selected camera in the portion included in the display range of the overhead view video of the prepared fitness map, and calculates the total value of the obtained fitness values. Recording in the buffer (step S603).

  Next, the camera selection determination unit 130 sequentially repeats the processes in steps S601 to S603 for all the cameras, and determines whether or not there are any unprocessed cameras remaining in the processes in steps S601 to S603 (step S604).

  If it is determined in step S604 that an unprocessed camera remains, the camera selection determination unit 130 returns to the process in step 601, selects a new camera, and repeats the processes in steps S601 to S603.

  If it is determined in step S604 that the processing has been completed for all the cameras, the camera selection determination unit 130 determines the predetermined values from the cameras C1 to Cm in descending order of the addition value of the fitness values recorded in the buffer. A predetermined number of camera groups used for video composition is selected and determined so as to be equal to or less than the number of cameras (step S605). Here, as an example, for the display range S shown in FIG. 8A, it is assumed that a camera set of the camera C2, the camera C3, the camera C4, and the camera C8 is selected and determined.

In the example shown in FIG. 6, the total value of the fitness values of the parts included in the display range of the overhead view video is used as the “fitness value in the display range”. It is not limited. For example, the average value, the maximum value, the minimum value, or the area or the number of segments of a part exceeding a predetermined threshold may be used as the degree of fitness of the camera in the display range. it can.
(Camera video assignment decision processing)
In the present embodiment, the camera image assignment determination unit 150 stores camera fitness data in which the fitness value, which is the degree to which the video of the camera is suitable, is determined for each camera for each part of the overhead video. The data storage unit can be accessed. The camera image allocation determining unit 150 selects a camera image having the highest fitness value for the overlapping portion of the video images of the cameras in the overhead view image among the predetermined number of camera images determined by the camera selection determining unit. It can be assigned to the synthesis of the overhead view video of the overlapping portion.

  Thereby, since the video of the camera with the highest fitness value is assigned, the optimal overhead view video can be synthesized efficiently using the camera.

  Next, details of the camera video assignment determination process (step S503) will be described with reference to the flowchart of FIG.

  First, for each segment (S11 to Smn) included in the overhead video display range S, the camera video assignment determination unit 150 prepares a camera that uses video and a buffer that can record the fitness value, and clears the contents. (Step S701). In this example, a buffer for each segment is prepared for the selected cameras C2, C3, C4, and C8.

  Next, the camera video assignment determining unit 150 selects one segment (step S702). Here, it is assumed that the segment S11 is first selected.

  Next, the camera video assignment determining unit 150 selects one camera among the cameras determined by the camera selection determining unit 130 (step S703). Here, it is assumed that the camera C2 is first selected.

  The camera image allocation determination unit 150 uses the fitness value for the segment in the fitness map of the currently selected camera, rather than the fitness value of the camera recorded in the buffer prepared in step 701 for the segment selected in step S702. It is determined whether or not the value is high (step S704). Here, the segment S11 is first determined. When the buffer is cleared, the camera and its fitness value are recorded as they are.

  If the fitness value is high in step S704, the camera video assignment determining unit 150 records the camera name and the fitness value in the segment buffer (step S705). Here, first, the camera C2 and its fitness value “80” are recorded in the buffer of the segment S11. On the other hand, if the fitness value is not high in step S704, the camera video assignment determination unit 150 proceeds with the process to step S706.

  In step S706, the camera video assignment determination unit 150 returns the process to step S703 if the process has not been completed for all the cameras determined to be selected, and proceeds to the next step S707 if it has been completed.

  In step S707, the camera video assignment determination unit 150 returns to the process in step S702 if all the segments have not been processed, selects one segment next to the segment S11, and repeats the process. In that case, the process proceeds to the next step S708.

  That is, for each segment from segment S11 to segment Smn sequentially, the fitness values of camera C2, camera C3, camera C4, and camera C8 are sequentially read from the camera fitness map shown in FIG. 4 and compared. As a result, the camera having a high fitness value and the fitness value are updated and recorded in the buffer of each segment.

  For example, in the example of FIG. 4, when the segment S13 is in the straight traveling mode (M1), first, the camera C2 and the fitness value “70” are recorded in the buffer. Next, the fitness value “100” of the camera C3 is read. Since the fitness value “70” of the camera C2 and the fitness value “100” of the camera C3 are higher, the fitness value of the camera C3 is higher. The camera C3 and the fitness value “100” are updated and recorded in the buffer. Next, the fitness value “40” of the camera C4 is read, and the fitness value of the camera C3 and the fitness value “40” of the camera C4 are higher than the fitness value of the camera C3. , Not updated. Finally, the fitness value “100” of the camera C3 and the fitness value “0” of the camera C8 are not updated because the fitness value of the camera C3 is higher. That is, the camera C3 and the fitness value “100” are recorded in the buffer of the segment C13.

  At step 708, the camera video assignment determination unit 150 uses the video of the camera having the highest fitness value recorded in the buffer of each segment recorded in the buffer as the video of the camera used for combining the segments. Determine as an assignment.

  As described above, it is possible to automatically and appropriately determine which camera video is used to synthesize each segment of the overhead video display area.

  That is, according to the procedure as described above, the fitness map as shown in FIG. 4 is defined, and the maximum number of cameras used for video composition is determined to be 4 and displayed as shown in FIG. When the range is determined and the display mode is the left turn mode (M3), the cameras C2, C3, C4, and C8 are selected as the cameras used for the overhead video composition, and the camera video allocation is determined as shown in FIG. The

  For example, when the display range is determined as shown in FIG. 9A and the display mode is the straight-ahead mode (M1), for example, the cameras C1, C3, C5, and C7 are selected as the cameras used for the synthesis of the overhead view video. The camera having the highest fitness value of each segment may be determined as camera video allocation as shown in FIG. 9B.

  As described above, according to the present invention, in an apparatus for synthesizing one overhead image from a plurality of camera images, the number of cameras to be used can be reduced by efficiently allocating the images of the cameras used for the synthesis of the overhead image. It is possible to automatically display an optimum overhead view image so that the resolution of the image does not decrease even if the number increases.

  The configuration described in the above embodiment merely shows a specific example, and does not limit the technical scope of the present invention. Any configuration can be employed within the scope of the effects of the present invention. For example, the camera selection determination unit 130 may select and determine a camera without using the fitness map. Further, the camera video assignment determination unit 150 may determine the assignment without using the fitness map.

  For example, instead of preparing a fitness map in advance, calculate the fitness value of the camera for each part or each pixel of the overhead view image to be synthesized from the installation position and direction of each camera, the viewing angle, etc. You may use for camera selection determination or camera image allocation determination. In this case, there is no need to prepare a fitness map in advance, which makes it easier to respond to changes in the camera position and orientation, but the processing amount is increased compared to the method using the fitness map, and details for each display mode are required. Tuning can be difficult.

  In addition, although this embodiment demonstrated the example applied to the vehicle-mounted system, for example, in addition to the installation system on an airplane or a ship, the monitoring system of a building, etc., a plurality of camera videos (real-time video or recorded video) are input. Can be applied to various systems that synthesize and display and record video.

  In the embodiment of the present invention, a software program that realizes the above-described embodiment (in the embodiment, a program corresponding to the flowchart shown in the figure) is supplied to the apparatus, and the computer of the apparatus supplies the program It is also possible to achieve this by reading and executing. Therefore, in order to realize the functional processing described in this embodiment by a computer, the program itself installed in the computer is also an embodiment of the present invention. That is, a program for realizing the functional processing of the present invention is also included in one aspect of the embodiment. A medium storing the program is also included in one aspect of the embodiment.

DESCRIPTION OF SYMBOLS 100 Image composition apparatus 110 Input information reception part 120 Display range determination part 130 Camera selection determination part 140 Conformity map memory | storage part 150 Camera image allocation determination part 160 Input image selection part 160
170 Video composition unit 170
180 Display C1-Cm Camera

Claims (6)

A video composition device that synthesizes a single bird's-eye view video from a plurality of camera images,
An input information receiving unit for receiving input information;
A display range determination unit that determines a display range of the overhead view video based on the input information received by the input information reception unit;
A camera selection determining unit that selects and determines a predetermined number of cameras that shoot a video used to synthesize an overhead video of the display range based on the display range of the overhead video determined by the display range determination unit;
A camera video assignment determining unit that determines which camera video is to be assigned to overlapping portions in a predetermined number of camera videos determined by the camera selection determining unit;
A video synthesis unit that synthesizes and outputs the overhead view video using videos taken by a predetermined number of cameras determined by the camera selection determination unit based on the allocation of the camera video allocation determination unit;
The camera selection determination unit
Access to a camera fitness data storage unit that stores camera fitness data determined for each camera for each part of the bird's-eye view video, which is a degree of suitability for use of the camera video,
The fitness value in the display range obtained by calculating the fitness value in the display range for each camera by reading out the fitness value of each part in the display range of the overhead view video from the camera fitness data storage unit. A video composition device that selects and determines a predetermined number of cameras in order from the camera with the highest value as a camera to be used for composition of the overhead view video. The camera selection decision unit, for each segment obtained by dividing an area for synthesizing the overhead Film image, calculates the fitness value, using the fitness values of segments included in the display range of the overhead Film image, wherein The video composition apparatus according to claim 1, wherein a camera to be used for composition of the overhead view video is selected and determined. The camera image assignment determination unit
Access to a camera fitness data storage unit that stores camera fitness data determined for each camera for each part of the bird's-eye view video, which is a degree of suitability for use of the camera video,
Of the predetermined number of camera images determined by the camera selection determining unit, the overlapping portion of the video of the camera having the highest fitness value for the overlapping portion of the video of each camera in the overhead view image is The video composition apparatus according to claim 1, which is assigned to the synthesis of the overhead view video. The display range determination unit
Based on the input information received by the input information receiving unit, determine the display mode,
The camera suitability data has data for each display mode,
The camera selection determination unit selects and determines a camera using camera suitability data corresponding to the display mode determined by the display range determination unit, or the camera image allocation determination unit includes the display range determination unit. The video composition device according to claim 2 or 3, wherein the camera composition data corresponding to the determined display mode is used to synthesize the overhead video.   The camera suitability data has data for each display mode, and when there is an intermediate third display mode for switching from the first display mode to the second display mode, the conformity of the first display mode is present. The new fitness data is created by using an average value of a value weighted to the degree value and a value weighted to the fitness value of the second display mode as an intermediate fitness value of the third display mode. Item 5. The video composition device according to Item 4. A video composition program for synthesizing a single bird's-eye view video from a plurality of camera images,
On the computer,
An input information receiving step for receiving input information;
A display range determination step for determining a display range of the overhead view video based on the input information received in the input information reception step;
Based on the display range of the overhead view video determined in the display range determination step, a camera selection determination step of selecting and determining a predetermined number of cameras for shooting the video used for the synthesis of the overhead view video of the display range;
Camera video assignment determination step for determining which camera video to be assigned to overlapping portions in a predetermined number of camera videos determined in the camera selection determination step;
Based on the allocation in the camera video allocation step, using the video captured by the predetermined number of cameras determined in the camera selection determination step, to perform a video synthesis step of combining and displaying the overhead view video,
The camera selection determination step includes
Access the camera fitness data storage unit that stores the camera fitness data determined for each camera for each part of the overhead view video, a fitness value that is a degree that the use of the camera video is suitable,
The fitness value in the display range obtained by calculating the fitness value in the display range for each camera by reading out the fitness value of each part in the display range of the overhead view video from the camera fitness data storage unit. A video synthesizing program including processing for selecting and determining a predetermined number of cameras in order from the camera with the highest value as a camera to be used for synthesizing the overhead view video.

Images