JP2020166876A - Image generation system, image generation method, and program - Google Patents

Abstract

To provide an image generation system, an image generation method, and a program with which it is possible to generate an image obtained by combining a captured image obtained by capturing real space and an image of virtual space with a low processing load.SOLUTION: A captured image storage part 100 associates and stores captured images with positions of a virtual vehicle in virtual space, the captured images being captured by a camera provided in a real railway vehicle that moves in real space. A virtual vehicle position update part 106 updates the position of the virtual vehicle in the virtual space in accordance with a user operation. A virtual space image generation part 110 generates a virtual space image representing appearance of the virtual space as seen from a virtual space viewpoint position corresponding to the updated position of the virtual vehicle. A captured image acquisition part 112 acquires, from the captured image storage part 100, one or more captured images that are identified on the basis of the updated position of the virtual vehicle. A composite image generation part 114 generates a composite image of the generated virtual space image and the acquired one or more captured images.SELECTED DRAWING: Figure 15

Description

The present invention relates to an image generation system, an image generation method and a program.

A railway simulator is known that can generate an image obtained by synthesizing a live-action image of a real space and an image of computer graphics (CG) such as an image of a virtual space. As an example of such a railway simulator, Patent Document 1 describes a railway simulator capable of generating a composite image in which a live-action moving image data of an actual scene seen from the driver's seat of a moving train and a three-dimensional CG model are combined. Is described.

In the technique described in Patent Document 1, the position on the screen of the feature point specified by inter-frame tracking from the live-action moving image data captured in advance and the three-dimensional information of the feature point, which is not the distance from the virtual viewpoint, are used. Distance information is calculated. Then, a composite image is generated by synthesizing the image of the three-dimensional CG model generated based on the calculated distance information and the reproduced video of the live-action moving image data.

Japanese Unexamined Patent Publication No. 2008-15576

However, in the technique described in Patent Document 1, in generating a composite image, various arithmetic processes such as identification of the position of a feature point on the screen, calculation of distance information, and generation of a CG model image based on the distance information are executed. It is necessary to do so, and the processing load is heavy.

The present invention has been made in view of the above problems, and one of the objects thereof is an image generation system capable of generating an image in which a live-action image taken in a real space and an image in a virtual space are combined with a light processing load. The purpose is to provide an image generation method and a program.

The image generation system according to the present invention is a live image storage means for storing a live image taken by a camera provided in a real railroad vehicle moving in the real space in association with a virtual vehicle position in the virtual space, and a user. The virtual space was viewed from the virtual vehicle position updating means for updating the virtual vehicle position in the virtual space and the position of the viewpoint in the virtual space corresponding to the updated virtual vehicle position in response to the operation of. A virtual space image generation means for generating a virtual space image representing a state, and a live image acquisition means for acquiring one or a plurality of the live images specified based on the updated virtual vehicle position from the live image storage means. , A composite image generation means for generating a composite image of the generated virtual space image and one or a plurality of the acquired live images.

In one aspect of the present invention, the live image storage means stores a combination of a plurality of the live images taken by the plurality of cameras provided in the real railroad vehicle in association with the virtual vehicle position. The virtual space image generation means viewed the virtual space from the position of the viewpoint with respect to a plurality of viewpoints in the virtual space corresponding to the camera corresponding to the updated virtual vehicle position. The virtual space image representing the state is generated, and the live image acquisition means acquires one or a plurality of the combinations specified based on the updated virtual vehicle position from the live image storage means, and obtains the composite. The image generation means is included in one or a plurality of the acquired virtual space images, which represent a state in which the virtual space is viewed from the position of the viewpoint corresponding to the cameras for each of the plurality of cameras. Generates a composite image with one or more of the live images taken by the camera.

In this aspect, a driver's console on which a simulated driving operation is performed by a driver trainer, a simulated railroad vehicle on which a conductor trainer can get on and off, a simulated platform provided adjacent to the simulated railroad vehicle, and the like. Combining one or more live images taken by the camera located in front of the actual railroad vehicle and facing forward with the virtual space image showing the state of viewing the virtual space from the position of the viewpoint corresponding to the camera. A display panel arranged in front of the driver's console for displaying images, and one or more live images taken by the camera located on the side of the actual railroad vehicle and facing forward, and the camera. A display panel provided in front of the simulated platform, which displays a composite image with the virtual space image showing how the virtual space is viewed from the corresponding viewpoint position, may be further included.

Alternatively, the image was taken by a simulated railroad vehicle on which a conductor trainer can get on and off, a simulated platform provided adjacent to the side surface of the simulated railroad vehicle, and the camera located behind the actual railroad vehicle and facing rearward. A display provided behind the simulated railroad vehicle that displays a composite image of one or more live images and the virtual space image showing how the virtual space is viewed from the position of the viewpoint corresponding to the camera. The virtual space representing the state of viewing the virtual space from the panel, one or more live images taken by the camera located on the side of the actual railroad vehicle and facing forward, and the position of the viewpoint corresponding to the camera. It may further include a display panel provided in front of the simulated platform that displays a composite image with the spatial image.

Further, in one aspect of the present invention, the composite image generation means repeatedly executes the generation of the composite image according to the operation of the user at predetermined time intervals.

Further, in the image generation method according to the present invention, from the step of updating the virtual vehicle position in the virtual space according to the operation of the user and the position of the viewpoint in the virtual space corresponding to the updated virtual vehicle position. A step of generating a virtual space image showing a state of seeing the virtual space, and a live image taken by a camera provided on a real railcar moving in the real space are used as the virtual vehicle position in the virtual space. A step of acquiring one or a plurality of the live images specified based on the updated virtual vehicle position from the live image storage means to be associated and stored, and one or a plurality of the virtual space images to be generated. A step of generating a composite image with the live image of the above.

Further, the program according to the present invention includes a procedure for updating a virtual vehicle position in a virtual space according to a user operation, and the virtual space from a viewpoint position in the virtual space corresponding to the updated virtual vehicle position. A procedure for generating a virtual space image showing a state of viewing, a live image taken by a camera provided in a real railroad vehicle moving in the real space is stored in association with the virtual vehicle position in the virtual space. A procedure for acquiring one or more of the live images specified based on the updated virtual vehicle position from the live image storage means, the generated virtual space image and the acquired one or more live images. Have the computer perform the procedure for generating a composite image of.

It is an external perspective view which shows an example of the whole structure of the railway simulator system which concerns on one Embodiment of this invention. It is a figure which shows an example of the structure of the simulated railroad vehicle and the driver's operation console. It is a figure which shows an example of the data structure of the live-action image management data. It is a figure which shows an example of a virtual space. It is explanatory drawing explaining an example of specifying the position and direction of a virtual railroad vehicle object. It is a figure which shows an example of the situation where a virtual person object is located in front of a virtual railroad vehicle object in a virtual space. It is a figure which shows an example of the forward virtual space image. It is a figure which shows an example of the forward live-action frame image. It is a figure which shows an example of the forward composite image. It is a figure which shows an example of the situation where a virtual person object is located on the right side of a virtual railroad vehicle object in a virtual space. It is a figure which shows an example of the forward composite image. It is a figure which shows an example of the virtual space image on the right side. It is a figure which shows an example of the right side live-action frame image. It is a figure which shows an example of the right-side composite image. It is a figure which shows an example of the situation where a virtual person object is located on the right side of a virtual railroad vehicle object in a virtual space. It is a figure which shows an example of the virtual space image on the right side. It is a figure which shows an example of the right side live-action frame image. It is a figure which shows an example of the right-side composite image. It is a figure which shows an example of the situation where a virtual person object is located behind a virtual railroad vehicle object in a virtual space. It is a figure which shows an example of the rear virtual space image. It is a figure which shows an example of the rear live-action frame image. It is a figure which shows an example of the rear composite image. It is a functional block diagram which shows an example of the function of the server which concerns on one Embodiment of this invention. It is a flow diagram which shows an example of the flow of the process performed in the server which concerns on one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 1 is an external perspective view showing an example of the overall configuration of a railway simulator system 1 according to an embodiment of the present invention.

The railway simulator system 1 illustrated in FIG. 1 is a system that imitates an actual railway vehicle (actual railway vehicle) that moves in a real space and is used for training the work of a conductor or a driver. Hereinafter, a person who trains the conductor's work will be referred to as a conductor trainer, and a person who trains the driver's work will be referred to as a driver trainer. The application of the railway simulator system 1 illustrated in FIG. 1 is not limited to the training of conductors and drivers. The railway simulator system 1 may be used, for example, for a simulated experience of a conductor or a driver.

As shown in FIG. 1, the railway simulator system 1 according to the present embodiment includes a simulated railway vehicle 10, a driver operation console 20, a conductor instructor desk 30, and a driver instructor desk 40 as main configurations. included.

The simulated railway vehicle 10 imitates a part of the rear part of the actual railway vehicle. The conductor trainer can get on and off the simulated railway vehicle 10. A simulated conductor room 11 is provided in the simulated railway vehicle 10, and a rear display panel P1 is provided behind the simulated conductor room 11.

Further, a left simulated platform 12 is provided adjacent to the left side surface of the simulated railway vehicle 10. A left side display panel P2 is provided in front of the left side simulated platform 12. Further, a right side simulated platform 13 is provided adjacent to the right side surface of the simulated railway vehicle 10. A right side display panel P3 is provided in front of the right side simulated platform 13.

The driver's console 20 imitates the console in the driver's cab of an actual railway vehicle. On the driver operation console 20, a simulated driving operation is performed by a driver trainer. The driver's operating console 20 is provided with a traveling command unit 21 including, for example, a lever and the like. Further, a front display panel P4 is provided in front of the driver's operation console 20.

Each of the simulated conductor room 11 and the driver's console 20 may be provided with a voice communication device for the driver trainer and the conductor trainer to communicate with each other.

Further, the simulated railway vehicle 10 and the driver's console 20 do not have to be provided in the same space, and may be provided in adjacent rooms separated by a wall, for example. Further, the simulated railway vehicle 10 and the driver's console 20 may be provided at locations separated from each other. As a result, even when the conductor trainer and the driver trainer are in remote locations, the training can be performed in cooperation with each other.

At the conductor instructor table 30, the conductor instructor monitors the conductor trainer. A plurality of display panels and the like are provided on the conductor instructor table 30, and the conductor instructor can check the display panels to monitor the training status of the conductor trainer.

At the driver instructor table 40, the driver instructor monitors the driver trainer and the like. A plurality of display panels and the like are provided on the driver instructor table 40, and the training status of the driver trainer can be monitored by the driver instructor confirming these display panels.

FIG. 2 is a diagram showing an example of the configuration of the simulated railway vehicle 10 and the driver's console 20. As shown in FIG. 2, the simulated railway vehicle 10 according to the present embodiment includes a server 50. Further, the driver operation console 20 according to the present embodiment includes a server 52.

The server 50 includes, for example, a processor 50a, a storage unit 50b, and a communication unit 50c. The processor 50a is, for example, a program control device such as a CPU that operates according to a program installed on the server 50. The storage unit 50b is a storage element such as a ROM or RAM, a hard disk drive, or the like. A program or the like executed by the processor 50a is stored in the storage unit 50b. The communication unit 50c is a communication interface such as a network board for exchanging data with the server 52, for example. The server 50 transmits / receives information to / from the server 52 via the communication unit 50c.

The server 52 includes, for example, a processor 52a, a storage unit 52b, and a communication unit 52c. The processor 52a is a program control device such as a CPU that operates according to a program installed on the server 52, for example. The storage unit 52b is a storage element such as a ROM or RAM, a hard disk drive, or the like. A program or the like executed by the processor 52a is stored in the storage unit 52b. The communication unit 52c is a communication interface such as a network board for exchanging data with the server 50, for example. The server 52 transmits / receives information to / from the server 50 via the communication unit 52c.

In the present embodiment, while the actual railway vehicle is traveling on the railway line in the real space to be trained using the railway simulator system 1, the moving image used for the training by the camera provided on the actual railway vehicle in advance. The statue is photographed.

Here, for example, it is assumed that a moving image is taken by four cameras provided in an actual railway vehicle. Hereinafter, these four cameras will be referred to as a rear real camera, a left side real camera, a right side real camera, and a front real camera, respectively. Here, the rear actual camera is, for example, a camera that is located behind the rearmost vehicle of the actual railway vehicle and faces the rear, and captures the rear from the conductor room. The left-side actual camera is, for example, a camera that is located on the left side of the conductor room of the rearmost vehicle of the actual railway vehicle and faces forward, and captures the front from the left side of the conductor room. The right-side actual camera is, for example, a camera that is located on the right side of the conductor room of the rearmost vehicle of the actual railway vehicle and faces forward, and captures the front from the right side of the conductor room. Further, the front actual camera is, for example, a camera that is located in front of the leading vehicle of the actual railway vehicle and faces forward, and captures the front from the driver's seat.

When moving images are taken by four cameras provided in an actual railway vehicle as described above, four moving images including a plurality of frame images are generated. Hereinafter, the moving image generated by the camera provided in the actual railway vehicle will be referred to as a live-action moving image. Then, for example, the live-action moving images generated by the rear real camera, the left-side real camera, the right-side real camera, and the front real camera are, respectively, the rear real-action image, the left-side real-action image, and the right-side real-action image. And, it will be called a forward live-action image.

In the present embodiment, it is assumed that the shooting timings of the rear live-action image, the left-side live-action image, the right-side live-action image, and the front live-action image are synchronized. Then, the same frame number is assigned to the frame image of the rear live-action image, the frame image of the left side live-action image, the frame image of the right side live-action image, and the frame image of the front live-action image taken at a certain timing. It shall be set.

Further, in the present embodiment, the position of the actual railway vehicle at the shooting timing of the live-action image, which is the frame image included in the live-action moving image, is specified in advance. Hereinafter, the position of the actual railway vehicle specified in this way will be referred to as the actual vehicle position. Here, for example, by using a GPS module or the like, when shooting each frame image included in the live-action moving image, the actual vehicle position at the shooting timing of the frame image may be specified. Here, the position of a point representing the actual railway vehicle, such as the position of the actual front camera or the position of the center of the entire actual railway vehicle having a plurality of cars, may be specified as the actual vehicle position.

Then, in the present embodiment, the live-action moving image taken as described above, the actual vehicle position at the shooting timing of the live-action moving image, and the like are managed by the live-action image management data (see FIG. 3). The live-action image management data may be stored in the storage unit 50b of the server 50, for example.

FIG. 3 is a diagram showing an example of a data structure of live-action image management data according to the present embodiment. As shown in FIG. 3, the live-action image management data includes the live-action image set ID, the rear live-action frame image, the left-side live-action frame image, the right-side live-action frame image, the front live-action frame image, the real vehicle position data, and the virtual vehicle position data. Is included. The live-action image management data is data associated with one frame image included in the live-action moving image. Therefore, for example, the storage unit 50b of the server 50 stores the live-action image management data of the number of frame images included in the live-action video (the number of frames of the live-action video).

The live-action image set ID is identification information of the live-action image management data. Here, for example, the frame number may be set as the value of the live-action image set ID.

The rear live-action frame image, the left side live-action frame image, the right-side live-action frame image, and the front live-action frame image included in the live-action image management data are frame images associated with the live-action image set ID included in the live-action image management data. A live-action image is set. Here, for example, as the rear live-action frame image, a live-action image which is a frame image included in the rear live-action image is set. Further, as the left-side live-action frame image, a live-action image which is a frame image included in the left-side live-action image is set. Further, as the right-side live-action frame image, a live-action image which is a frame image included in the right-side live-action image is set. Further, as the front live-action frame image, a live-action image which is a frame image included in the front live-action image is set. As described above, the live-action image management data according to the present embodiment includes a combination of a plurality of live-action images.

As the value of the actual vehicle position data included in the live-action image management data, for example, a value indicating the actual vehicle position at the shooting timing of the frame image associated with the live-action image management data is set. Here, as described above, a value indicating the position of a point representing the actual railway vehicle may be set as the value of the actual vehicle position data. Further, the value of the actual vehicle position data may be expressed by, for example, a combination of latitude and longitude.

As the value of the virtual vehicle position data included in the live-action image management data, the position in the virtual space 60 illustrated in FIG. 4 associated with the actual vehicle position indicated by the real vehicle position data included in the live-action image management data is used. The indicated value is set. Hereinafter, the position in the virtual space 60 associated with the actual vehicle position will be referred to as a virtual vehicle position. Here, the value of the virtual vehicle position data may be represented by, for example, a combination of the X coordinate value, the Y coordinate value, and the Z coordinate value.

FIG. 4 is a diagram showing an example of the virtual space 60 according to the present embodiment. The virtual space 60 shown in FIG. 4 is a three-dimensional virtual space. Further, as shown in FIG. 4, in the virtual space 60 according to the present embodiment, a virtual railway vehicle object 62 composed of a plurality of polygons, which is a virtual object associated with the actual railway vehicle, is arranged. Here, the virtual railroad vehicle object 62 according to the present embodiment is an invisible virtual object set to be transparent.

Here, for example, when the actual railway vehicle is a 6-car train, six virtual railway vehicle objects 62 (62a, 62b, 62c, 62d, 62e, and 62f) connected to each other are arranged in the virtual space 60. Will be done. The virtual railroad vehicle object 62a is associated with, for example, the leading vehicle of the real railroad vehicle. Further, the virtual rail vehicle object 62b, the virtual rail vehicle object 62c, the virtual rail vehicle object 62d, and the virtual rail vehicle object 62e are associated with, for example, the second, third, fourth, and fifth vehicles from the beginning of the actual rail vehicle, respectively. .. Further, the virtual railway vehicle object 62f is associated with, for example, the last vehicle of the actual railway vehicle.

Then, in the present embodiment, as described above, the actual vehicle position in the real space and the virtual vehicle position in the virtual space 60 are associated with each other on a one-to-one basis. Here, in the present embodiment, it is assumed that the track corresponding to the track of the actual railroad vehicle in the real space when the live-action moving image is taken is set in the virtual space 60 in advance. Then, in the present embodiment, as shown in FIG. 5, the positions and orientations of the six virtual railroad vehicle objects 62 are uniquely specified based on the virtual vehicle positions and the track.

FIG. 5 is an explanatory diagram illustrating an example of specifying the position and orientation of the virtual railroad vehicle object based on the virtual vehicle position and track. In FIG. 5, the track of the virtual railroad vehicle object 62 that gently turns to the right is shown by a chain double-dashed line. Further, in FIG. 5, the virtual railroad vehicle object 62 travels along the track from the bottom to the top in FIG.

Here, for example, it is assumed that the actual vehicle position is the position of the front actual camera. Then, it is assumed that the position A1 at the center of the front surface of the virtual railroad vehicle object 62a is the virtual vehicle position. In this case, the position A2 at the intersection of the circle and the track, which is located behind the position A1 with respect to the traveling direction of the virtual railway vehicle object 62 and has a radius length d centered on the position A1, is specified. Similarly, the position A3 at the intersection of the circle and the orbit, which is located behind the position A2 with respect to the above-mentioned traveling direction and has a radius length d centered on the position A2, is specified. Further, the position A4 at the intersection of the circle and the orbit, which is located behind the position A3 with respect to the above-mentioned traveling direction and has a radius length d centered on the position A3, is specified. Further, the position A5 of the intersection of the circle and the orbit, which is located behind the position A4 with respect to the above-mentioned traveling direction and has a radius length d centered on the position A4, is specified. Further, the position A6 of the intersection of the circle and the orbit, which is located behind the position A5 with respect to the above-mentioned traveling direction and has a radius length d centered on the position A5, is specified. Here, the length d of the radius is not particularly limited, but may be, for example, the length of one virtual railroad vehicle object 62, or the length of one virtual railroad vehicle object 62 plus the margin length. There may be.

Then, the virtual railroad vehicle object 62a is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62a is located at the position A1 and the normal direction of the front surface faces the tangential direction B1 of the track at the position A1. Further, the virtual railroad vehicle object 62b is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62b is located at the position A2 and the normal direction of the front surface faces the tangential direction B2 of the track at the position A2. Further, the virtual railroad vehicle object 62c is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62c is located at the position A3 and the normal direction of the front surface faces the tangential direction B3 of the track at the position A3. Further, the virtual railroad vehicle object 62d is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62d is located at the position A4 and the normal direction of the front surface faces the tangential direction B4 of the track at the position A4. Further, the virtual railroad vehicle object 62e is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62e is located at the position A5 and the normal direction of the front surface faces the tangential direction B5 of the track at the position A5. Further, the virtual railroad vehicle object 62a is arranged in the virtual space 60 so that the center of the front surface of the virtual railroad vehicle object 62f is located at the position A6 and the normal direction of the front surface at the position A6 faces the tangential direction B6 of the track. In this way, the positions and orientations of the six virtual rail vehicle objects 62 corresponding to the virtual vehicle positions are specified.

Further, in the virtual space 60, the position of the viewpoint (virtual camera) 64 associated with the camera provided in the actual railway vehicle and the direction of the line of sight (line of sight direction 66) at the viewpoint 64 are set.

FIG. 4 shows the positions of the viewpoint 64a, the viewpoint 64b, the viewpoint 64c, and the viewpoint 64d. Further, FIG. 4 shows the line-of-sight direction 66a, which is the direction of the line of sight at the viewpoint 64a, the line-of-sight direction 66b, which is the direction of the line of sight at the viewpoint 64b, the line-of-sight direction 66c, which is the direction of the line of sight at the viewpoint 64c, and the direction of the line of sight at the viewpoint 64d. The line-of-sight direction 66d is also shown.

The position of the viewpoint 64a, the position of the viewpoint 64b, the position of the viewpoint 64c, and the position of the viewpoint 64d in the virtual space 60 shown in FIG. 4 are the rear real camera, the left side real camera, the right side real camera, and the front real camera, respectively. It is associated with a position in real space. Further, the line-of-sight direction 66a, the line-of-sight direction 66b, the line-of-sight direction 66c, and the line-of-sight direction 66d shown in FIG. 4 are photographed in the real space of the rear real camera, the left side real camera, the right side real camera, and the front real camera, respectively. It is associated with the direction.

Therefore, in the present embodiment, as shown in FIG. 4, for example, the viewpoint 64a is arranged behind the virtual railway vehicle object 62f, and the line-of-sight direction 66a faces rearward. Further, for example, the viewpoint 64b is arranged on the left side of the virtual railway vehicle object 62f, and the line-of-sight direction 66b faces forward. Further, for example, the viewpoint 64c is arranged on the right side of the virtual railway vehicle object 62f, and the line-of-sight direction 66c faces forward. Further, for example, the viewpoint 64d is arranged in front of the virtual railway vehicle object 62a, and the line-of-sight direction 66d faces forward.

In the present embodiment, for example, the positions of the viewpoint 64a, the viewpoint 64b, and the viewpoint 64c are fixed relative to the position of the virtual railway vehicle object 62f. Further, with respect to the line-of-sight direction 66a, the line-of-sight direction 66b, and the line-of-sight direction 66c, the directions relative to the directions of the virtual railway vehicle object 62f are fixed. Further, for the viewpoint 64d, the position relative to the position of the virtual railway vehicle object 62a is fixed. Further, with respect to the line-of-sight direction 66d, the direction relative to the direction of the virtual railway vehicle object 62a is fixed.

Further, in the present embodiment, as described above, the positions and orientations of the six virtual railroad vehicle objects 62 are uniquely specified based on the virtual vehicle positions. Therefore, as shown in FIG. 5, the virtual vehicle also has the position of the viewpoint 64a, the position of the viewpoint 64b, the position of the viewpoint 64c, the position of the viewpoint 64d, the line of sight direction 66a, the line of sight direction 66b, the line of sight direction 66c, and the line of sight direction 66d. It will be uniquely specified based on the position (for example, the position A1 in FIG. 5).

As will be described later, virtual objects other than the virtual railroad vehicle object 62, such as a virtual person object 68 composed of a plurality of polygons (see FIGS. 6, 8, 11 and 13), are arranged in the virtual space 60. It may have been done.

In the present embodiment, an actual railway vehicle simulated in the railway simulator system 1 is performed in response to an operation on the traveling command unit 21 performed by a user such as a driver trainer during the training period using the railway simulator system 1. The actual vehicle position in. Then, the virtual vehicle position changes according to the change in the actual vehicle position, and the position and orientation of the virtual railroad vehicle object 62 in the virtual space 60 changes according to the change in the virtual vehicle position. In the present embodiment, the position and orientation of the virtual railroad vehicle object 62 change along the track set in the virtual space 60 according to the change in the virtual vehicle position.

Then, in the present embodiment, a combination of four frame images is generated at a predetermined frame rate (for example, at 1/60 second intervals) during the period during which the training using the railway simulator system 1 is performed. Then, each of the frame images included in the generated combination is displayed on the rear display panel P1, the left side display panel P2, the right side display panel P3, and the front display panel P4. In this way, in the present embodiment, the images are displayed on the rear display panel P1, the left side display panel P2, the right side display panel P3, and the front display panel P4 at a predetermined frame rate during the training period. The frame image is updated. Further, in the present embodiment, the moving image is displayed on each of the rear display panel P1, the left side display panel P2, the right side display panel P3, and the front display panel P4 during the training period. ..

Here, in the present embodiment, for example, on the rear display panel P1, a composite image obtained by synthesizing a rear live-action frame image and a virtual space image showing a state in which the line-of-sight direction 66a is viewed from the viewpoint 64a in the virtual space 60 is used as a frame image. The including moving image is displayed. Hereinafter, the virtual space image will be referred to as a rear virtual space image, and the composite image will be referred to as a rear composite image.

Further, on the left side display panel P2, a moving image including a composite image obtained by synthesizing a left side live-action frame image and a virtual space image showing a state in which the line-of-sight direction 66b is viewed from the viewpoint 64b in the virtual space 60 is displayed as a frame image. Will be done. Hereinafter, the virtual space image will be referred to as a left-side virtual space image, and the composite image will be referred to as a left-side composite image.

Further, on the right side display panel P3, a moving image including a composite image obtained by synthesizing a right side live-action frame image and a virtual space image showing a state in which the line-of-sight direction 66c is viewed from the viewpoint 64c in the virtual space 60 is displayed as a frame image. Will be done. Hereinafter, the virtual space image will be referred to as a right-side virtual space image, and the composite image will be referred to as a right-side composite image.

Further, on the front display panel P4, a moving image including a composite image obtained by synthesizing a front live-action frame image and a virtual space image showing a state in which the line-of-sight direction 66d is viewed from the viewpoint 64d in the virtual space 60 is displayed as a frame image. .. Hereinafter, the virtual space image will be referred to as a front virtual space image, and the composite image will be referred to as a front composite image.

The virtual space image according to the present embodiment is a computer graphics (CG) image obtained by rendering a state of viewing the virtual space 60.

Hereinafter, an example of generating a composite image will be described with reference to FIGS. 6 to 10C.

For example, at a certain time point (time point t1), as shown in FIG. 6, it is assumed that the virtual person object 68 is located in front of the virtual railroad vehicle object 62a in the virtual space 60. In FIG. 6, the track of the virtual railway vehicle object 62 is shown by a chain double-dashed line. Here, for example, it is assumed that when the live-action moving image associated with the virtual space 60 shown in FIG. 6 is taken, the real railroad vehicle gently follows a 1021 turning track in the real space. In this case, as shown in FIG. 6, the track of the virtual railroad vehicle object 62 in the virtual space 60 is a track that gently turns to the right.

Then, based on the virtual vehicle position in this situation, the position of the viewpoint 64a, the position of the viewpoint 64b, the position of the viewpoint 64c, the position of the viewpoint 64d, the line of sight direction 66a, the line of sight direction 66b, the line of sight direction 66c, and the line of sight direction 66d are specified. Will be done. Then, a combination of the rear virtual space image, the left side virtual space image, the right side virtual space image, and the front virtual space image is generated based on the position of the specified viewpoint 64 and the line-of-sight direction 66. FIG. 7A shows an example of the forward virtual space image 70 generated in this situation.

Then, the live-action image management data in which the virtual vehicle position in this situation is set as the value of the virtual vehicle position data is acquired. Then, a combination of the rear live-action frame image, the left side live-action frame image, the right side live-action frame image, and the front live-action frame image included in the live-action image management data is acquired. FIG. 7B shows an example of the forward live-action frame image 72 included in the combination acquired in this situation.

Then, the rear composite image is generated by synthesizing the generated rear virtual space image and the rear live-action frame image included in the acquired combination. Further, the left side composite image is generated by synthesizing the generated left side virtual space image and the left side live-action frame image included in the acquired combination. Further, a right-side composite image is generated by synthesizing the generated right-side virtual space image and the right-side live-action frame image included in the acquired combination. Further, the front composite image is generated by synthesizing the generated front virtual space image and the front live-action frame image included in the acquired combination. FIG. 7C shows an example of the front composite image 74 generated by synthesizing the front virtual space image 70 shown in FIG. 7A and the front live-action frame image 72 shown in FIG. 7B.

It is conceivable that there is no live-action image management data in which the virtual vehicle position in this situation is set as the value of the virtual vehicle position data. In such a case, a plurality of live-action image management data specified based on the virtual vehicle position in this situation may be acquired. Then, a composite image obtained by synthesizing the image obtained by interpolating the live-action image included in each of the plurality of live-action image management data and the virtual space image may be generated.

For example, live-action image management data including virtual vehicle position data indicating the position closest to the virtual vehicle position for each of the front and rear along the above-mentioned track of the virtual vehicle position in this situation may be acquired. .. Then, a rear composite image may be generated by synthesizing the image obtained by interpolating the rear live frame image included in each of the two live image management data acquired in this way and the generated rear virtual space image. .. Further, the left-side composite image may be generated by synthesizing the image obtained by interpolating the left-side live-action frame image included in each of the two acquired live-action image management data and the generated left-side virtual space image. Further, the right-side composite image may be generated by synthesizing the image obtained by interpolating the right-side live-action frame image included in each of the two acquired live-action image management data and the generated right-side virtual space image. Further, the front composite image may be generated by synthesizing the image obtained by interpolating the front live frame image included in each of the two acquired live image management data and the generated front virtual space image.

Then, the rear composite image, the left composite image, the right composite image, and the front composite image generated in this manner are the rear display panel P1, the left side display panel P2, the right side display panel P3, and the front composite image, respectively. It is displayed on the front display panel P4.

After that, it is assumed that the virtual railway vehicle object 62 moves to the position shown in FIG. 8 at the time point t2 after the time point t1. In the situation shown in FIG. 8, the position of the virtual person object 68 in the virtual space 60 has not changed, but the virtual person object 68 is positioned on the right side of the virtual railroad vehicle object 62f due to the movement of the virtual railroad vehicle object 62. are doing.

FIG. 9 shows an example of the forward composite image 76 generated in the situation at time point t2. As shown in FIG. 8, in the situation at the time point t2, the virtual person object 68 is not included in the visual field range of the viewpoint 64d. Therefore, the front composite image 76 shown in FIG. 9 does not include the image of the virtual person object 68.

FIG. 10A shows an example of the right-side virtual space image 78 generated in the situation at time point t2. Further, FIG. 10B shows an example of the right-side live-action frame image 80 included in the combination acquired in the situation at time point t2. In the situation at time point t2, the virtual person object 68 is included in the visual field range of the viewpoint 64c. Therefore, the image of the virtual person object 68 is included in the right-side virtual space image 78 shown in FIG. 10A. Then, by synthesizing the right-side virtual space image 78 shown in FIG. 10A and the right-side live-action frame image 80 shown in FIG. 10B, the right-side composite image 82 shown in FIG. 10C is generated. The right-side composite image 82 generated in this way is displayed on the right-side display panel P3.

As described above, in the present embodiment, the image of the virtual person object 68 displayed on the front display panel P4 at the time point t1 is displayed on the right side display panel P3 while not being displayed on the front display panel P4 at the time point t2. It will be possible to make it.

Hereinafter, another example of generating a composite image will be described with reference to FIGS. 11 to 14C.

For example, at a certain time point (time point t3), as shown in FIG. 11, it is assumed that the virtual person object 68 is located on the right side of the virtual railroad vehicle object 62e in the virtual space 60. In FIG. 11, the track of the virtual railway vehicle object 62 is shown by a chain double-dashed line. Here, for example, it is assumed that when the live-action moving image associated with the virtual space 60 shown in FIG. 11 is taken, the real railroad vehicle travels on a track that gently turns to the right in the real space. In this case, as shown in FIG. 11, the track of the virtual railroad vehicle object 62 in the virtual space 60 is a track that gently turns to the right.

FIG. 12A shows an example of the right side virtual space image 84 generated in the situation at time point t3. Further, FIG. 12B shows an example of the right-side live-action frame image 86 included in the combination acquired in the situation at the time point t3. As shown in FIG. 11, in the situation at the time point t3, the virtual person object 68 is included in the visual field range of the viewpoint 64c. Therefore, the image of the virtual person object 68 is included in the right-side virtual space image 84 shown in FIG. 12A. Then, by synthesizing the right-side virtual space image 84 shown in FIG. 12A and the right-side live-action frame image 86 shown in FIG. 12B, the right-side composite image 88 shown in FIG. 12C is generated. The right-side composite image 88 generated in this way is displayed on the right-side display panel P3.

Then, it is assumed that the virtual railway vehicle object 62 moves to the position shown in FIG. 13 at the time point t4, which is later than the time point t3. In the situation shown in FIG. 13, the position of the virtual person object 68 in the virtual space 60 has not changed from the time point t3, but the virtual person object 68 is behind the virtual railroad vehicle object 62f due to the movement of the virtual railroad vehicle object 62. Is located in.

FIG. 14A shows an example of the rear virtual space image 90 generated in the situation at time point t4. Further, FIG. 14B shows an example of the rear-view live-action frame image 92 included in the combination acquired in the situation at the time point t4. In the situation at the time point t4, the virtual person object 68 is not included in the visual field range of the viewpoint 64c, but the virtual person object 68 is included in the visual field range of the viewpoint 64a. Therefore, the rear virtual space image 90 shown in FIG. 14A includes an image of the virtual person object 68. Then, by synthesizing the rear virtual space image 90 shown in FIG. 14A and the rear live-action frame image 92 shown in FIG. 14B, the rear composite image 94 shown in FIG. 14C is generated. The rear composite image 94 generated in this way is displayed on the rear display panel P1.

As described above, in the present embodiment, the image of the virtual person object 68 displayed on the right side display panel P3 at the time point t3 is not displayed on the right side display panel P3 at the time point t4, and is displayed on the rear display panel P1. It will be possible to display it.

In the present embodiment, it is not necessary to manually superimpose CG on the live-action image when generating the composite image. Further, in the present embodiment, in generating the composite image, it is not necessary to perform heavy processing such as arithmetic processing for determining the position where the computer graphics (CG) is arranged by image analysis of the live-action image. Therefore, according to the present embodiment, it is possible to generate an image in which a live-action image and a virtual space image are combined with a light processing load without any trouble.

Further, in the present embodiment, by modeling a virtual object such as the virtual person object 68 and arranging it in the virtual space 60, the arrangement of the image of the virtual object for all the images displayed on the plurality of display panels is determined. The object. Therefore, according to the present embodiment, it is possible to easily display a moving image including an image obtained by synthesizing a live-action image and a virtual space image in a state of being consistent with each other on a plurality of display panels.

A live-action image of the real space taken by the camera may be mapped as a texture on the surface of the polygons constituting the virtual person object 68. In this way, a composite image closer to a live-action image can be generated.

Hereinafter, the functions of the server 50 according to the present embodiment and the processing executed by the server 50 according to the present embodiment will be further described.

FIG. 15 is a functional block diagram showing an example of the functions implemented in the server 50 according to the present embodiment. It should be noted that the server 50 according to the present embodiment does not have to implement all of the functions shown in FIG. 15, and may have functions other than the functions shown in FIG.

As shown in FIG. 15, functionally, the server 50 according to the present embodiment includes a live-action image storage unit 100, a virtual space data storage unit 102, a traveling signal reception unit 104, a virtual vehicle position update unit 106, and a viewpoint. The position specifying unit 108, the virtual space image generation unit 110, the live-action image acquisition unit 112, the composite image generation unit 114, the composite image output unit 116, and the composite image transmission unit 118 are included. The live-action image storage unit 100 and the virtual space data storage unit 102 are mainly mounted with the storage unit 50b. The traveling signal receiving unit 104 and the composite image transmitting unit 118 are mainly mounted with the communication unit 50c. The virtual vehicle position update unit 106, the viewpoint position identification unit 108, the virtual space image generation unit 110, the live-action image acquisition unit 112, and the composite image generation unit 114 are mainly mounted with the processor 50a. The composite image output unit 116 mainly mounts the processor 50a, the rear display panel P1, the left side display panel P2, and the right side display panel P3. Then, the server 50 plays a role as an image generation system for a railway simulator that generates a composite image according to the present embodiment.

The above functions may be implemented by executing a program installed on the server 50, which is a computer, including commands corresponding to the above functions on the processor 50a. This program may be supplied to the server 50 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like.

In the present embodiment, the live-action image storage unit 100 stores, for example, a live-action image taken by a camera provided in a real railcar moving in the real space in association with a virtual vehicle position in the virtual space 60. Here, the live-action image storage unit 100 may store a combination of a plurality of live-action images taken by a plurality of cameras provided in the real railcar in association with a virtual vehicle position in the virtual space 60. For example, as described above, the live-action image storage unit 100 may store a plurality of the above-mentioned live-action image management data including a combination of live-action images taken at the same timing.

In the present embodiment, the virtual space data storage unit 102 stores data indicating the position and orientation of each virtual object in the virtual space 60, for example. Further, the virtual space data storage unit 102 may store polygon data representing the virtual object, a texture image mapped to the virtual object, and the like. Further, the virtual space data storage unit 102 may store data indicating an orbit set in the virtual space 60. Further, the virtual space data storage unit 102 may store data indicating the position and orientation of the virtual railroad vehicle object 62 with respect to the virtual vehicle position. Further, the virtual space data storage unit 102 may store data indicating the relative positions of the viewpoint 64a, the viewpoint 64b, and the viewpoint 64c with respect to the position of the virtual railroad vehicle object 62f. Further, the virtual space data storage unit 102 may store data indicating the directions of the line-of-sight direction 66a, the line-of-sight direction 66b, and the line-of-sight direction 66c relative to the direction of the virtual railroad vehicle object 62f. Further, the virtual space data storage unit 102 may store data indicating the position of the viewpoint 64d relative to the position of the virtual railroad vehicle object 62a. Further, the virtual space data storage unit 102 may store data indicating a direction relative to the direction of the virtual railway vehicle object 62a in the line-of-sight direction 66d.

In the present embodiment, the travel signal receiving unit 104 receives, for example, a travel signal corresponding to an operation input to the travel command unit 21. Here, for example, the travel command unit 21 may output a travel signal to the server 52 at a predetermined sampling rate, and the server 52 may transmit the travel signal to the server 50. Here, the time interval corresponding to the sampling rate may be the same as the time interval corresponding to the above-mentioned frame rate.

In the present embodiment, the virtual vehicle position update unit 106 updates the virtual vehicle position in the virtual space 60 according to, for example, a user operation. Here, for example, the virtual vehicle position update unit 106 may update the virtual vehicle position in the virtual space 60 according to the travel signal corresponding to the operation input received by the travel signal receiving unit 104.

In the present embodiment, the viewpoint position specifying unit 108 specifies, for example, the position of the viewpoint 64 corresponding to the updated virtual vehicle position. Here, for example, the viewpoint position specifying unit 108 may specify the position and orientation of the virtual railroad vehicle object 62 based on the virtual vehicle position updated by the virtual vehicle position updating unit 106. Then, the viewpoint position specifying unit 108 may specify the position of the viewpoint 64a, the position of the viewpoint 64b, the position of the viewpoint 64c, and the position of the viewpoint 64d based on the position and the orientation of the specified virtual railway vehicle object 62. Further, for example, the viewpoint position specifying unit 108 may specify the line-of-sight direction 66a, the line-of-sight direction 66b, the line-of-sight direction 66c, and the line-of-sight direction 66d based on the position and orientation of the specified virtual railway vehicle object 62. ..

In the present embodiment, the virtual space image generation unit 110 generates, for example, a virtual space image representing a state in which the virtual space 60 is viewed from the position of the viewpoint 64. For example, a virtual space image showing a state in which the virtual space 60 is viewed from the position of the viewpoint 64 corresponding to the updated virtual vehicle position is generated. Here, for example, as described above, a combination of the rear virtual space image, the left side virtual space image, the right side virtual space image, and the front virtual space image may be generated.

In the present embodiment, the live-action image acquisition unit 112 acquires, for example, one or a plurality of live-action images specified based on the position of the viewpoint 64 from the live-action image storage unit 100. For example, one or a plurality of live-action images specified based on the position of the viewpoint 64 corresponding to the updated virtual vehicle position are acquired from the live-action image storage unit 100. Here, for example, even if the live-action image acquisition unit 112 acquires one or a plurality of live-action image management data specified based on the virtual vehicle position in the virtual space 60 from the live-action image storage unit 100 as described above. Good.

In the present embodiment, the composite image generation unit 114 generates, for example, a composite image of the virtual space image generated by the virtual space image generation unit 110 and one or more live-action images acquired by the live-action image acquisition unit 112. For example, for each of the plurality of cameras, a virtual space image showing a state in which the virtual space 60 is viewed from the position of the viewpoint 64 corresponding to the camera, and one or one taken by the camera included in the acquired live-action image management data. A composite image of a plurality of live-action images may be generated. Here, for example, as described above, a combination of the rear composite image, the left composite image, the right composite image, and the front composite image may be generated. As described above, the composite image generation unit 114 may generate an image obtained by interpolating a plurality of live-action images acquired by the live-action image acquisition unit 112. Then, the composite image generation unit 114 may generate a composite image of the virtual space image generated by the virtual space image generation unit 110 and the interpolated image.

In the present embodiment, the composite image output unit 116 outputs, for example, the composite image generated by the composite image generation unit 114 to the display panel. Here, for example, the composite image output unit 116 may output the rear composite image generated by the composite image generation unit 114 to the rear display panel P1. In this case, the rear display panel P1 may display the rear composite image. Further, for example, the composite image output unit 116 may output the left-side composite image generated by the composite image generation unit 114 to the left-side display panel P2. In this case, the left side display panel P2 may display the left side composite image. Further, for example, the composite image output unit 116 may output the right-side composite image generated by the composite image generation unit 114 to the right-side display panel P3. In this case, the right side display panel P3 may display the right side composite image.

In the present embodiment, the composite image transmission unit 118 transmits, for example, the composite image generated by the composite image generation unit 114 to the server 52. Here, the composite image transmission unit 118 may transmit the forward composite image generated by the composite image generation unit 114 to the server 52. Then, the server 52 that has received the front composite image may output the front composite image to the front display panel P4. In this case, the front display panel P4 may display the front composite image.

In the present embodiment, for example, even if the virtual vehicle position update unit 106 changes the position or posture of the virtual person object 68 based on the position of the virtual person object 68 arranged in the virtual space 60 and the virtual vehicle position. Good. Specifically, for example, the position or posture of the virtual person object 68 may be controlled so as to behave like a person in contact with a vehicle. Then, the virtual space image generation unit 110 may generate a virtual space image including an image of the virtual person object 68 whose position or posture changes in this way.

Hereinafter, an example of the flow of processing repeatedly performed at a predetermined frame rate in the server 50 according to the present embodiment will be described with reference to the flow diagram illustrated in FIG.

First, the travel signal receiving unit 104 receives a travel signal from the server 52 in response to an operation input to the travel command unit 21 (S101).

Then, the virtual vehicle position update unit 106 determines the actual vehicle position and the virtual vehicle position in the frame based on the travel signal received in the process shown in S101 and the actual vehicle position and the virtual vehicle position in the immediately preceding frame (S102). ).

Then, the viewpoint position specifying unit 108 specifies the position and orientation of the virtual railroad vehicle object 62 based on the virtual vehicle position determined by the process shown in S102 (S103).

Then, the viewpoint position specifying unit 108 specifies the position of the viewpoint 64 and the line-of-sight direction 66 based on the position and orientation of the virtual railroad vehicle object 62 specified by the process shown in S103 (S104). Here, for example, the position of the viewpoint 64a, the position of the viewpoint 64b, the position of the viewpoint 64c, the position of the viewpoint 64d, the line-of-sight direction 66a, the line-of-sight direction 66b, the line-of-sight direction 66c, and the line-of-sight direction 66d may be specified.

Then, the virtual space image generation unit 110 generates a virtual space image based on the position of the viewpoint 64 and the line-of-sight direction 66 specified by the process shown in S104 (S105). Here, for example, as described above, a combination of the rear virtual space image, the left side virtual space image, the right side virtual space image, and the front virtual space image may be generated.

Then, the live-action image acquisition unit 112 acquires one or a plurality of live-action image management data including the virtual vehicle position data indicating the virtual vehicle position specified by the process shown in S102 (S106).

Then, the composite image generation unit 114 generates a composite image based on the live-action image included in the live-action image management data acquired by the process shown in S106 and the virtual space image generated by the process shown in S105 ( S107).

Here, for example, as described above, the rear composite image is based on the rear live-action frame image included in the live-action image management data acquired in the process shown in S106 and the rear virtual space image generated in the process shown in S105. It may be generated. Further, for example, a left-side composite image is generated based on the left-side live-action frame image included in the live-action image management data acquired by the process shown in S106 and the left-side virtual space image generated by the process shown in S105. May be good. Further, for example, a right-side composite image is generated based on the right-side live-action frame image included in the live-action image management data acquired by the process shown in S106 and the right-side virtual space image generated by the process shown in S105. May be good. Further, for example, a front composite image may be generated based on the front live-action frame image included in the live-action image management data acquired by the process shown in S106 and the front virtual space image generated by the process shown in S105.

Then, the composite image output unit 116 outputs the composite image, and the composite image transmission unit 118 transmits the composite image (S108). Here, for example, the composite image output unit 116 outputs the rear composite image to the rear display panel P1, outputs the left composite image to the left display panel P2, and outputs the right composite image to the right display panel P3. You may. In this case, as described above, the rear display panel P1 displays the rear composite image, the left side display panel P2 displays the left side composite image, and the right side display panel P3 displays the right side composite image and displays the front side. Panel P4 will display the front composite image. Further, the composite image transmission unit 118 may transmit the forward composite image to the server 52. Then, the server 52 may output the front composite image to the front display panel P4. In this case, as described above, the front display panel P4 displays the front composite image. Then, the process returns to the process shown in S101.

In this way, in this processing example, the processing shown in S101 to S108 is repeatedly executed at a predetermined frame rate. In this way, the composite image generation unit 114 may repeatedly generate the composite image according to the user's operation at predetermined time intervals.

The present invention is not limited to the above-described embodiment.

For example, some or all of the functions shown in FIG. 15 may be implemented by the server 52.

Further, for example, the railway simulator system 1 illustrated in FIG. 1 may not include the server 52. In this case, the server 50 may output the front composite image to the front display panel P4.

Further, for example, the railway simulator system 1 illustrated in FIG. 1 may not include the server 50. In this case, the server 52 may implement the function shown in FIG. Then, the server 52 may output the rear composite image to the rear display panel P1, output the left composite image to the left display panel P2, and output the right composite image to the right display panel P3.

Further, the above-mentioned specific character strings and numerical values, and specific character strings in the drawings are examples, and are not limited to these character strings and numerical values.

1 Railroad simulator system, 10 simulated railroad vehicle, 11 simulated driver's room, 12 left simulated platform, 13 right simulated platform, 20 driver's console, 21 driving command unit, 30 driver's instructor's desk, 40 driver's instructor's desk, 50 servers, 50a processor, 50b storage unit, 50c communication unit, 52 server, 52a processor, 52b storage unit, 52c communication unit, 60 virtual space, 62, 62a, 62b, 62c, 62d, 62e, 62f virtual railroad vehicle object, 64, 64a , 64b, 64c, 64d viewpoint, 66, 66a, 66b, 66c, 66d Line-of-sight direction, 68 virtual person object, 70 front virtual space image, 72 front live frame image, 74 front composite image, 76 front composite image, 78 right side Virtual space image, 80 Right side composite image, 82 Right side composite image, 84 Right side virtual space image, 86 Right side live frame image, 88 Right side composite image, 90 Rear virtual space image, 92 Rear live frame image, 94 Rear composite image, 100 live image storage unit, 102 virtual space data storage unit, 104 running signal reception unit, 106 virtual vehicle position update unit, 108 viewpoint position identification unit, 110 virtual space image generation unit, 112 live image acquisition unit, 114 Composite image generation unit, 116 composite image output unit, 118 composite image transmission unit, P1 rear display panel, P2 left side display panel, P3 right side display panel, P4 front display panel.

Claims (7)

A live-action image storage means that stores a live-action image taken by a camera installed in a real railcar moving in a real space in association with a virtual vehicle position in the virtual space.
A virtual vehicle position updating means for updating the virtual vehicle position in the virtual space according to a user operation, and
A virtual space image generation means for generating a virtual space image showing a state in which the virtual space is viewed from a position of a viewpoint in the virtual space corresponding to the updated virtual vehicle position.
A live-action image acquisition means for acquiring one or a plurality of the live-action images specified based on the updated virtual vehicle position from the live-action image storage means, and
A composite image generation means for generating a composite image of the generated virtual space image and one or a plurality of the acquired live-action images.
An image generation system characterized by including. The live-action image storage means stores a combination of a plurality of the live-action images taken by the plurality of cameras provided in the real railcar in association with the virtual vehicle position.
The virtual space image generation means viewed the virtual space from the position of the viewpoint with respect to a plurality of viewpoints in the virtual space corresponding to the camera corresponding to the updated virtual vehicle position. Generate the virtual space image showing the situation,
The live-action image acquisition means acquires one or a plurality of the combinations specified based on the updated virtual vehicle position from the live-action image storage means.
The composite image generation means obtains one or a plurality of the combinations of the virtual space image representing a state in which the virtual space is viewed from the position of the viewpoint corresponding to the camera for each of the plurality of cameras. Generates a composite image with one or more of the live-action images taken by the camera, which is included in the above.
The image generation system according to claim 1. A driver's console where simulated driving operations are performed by a driver trainer,
A simulated railcar that conductor trainers can get on and off,
A simulated platform provided adjacent to the simulated railway vehicle and
One or a plurality of live-action images taken by the camera located in front of the real railcar and facing forward, and the virtual space image showing the state of viewing the virtual space from the position of the viewpoint corresponding to the camera. A display panel located in front of the driver's console that displays a composite image,
One or a plurality of live-action images taken by the camera located on the side of the actual railroad vehicle and facing forward, and the virtual space image showing the state of viewing the virtual space from the position of the viewpoint corresponding to the camera. A display panel provided in front of the simulated platform, which displays a composite image of
The image generation system according to claim 2, wherein the image generation system is characterized in that. A simulated railcar that conductor trainers can get on and off,
A simulated platform provided adjacent to the side surface of the simulated railway vehicle,
One or a plurality of live-action images taken by the camera located behind the actual railcar and facing rearward, and the virtual space image showing the state of viewing the virtual space from the position of the viewpoint corresponding to the camera. A display panel provided behind the simulated railcar that displays a composite image, and
One or a plurality of live-action images taken by the camera located on the side of the actual railroad vehicle and facing forward, and the virtual space image showing the state of viewing the virtual space from the position of the viewpoint corresponding to the camera. Further includes a display panel provided in front of the simulated platform, which displays a composite image of the above.
The image generation system according to claim 2, wherein the image generation system is characterized in that. The composite image generation means repeatedly executes the generation of the composite image according to the operation of the user at predetermined time intervals.
The image generation system according to any one of claims 1 to 4, wherein the image generation system is characterized by the above. A step to update the virtual vehicle position in the virtual space according to the user's operation,
A step of generating a virtual space image showing a state in which the virtual space is viewed from the position of the viewpoint in the virtual space corresponding to the updated virtual vehicle position, and
From the live-action image storage means that stores the live-action image taken by the camera provided in the real railcar moving in the real space in association with the virtual vehicle position in the virtual space, to the updated virtual vehicle position. The step of acquiring one or more of the live-action images identified based on
A step of generating a composite image of the generated virtual space image and one or more of the acquired live-action images, and
An image generation method comprising. Procedure for updating the virtual vehicle position in the virtual space according to the user's operation,
A procedure for generating a virtual space image showing a state in which the virtual space is viewed from the position of the viewpoint in the virtual space corresponding to the updated virtual vehicle position.
From the live-action image storage means that stores the live-action image taken by the camera provided in the real railcar moving in the real space in association with the virtual vehicle position in the virtual space, to the updated virtual vehicle position. Procedure for acquiring one or more live-action images identified based on,
A procedure for generating a composite image of the generated virtual space image and one or more of the acquired live-action images.
A program characterized by having a computer execute.

Images