JP6929674B2 - Environmental image display system and environmental image display method - Google Patents

Description

The present invention relates to an environmental image display system and an environmental image display method, and more particularly to an environmental image display system and an environmental image display method for displaying an environmental image in an area where it is difficult for humans to enter.

In recent years, in order to grasp the situation of an area where people are in danger and cannot enter, a technology has been proposed to take an environmental image of the area using a camera mounted on a radio-controlled unmanned moving body. (Refer to Patent Document 1: Hereinafter referred to as "conventional example"). In this conventional technique, an environmental image is taken by a camera mounted on an unmanned moving body, and the shooting result is transmitted to a display device used by the user by wireless communication. Then, the display device displays the environment image corresponding to the received shooting result.

In the conventional technology, there are three cameras: a bird's-eye view camera that is directed downward to photograph the surroundings of an unmanned moving object, a front camera that photographs the front side, and a rear camera that photographs the rear side. Is to be mounted on an unmanned moving body.

Japanese Unexamined Patent Publication No. 2013-11028

In the conventional example, as described above, the shooting results of the three cameras of the bird's-eye view camera, the front camera, and the rear camera are displayed. However, in the conventional example, no disclosure is made as to how to display the shooting results by the three cameras.

For example, it is conceivable to display the shooting results of the three cameras as they are on the three display devices prepared for each of the three cameras. In such a case, when a situation occurs in which the image around the point of interest straddles the display images on two or more display devices, the image around the point is displayed on the same display device for convenience of observation. In order to display it, it is necessary to change the posture of the unmanned moving body by radio control. Such an operation makes the user feel complicated.

Therefore, there is a long-awaited new technology capable of displaying an environmental image around an unmanned moving object while suppressing the occurrence of complexity for the user. Responding to such a request is one of the problems to be solved by the present invention.

The environmental image display system of the present invention is arranged on an unmanned moving body that can be operated wirelessly, takes a peripheral image of the unmanned moving body, and transmits the shooting result data by wireless communication; and processes the shooting result data. An image processing device that generates display image data; and a head mount display device that is connected to the image processing device and displays an image according to the displayed image data; A spherical camera that captures images over the entire celestial sphere while using a plurality of photographing optical systems that are the same as each other and have different shooting directions; A wireless transmission unit that transmits shooting result data corresponding to each to the image processing device; the image processing device includes a wireless receiving unit that receives a plurality of shooting result data transmitted from the shooting device; An omnidirectional image data generator that generates omnidirectional image data by performing stitching processing on a plurality of received shooting result data; and an image of a display area determined according to the posture of the head mount display device. The head mount display device includes a display image data generation unit that extracts from the spherical image and generates the display image data; and the head mount display device has a display unit that displays an image corresponding to the display image data; The display image data generation unit includes a posture change amount detection unit that detects the attitude change amount of the head mount display device from a time point and sends the detection result to the image processing device; The environmental image display system is characterized in that it generates initial display image data in which the optical axis direction of the specific photographing optical system selected from the above is the initial line-of-sight direction of the user of the head mount display device.

In this environment image display system, a peripheral image of an unmanned moving body is taken by an omnidirectional camera of a photographing device arranged on a radio-controlled unmanned moving body. This spherical camera uses a plurality of photographing optical systems having the same wide field of view and different shooting directions, and captures an image over the entire celestial sphere while having an overlapping region. Then, the wireless transmission unit of the photographing device transmits the photographing result data corresponding to each of the plurality of photographing optical systems, which is the imaging result by the spherical camera of the photographing device, to the image processing device.

In the image processing device, the spherical image data generation unit receives a plurality of shooting result data transmitted from the shooting device via the wireless receiving unit. Subsequently, the spherical image data generation unit performs stitching processing on the received plurality of shooting result data to generate spherical image data.

The spherical image corresponding to the spherical image data generated in this way includes a joint portion by the stitching process. At such a joint portion, some discontinuity of the image will occur.

Further, the posture change amount detection unit of the head mount display device detects the posture change amount of the head mount display device from the previous detection time, and sends the detection result to the image processing device. Subsequently, the display image data generation unit of the image processing device extracts the image of the display area determined according to the posture of the head mount display device sent from the head mount display device from the spherical image, and outputs the display image data. Generate. Here, the display image data generation unit changes the posture of the head mount display device, with the optical axis direction of the specific photographing optical system selected from the plurality of photographing optical systems as the initial line-of-sight direction of the user of the head mount display device. Based on the received result of the quantity, the position of the display image in the spherical image is specified, and the display image data is generated. Then, the display image data generation unit sends the generated display image data to the head mount display device.

When the display image corresponding to the extracted display image data includes a joint by the stitching process, the display image includes discontinuity. When such a discontinuity occurs in the central portion of the displayed image, the user feels the greatest discomfort. Then, the farther the discontinuity generation portion is from the central portion of the displayed image, the less discomfort the user feels. When the seams formed by the stitching process are not included in the display image, the user does not feel uncomfortable due to the occurrence of discontinuity in the display image.

In the head mount display device, the display unit receives the display image data sent from the image processing device. Then, the display unit displays an image according to the display image data.

Of the images displayed in this way, since there is no image discontinuity in the central portion of the image corresponding to the initial line-of-sight direction (hereinafter referred to as "initial display image"), at least the initial display image is displayed. It is an image in which the occurrence of discomfort for the user due to the occurrence of discontinuity in the image is suppressed. Then, by changing the posture of the head and changing the posture of the head mount display device, the user can display an image in an arbitrary direction around the unmanned moving body and observe the surroundings of the moving body.

Therefore, according to the environmental image display system of the present invention, the environment image around the unmanned moving body is displayed while suppressing the complexity for the user and the occurrence of discomfort for the user due to the discontinuity of the image. can do.

In the environmental image display system of the present invention, either the image processing device or the head mount display device receives an initial line-of-sight direction input that sets the initial line-of-sight direction to a direction corresponding to the current attitude of the head mount display device. The input unit to be performed; may be further provided. In this case, the input unit can be used to set the direction in which the user wearing the head mount display device is likely to change the desired posture, such as the direction toward the front, as the initial line-of-sight direction. Therefore, the convenience of the user can be improved.

Further, in the environmental image display system of the present invention, the spherical image data generation unit may be configured to perform a blending process on the overlapping region at the time of the stitching process. In this case, even if the display image includes the stitching processing portion, it is possible to reduce the discomfort of the user who sees the display image.

Further, in the environmental image display system of the present invention, the region corresponding to each of the photographing optical systems in the spherical image has a width capable of including the display region of the image corresponding to the initial display image data. Can be. In this case, the initial display image does not include a stitching processed portion where image discontinuity may occur. Therefore, it is possible to effectively suppress the occurrence of discomfort for the user due to the occurrence of image discontinuity.

Further, in the environmental image display system of the present invention, the number of the plurality of photographing optical systems may be "2". In this case, since the stitching processing portion can be minimized, it is possible to minimize the occurrence of discomfort for the user due to the occurrence of image discontinuity.

The environmental image display method of the present invention is a photographing device which is arranged on an unmanned moving body capable of wireless control, captures a peripheral image of the unmanned moving body, and transmits the imaging result data by wireless communication; and processes the imaging result data. An environmental image display used in an environmental image display system including an image processing device that generates display image data; and a head mount display device that is connected to the image processing device and displays an image according to the displayed image data. In this method, the imaging device uses a plurality of imaging optical systems having the same imaging field width and different imaging directions to obtain an image over the entire celestial sphere while having overlapping regions. A shooting step of taking a picture and transmitting the shooting result data corresponding to each of the plurality of shooting optical systems to the image processing device; the image processing device with respect to the plurality of shooting result data transmitted in the shooting step. An all-sky image data generation step of performing stitching processing to generate all-sky image data; an image of a display area determined by the image processing device according to the posture of the head-mounted display device is an all-sky image. A display image data generation step of extracting from the display image data to generate the display image data; a display step in which the head mount display device displays an image corresponding to the display image data; A posture change amount detection step of detecting the posture change amount of the head mount display device from a time point and sending the detection result to the image processing device; It is an environment image display method characterized by generating initial display image data in which the optical axis direction of the specific photographing optical system selected from the above is the initial line-of-sight direction of the user of the head mount display device.

In this environmental image display method, in the shooting process, the shooting devices use a plurality of shooting optical systems having the same wide field of view and different shooting directions, and have overlapping areas as a whole. An image over the celestial sphere is captured, and imaging result data corresponding to each of the plurality of imaging optical systems is transmitted to the image processing device. Further, in parallel with the photographing step, in the posture change amount detection step, the head mount display device detects the posture change amount of the head mount display device from the previous detection time, and sends the detection result to the image processing device.

Upon receiving a plurality of shooting result data transmitted from the shooting device, the image processing device performs stitching processing on the plurality of shooting result data in the spherical image data generation step to obtain the spherical image data. Generate. Subsequently, in the display image data generation step, the image processing device extracts the image of the display area determined according to the posture of the head mount display device from the spherical image to generate the display image data. Then, in the display process, the head mount display device displays an image corresponding to the display image data.

Therefore, according to the environment image display method of the present invention, the environment image around the unmanned moving body is displayed while suppressing the complexity for the user and the occurrence of discomfort for the user due to the discontinuity of the image. can do.

As described above, according to the environmental image display system and the environmental image display method of the present invention, it is unmanned while suppressing the complexity for the user and the occurrence of discomfort for the user due to the discontinuity of the image. It has the effect of being able to display an environmental image around the moving object.

It is a figure which shows the structure of the environmental image display system which concerns on one Embodiment of this invention. It is a block diagram for demonstrating the structure of the photographing apparatus of FIG. It is a figure for demonstrating the image taken by the photographing apparatus of FIG. It is a block diagram for demonstrating the structure of the image processing apparatus of FIG. It is a figure for demonstrating the relationship between the spherical image corresponding to the spherical image data generated by the spherical image data generation part of FIG. 4 and the image captured by the photographing apparatus of FIG. It is a block diagram for demonstrating the structure of the head mount display device of FIG. It is a flowchart for demonstrating the process executed by the image processing apparatus of FIG. It is a figure for demonstrating an example of the relationship between a spherical image and a display image.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 to 8. In the following description and drawings, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

[composition]
As shown in FIG. 1, the environment image display system 100 according to the embodiment includes a photographing device 200, an image processing device 300, and a head mount display device 400.

Here, the photographing device 200 is arranged in the unmanned moving body MV. The unmanned moving body MV is remotely controlled by the remote control device RM. In the present embodiment, the unmanned moving body MV is a vehicle traveling on the ground.

<Configuration of imaging device 200>
As shown in FIG. 2, the photographing device 200 includes an omnidirectional camera 210 and a wireless communication unit 220.

The omnidirectional camera 210 has two photographing optical systems OP1 and OP2. The wide field of view corresponding to each of the shooting optical systems OP1 and OP2 is the same as each other. Here, the optical axis direction of the photographing optical system OP1 is the first direction which is the front direction of the unmanned moving body MV. Further, the optical axis direction of the photographing optical system OP2 is the opposite direction of the first direction, that is, the second direction which is the rear direction of the unmanned moving body MV.

Then, the spherical camera 210 uses the photographing optical systems OP1 and OP2 to capture an image over the entire celestial sphere while having an overlapping region. Then, the spherical camera 210 sends the shooting result data corresponding to each of the shooting optical systems OP1 and OP2 to the wireless communication unit 220.

Note that FIG. 3A shows a shooting range of an image corresponding to the shooting optical system OP1. Further, FIG. 3B shows a shooting range of an image corresponding to the shooting optical system OP2.

In FIG. 3A, the horizontal direction is the width direction of the unmanned moving body MV, and the vertical direction is the height direction of the unmanned moving body MV. The first direction is the (0 °, 0 °) direction.

Further, in FIG. 3B, the horizontal direction is the width direction of the unmanned moving body MV, and the vertical direction is the height direction of the unmanned moving body MV. The second direction is the (0 °, 0 °) direction.

The wireless communication unit 220 receives two shooting result data sent from the spherical camera 210. Then, the wireless communication unit 220 transmits the two shooting result data to the image processing device 300 by wireless communication.

<Configuration of image processing device 300>
As shown in FIG. 4, the image processing device 300 includes a first wireless communication unit 310 and a spherical image data generation unit 320. Further, the image processing device 300 includes a display image data generation unit 330 and a second wireless communication unit 340.

The first wireless communication unit 310 performs wireless communication with the photographing device 200. The first wireless communication unit 310 receives two shooting result data transmitted from the shooting device 200. Then, the first wireless communication unit 310 sends the two shooting result data to the spherical image data generation unit 320.

The spherical image data generation unit 320 receives two imaging result data sent from the first wireless communication unit 310. Subsequently, the spherical image data generation unit 320 performs stitching processing on the two shooting result data to generate spherical image data.

In FIG. 5, the spherical image FSI corresponding to the spherical image data, the image (1) corresponding to the photographing optical system OP1, and the images (2-1) to (2-) corresponding to the photographing optical system OP2 are shown. The relationship with 8) is shown. In FIG. 5, the stitching processing area is shown by sprinkling small black circles.

In the present embodiment, the spherical image data generation unit 320 performs a blending process on the overlapping region of the image corresponding to the photographing optical system OP1 and the image corresponding to the photographing optical system OP2 during the stitching process. ing. Therefore, the discontinuity of the image in the vicinity of the joint in the spherical image is reduced as compared with the case where the blending process is not performed.

In the present embodiment, during the blending process, the spherical image data generation unit 320 increases the blend ratio of the image region corresponding to the photographing optical system to which each pixel belongs as the distance from the joint increases. It has become.

The display image data generation unit 330 receives the spherical image data sent from the spherical image data generation unit 320. Further, the display image data generation unit 330 receives a posture change amount of the head mount display device 400 from the previous detection time received from the head mount display device 400 by using the second wireless communication unit 340 (hereinafter, simply "posture change". (Also called "quantity") and receive the initial line-of-sight direction setting command. Subsequently, the display image data generation unit 330 displays the display area in the spherical image based on the initial line-of-sight direction set in response to the latest initial line-of-sight direction setting command and the amount of change in the posture of the head mount display device 400. decide.

Next, the display image data generation unit 330 extracts the image of the determined display area from the spherical image corresponding to the latest spherical image data, and generates the display image data. Then, the display image data generation unit 330 sends the generated display image data to the second wireless communication unit 340.

The second wireless communication unit 340 performs wireless communication with the head mount display device 400. The second wireless communication unit 340 receives the display image data sent from the display image data generation unit 330. Then, the second wireless communication unit 340 transmits the display image data to the head mount display device 400.

Further, the second wireless communication unit 340 receives the posture change amount of the head mount display device 400 transmitted from the head mount display device 400. Then, the second wireless communication unit 340 sends the posture change amount to the display image data generation unit 330.

Further, the second wireless communication unit 340 receives the initial line-of-sight direction setting command transmitted from the head mount display device 400. Then, the second wireless communication unit 340 sends the initial line-of-sight direction setting command to the display image data generation unit 330.

<Configuration of head mount display device 400>
As shown in FIG. 6, the head mount display device 400 includes a wireless communication unit 410 and a display unit 420. Further, the head mount display device 400 includes a posture change amount detecting unit 430 and an input unit 440.

The wireless communication unit 410 performs wireless communication with the image processing device 300. The wireless communication unit 410 receives the display image data transmitted from the image processing device 300. Then, the wireless communication unit 410 sends the display image data to the display unit 420.

Further, the wireless communication unit 410 receives the posture change amount of the head mount display device 400 sent from the posture change amount detection unit 430. Then, the wireless communication unit 410 sends the posture change amount to the image processing device 300.

Further, the wireless communication unit 410 receives the initial line-of-sight direction setting command sent from the input unit 440. Then, the wireless communication unit 410 sends the initial line-of-sight direction setting command to the image processing device 300.

The display unit 420 includes a display device such as a liquid crystal panel. The display unit 420 receives the display image data sent from the wireless communication unit 410. Then, the display unit 420 displays an image corresponding to the display image data.

The posture change amount detection unit 430 is configured to include, for example, a three-dimensional gyro sensor or the like. The attitude change amount detection unit 430 calculates the attitude change amount from the previous detection time by detecting the three-dimensional angular velocity at each time point. Then, the attitude change amount detection unit 430 sends the attitude change amount, which is the detection result, to the wireless communication unit 410.

The input unit 440 is configured to include a press button. Such a pressing button is a momentary operation button that is turned "ON" by the pressing operation and turned "OFF" when the pressing operation is completed. The input unit 440 sends an initial line-of-sight direction setting command to the wireless communication unit 410 each time a pressing operation is performed on the pressing button.

[motion]
Next, the operation of the environment image display system 100 configured as described above will be described mainly focusing on the processing executed by the image processing apparatus 300.

It is assumed that the photographing device 200 has started operation, and the photographing device 200 sequentially transmits the photographing result data to the image processing device 300. Further, it is assumed that the head mount display device 400 has also started operation, and the head mount display device 400 sequentially transmits the detected posture change amount to the image processing device 300. Further, the pressing operation of the pressing button of the input unit 440 in the head mount display device 400 has been pressed once in the past, and the initial line-of-sight direction setting command has been transmitted from the head mount display device 400 to the image processing device 300. And.

In the processing of the image processing device 300 executed under the above-mentioned operating environment, as shown in FIG. 7, first, in step S11, the spherical image data generation unit 320 sets the first wireless communication unit 310. It is determined whether or not new shooting result data sent from the shooting device 200 is received via the camera. If the result of the determination in step S11 is negative (step S11: N), the process proceeds to step S14, which will be described later.

If the result of the determination in step S11 is affirmative (step S11: Y), the process proceeds to step S12. In this step S12, the all-sky image data generation unit 320 receives the new shooting result data (two new shooting results of the shooting result data corresponding to the shooting optical system OP1 and the shooting result data corresponding to the shooting optical system OP2). Data) is used to generate all-sky image data. When generating the spherical image data, the spherical image data generation unit 320 performs the stitching process described above to generate the spherical image data. Then, the spherical image data generation unit 320 sends the generated spherical image data to the display image data generation unit 330.

Next, in step S13, the display image data generation unit 330 is based on the accumulation of the reception result of the posture change amount from the reception time of the latest initial line-of-sight direction setting command at that time via the second wireless communication unit 340. Then, the image in the display area is extracted from the spherical image corresponding to the latest spherical image data, and the display image data is generated. Then, the display image data generation unit 330 sends the generated display image data to the head mount display device 400 via the second wireless communication unit 340. As a result, the image corresponding to the displayed image data is displayed on the display unit 420 of the head mount display device 400.

Next, in step S14, it is determined whether or not the display image data generation unit 330 has received the new initial line-of-sight direction setting command sent from the photographing device 200 via the second wireless communication unit 340. If the result of the determination in step S14 is negative (step S14: N), the process returns to step S11.

If the result of the determination in step S14 is positive (step S14: Y), the process proceeds to step S15. In this step S15, the display image data generation unit 330 generates new initial display image data. When generating the new initial display image data, the display image data generation unit 330 sets the (0 °, 0 °) position in the latest spherical image as the center position of the new initial display image, and sets the image in the display area. Extract and generate new initial display image data. Then, the display image data generation unit 330 sends the generated initial display image data to the head mount display device 400 via the second wireless communication unit 340. As a result, the image corresponding to the new initial display image data is displayed on the display unit 420 of the head mount display device 400.

When the process of step S15 is completed, the process returns to step S11. After that, the processes of steps S11 to S15 are repeated.

Note that FIG. 8A shows the position of the display image area DSR in the spherical image FSI immediately after the initial line-of-sight direction setting is performed. Further, FIG. 8B shows an example of the position of the display image area DSR according to the posture change of the head mount display device 400 after the initial line-of-sight direction is set.

As described above, in the environment image display system 100 of the present embodiment, the peripheral image of the unmanned moving body MV is captured by the spherical camera 210 of the photographing device 200 arranged in the unmanned moving body MV that can be radio-controlled. .. This omnidirectional camera 210 uses the two imaging optical systems OP1 and OP2, which have the same imaging field of view and two imaging directions, and can capture an image over the entire celestial sphere while having an overlapping region. Take a picture. Then, the wireless transmission unit 220 of the imaging device 200 transmits the imaging result data corresponding to each of the two imaging optical systems OP1 and OP2, which are the imaging results of the spherical camera 210, to the image processing device 300.

In the image processing device 300, the spherical image data generation unit 320 receives the two imaging result data transmitted from the imaging device 200 via the first wireless receiving unit 310. Subsequently, the spherical image data generation unit 320 generates the spherical image data by performing stitching processing on the two received shooting result data.

Further, the posture change amount detection unit 430 of the head mount display device 400 detects the posture change amount of the head mount display device 400 from the previous detection, and sends the detection result to the image processing device 300. Subsequently, the display image data generation unit 330 of the image processing device 300 extracts an image of the display area determined according to the posture of the head mount display device 400 sent from the head mount display device 400 from the spherical image. Generate display image data. Here, the display image data generation unit 330 sets the optical axis direction of the specific photographing optical system OP1 selected from the two photographing optical systems OP1 and OP2 as the initial line-of-sight direction of the user of the head mount display device 400, and the display image data generation unit 330 heads. Based on the posture of the mount display device 400, the position of the display image in the spherical image is specified, and the display image data is generated. Then, the display image data generation unit 330 sends the generated display image data to the head mount display device 400.

In the head mount display device 400, the display unit 420 receives the display image data sent from the image processing device 300 via the wireless communication unit 410. Then, the display unit 420 displays an image according to the display image data.

Therefore, according to the environment image display system 100 of the present embodiment, the environment around the unmanned moving body MV is suppressed while suppressing the complexity for the user and the occurrence of discomfort for the user due to the discontinuity of the image. Images can be displayed.

Further, in the environment image display system 100 of the present embodiment, the head mount display device 400 is an input unit in which the initial line-of-sight direction input is performed to set the initial line-of-sight direction to a direction according to the current posture of the head mount display device 400. 440 is provided. Therefore, the input unit 440 can be used to set a direction in which a user wearing the head mount display device 400 is likely to change a desired posture, such as a direction facing the front, as the initial line-of-sight direction. Therefore, the convenience of the user can be improved.

Further, in the environment image display system 100 of the present embodiment, the spherical image data generation unit 320 performs a blending process on the overlapping region during the stitching process. For this reason, since the joint portion of the image is blended, even if the joint portion is included in the display image, it is possible to reduce the discomfort of the user who sees the display image. ..

Further, in the environment image display system 100 of the present embodiment, the area corresponding to each of the photographing optical systems OP1 and OP2 in the spherical image can include the image display area corresponding to the initial display image data. Is designed to have. Therefore, the initial display image does not include a stitching processing portion where discontinuity of the image may occur. As a result, it is possible to effectively suppress the occurrence of discomfort for the user due to the occurrence of image discontinuity.

Further, in the environmental image display system of the present invention, the number of photographing optical systems in the spherical camera 210 is set to "2". Therefore, since the stitching processing portion can be minimized, it is possible to minimize the occurrence of discomfort for the user due to the occurrence of image discontinuity.

[Modification of Embodiment]
The present invention is not limited to the above embodiment, and various modifications can be made.

For example, in the above embodiment, the unmanned moving vehicle is an unmanned vehicle traveling on the ground, but the unmanned moving vehicle may be an unmanned aerial vehicle such as a so-called drone.

Further, in the above embodiment, the number of photographing optical systems used in the photographing apparatus is set to "2", but it may be set to "3" or more.

Further, the blending method adopted in the stitching process in the above embodiment is an example, and another blending method may be adopted.

Further, in the above embodiment, the image processing device and the head mount display device are wirelessly connected, but the image processing device and the head mount display device may be connected by a wired connection.

Further, in the above embodiment, the head mount display device is provided with an input unit for performing initial line-of-sight direction input for setting the initial line-of-sight direction in a direction corresponding to the current head mount display device posture. The image processing apparatus may include the unit.

As described above, the present invention can be applied to an environmental image display system and an environmental image display method for displaying an environmental image around an unmanned moving body.

100 ... Environmental image display system, 200 ... Imaging device, 210 ... Spherical camera, 220 ... Wireless communication unit (wireless transmission unit), 300 ... Image processing device, 310 ... First wireless communication unit (wireless reception unit), 320 ... Spherical image data generation unit, 330 ... Display image data generation unit, 340 ... Second wireless communication unit, 400 ... Head mount display device, 410 ... Wireless communication unit, 420 ... Display unit, 430 ... Posture change amount detection unit 440 ... Input unit, MV ... Moving object, RM ... Remote control device, OP1, OP2 ... Shooting optical system, FSI ... Spherical image, DSR ... Display image area.

Claims (4)

An imaging device that is placed on a radio-controlled unmanned moving object, captures a peripheral image of the unmanned moving object, and transmits the imaging result data by wireless communication;
With an image processing device that processes the shooting result data and generates display image data;
A head mount display device connected to the image processing device and displaying an image according to the display image data;
The photographing device is
A spherical camera that captures images over the entire celestial sphere while using two imaging optical systems that have the same field of view and opposite shooting directions and have overlapping areas;
A wireless transmission unit that transmits shooting result data corresponding to each of the two shooting optical systems to the image processing device;
The image processing device is
A wireless receiver that receives two types of shooting result data transmitted from the shooting device;
An omnidirectional image data generation unit that generates omnidirectional image data by performing stitching processing on the two types of received shooting result data;
A display image data generation unit that extracts an image of a display area determined according to the posture of the head mount display device from the spherical image and generates the display image data;
The head mount display device is
With a display unit that displays an image corresponding to the display image data;
A posture change amount detection unit that detects the posture change amount of the head mount display device from the previous detection time and sends the detection result to the image processing device;
Either the image processing device or the head mount display device
An input unit for performing initial line-of-sight direction input for setting the direction according to the current posture of the head-mounted display device to the initial line-of-sight direction of the user of the head-mounted display device is further provided.
The display image data generation unit generates initial display image data in which the optical axis direction of the specific photographing optical system selected from the two photographing optical systems is the direction corresponding to the initial line-of-sight direction.
An environmental image display system characterized by this. The environmental image display system according to claim 1 , wherein the spherical image data generation unit performs a blending process on the overlapping region at the time of the stitching process. The region corresponding to each of the photographing optical systems in the spherical image has a size capable of including the display area of the image corresponding to the initial display image data, according to claim 1 or 2 . Described environmental image display system. An imaging device that is placed on a wirelessly maneuverable unmanned moving object, captures a peripheral image of the unmanned moving object, and transmits the imaging result data by wireless communication; the imaging result data is processed to generate display image data. An image processing device; a head mount display device connected to the image processing device and displaying an image according to the displayed image data; the image processing device and the head mount display device are provided with the head mount display. the direction corresponding to the current posture of the apparatus, the head-mounted display device of the environmental image display to be used in the user initial viewing direction input further environmental image display system Ru an input unit performed to set the initial viewing direction of It ’s a method,
The photographing apparatus uses two photographing optical systems in which the width of the photographing field of view is the same and the photographing directions are opposite to each other, and captures an image over the entire celestial sphere while having an overlapping region. A shooting step of transmitting shooting result data corresponding to each of the two shooting optical systems to the image processing apparatus;
A spherical image data generation step in which the image processing apparatus performs a stitching process on two types of shooting result data transmitted in the shooting step to generate spherical image data;
A display image data generation step in which the image processing device extracts an image of a display area determined according to the posture of the head mount display device from the spherical image to generate the display image data;
A display process in which the head mount display device displays an image corresponding to the display image data;
The head mount display device includes a posture change amount detection step of detecting the posture change amount of the head mount display device from the previous detection time and sending the detection result to the image processing device.
In the display image data generation step, initial display image data is generated in which the optical axis direction of the specific photographing optical system selected from the two photographing optical systems is the direction corresponding to the initial line-of-sight direction.
An environmental image display method characterized by this.

Images