JP2022075212A - Moving body-mounted video device - Google Patents

Abstract

To provide a moving body-mounted video device that can more reliably ensure a safety of a moving body even when a passenger other than a driver wears an HMD as a user.SOLUTION: A moving body-mounted video device 10 comprises: an HMD 12 that provides an image to a user 110 while covering a view of the user 110; an input device 16 that accepts an operation from the user 110; a display controller 14 that controls a display of the HMD 12; and a monitoring device 18 that monitors a status of a driver 102, wherein: when the monitoring device 18 determines that the driver 102 is in an emergency state in which the driver cannot operate properly, the display controller 14 provides the user 110 with a driving environment image 62 equivalent to a view seen from the driver's seat via the HMD 12, and the display controller executes an alternative driving process that accepts the operation of the input device 16 as an operation for an emergency stop of the vehicle 100.SELECTED DRAWING: Figure 2

Description

This specification discloses a moving body-mounted video device capable of providing a realistic image to a passenger who is a occupant other than the driver of the moving body.

In recent years, head-mounted displays (hereinafter referred to as "HMDs") are often used to enable realistic video experiences. The head-mounted display is a video output device that is mounted on the user's head and provides video to the user while covering the user's field of view. The images constituting the image are displayed in the display area set in front of the user. Since such an HMD covers the user's field of view, the user can concentrate on the image provided via the HMD and can obtain a high sense of presence.

It has been partially proposed to use such an HMD in a moving body such as a vehicle. For example, Patent Document 1 discloses a technique in which an HMD is attached to a vehicle occupant to display an image linked to the movement of the vehicle. According to such a technique, the occupant can watch the video with a high sense of presence.

Japanese Unexamined Patent Publication No. 2017-146963

Here, since the HMD covers the user's field of view, it is difficult for the user to grasp the change in the surrounding environment while viewing the video. Therefore, in the moving body, when a passenger other than the driver wears the HMD, the passenger cannot notice the change of the driver of the moving body (for example, dozing) at an early stage, and the passenger cannot notice the change of the driver of the moving body at an early stage. There was a risk that safety could not be ensured.

Therefore, the present specification discloses a moving body-mounted video device that can more reliably ensure the safety of the moving body even when a passenger other than the driver wears the HMD as a user.

The mobile body-mounted video device disclosed in the present specification is a head-mounted display that is mounted on the head of a user who is an occupant other than the driver of the mobile body and provides the user with video while covering the user's view. The display controller includes an input device that receives an operation from the user, a display controller that controls the display of the head-mounted display, and a monitoring device that monitors the state of the driver. When it is determined that the driver is in an emergency state in which he / she cannot drive properly, the user is provided with a driving environment image equivalent to the view seen from the driver's seat via the head-mounted display, and the input device is used. The operation is accepted as an operation for an emergency stop of the moving body, and an alternative operation process is executed.

With such a configuration, the user can quickly grasp that the driver is in an emergency state. Further, in the event of an emergency, the user can perform an operation for emergency stop without removing the head-mounted display. Therefore, the user can quickly stop the moving object in an emergency, and can more reliably ensure the safety of the moving object.

In this case, in the alternative operation process, the display controller may accept the operation of the input device as at least one operation of acceleration / deceleration of the moving body, steering, and lighting of the turn signal.

With such a configuration, the user can perform the emergency stop operation with the same feeling as in normal driving.

Further, in the alternative operation process, the display controller provides a guidance image explaining the operation content of the input device necessary for the emergency stop prior to the operation environment image or together with the operation environment image. It may be provided to the user via the head-mounted display.

With such a configuration, the user can grasp the operation contents necessary for the emergency stop. As a result, the user can make an emergency stop of the moving object more smoothly, and can ensure the safety of the moving object more reliably.

Further, the driving environment image may be a virtual reality image in which the direction of the viewpoint in the image changes in conjunction with the movement of the user's head.

With such a configuration, the user can obtain the same feeling as when actually sitting in the driver's seat, and can operate the moving body more naturally.

Further, the display controller may invalidate the operation by the driver during the execution period of the alternative operation process.

With such a configuration, it is possible to prevent a mixture of operation instructions of both the user and the driver, so that the moving body can be stopped more safely.

Further, when the driver is no longer in the emergency state during the execution period of the alternative operation process, the display controller stops the alternative operation process and invalidates the operation for emergency stop by the input device. At the same time, the driving operation by the driver may be enabled.

In this case, since the driver who is in a state of being able to drive properly can drive, the safety of the moving body can be ensured more reliably.

Further, the driver is provided with a stimulus giving device for giving a stimulus for recovering consciousness to the driver who has lost consciousness, and the display controller is provided with a stimulus by the stimulus giving device in parallel with the execution of the alternative driving process. May be executed.

With such a configuration, the possibility that the driver regains consciousness increases, and the possibility that the driver can ensure safety increases.

Further, the display controller may provide the user with a tutorial video that teaches the operation to be performed by the user in the alternative operation process at the start of using the head-mounted display via the head-mounted display.

By providing the tutorial video in advance, the user can smoothly perform the operation for emergency stop in the event of an emergency.

According to the moving body-mounted video apparatus disclosed in the present specification, the safety of the moving body can be ensured more reliably.

It is an image diagram which shows the state of use of a video apparatus. It is a block diagram which shows the structure of a video apparatus. It is a figure which shows an example of a driving environment image. It is a figure which shows another example of a driving environment image. It is a figure which shows an example of a guidance video. It is a flowchart which shows the processing flow of a video apparatus. It is a block diagram which shows the structure of another video apparatus. It is a flowchart which shows the processing flow of the video apparatus of FIG.

Hereinafter, the configuration of the moving body-mounted video device 10 (hereinafter abbreviated as “video device 10”) will be described with reference to the drawings. FIG. 1 is an image diagram showing a usage state of the video apparatus 10. Further, FIG. 2 is a block diagram showing the configuration of the video apparatus 10.

The imaging device 10 is mounted on a moving vehicle 100. The vehicle 100 is a vehicle that requires an operation input from the driver 102 in the process of traveling thereof. Therefore, the vehicle 100 may be a fully manually driven vehicle in which the driver 102 performs all driving tasks. Further, the vehicle 100 may have a driving support function for executing a part of the driving task on the vehicle side. Further, the vehicle 100 may be a conditional automatic driving vehicle in which the vehicle side requests a predetermined operation from the vehicle side in the traveling process, although all the driving tasks are executed on the vehicle side. In any case, the driver 102 needs to confirm the driving situation and perform an appropriate operation as necessary.

The video apparatus 10 uses a head-mounted display (hereinafter abbreviated as “HMD12”) to provide a user with a highly realistic image. Here, as described above, in the vehicle 100 of this example, the driver 102 needs to confirm the driving situation. Therefore, the video apparatus 10 is used by an occupant other than the driver 102.

As shown in FIG. 2, the video apparatus 10 includes an HMD 12, a display controller 14, an input device 16, a monitoring device 18, and a driving camera 20. The HMD 12 is a display device that is attached to the head of the user 110 and provides an image to the user 110 while covering the field of view of the user 110. Here, the "video" is at least content composed of images, and the "video" handled in this example is composed of an image and audio linked to the image.

The HMD 12 is a non-transmissive HMD that completely covers the periphery of the eye of the user 110 when worn. Therefore, when the user 110 wears the HMD 12, the user cannot directly see the outside with his / her own eyes. The HMD 12 has a display 36, headphones 38, and a motion sensor 40. The display 36 displays an image in a display area set in front of the user 110. The display 36 may be, for example, a display having an organic EL or the like, or a projector that projects an image onto a display area. The image data displayed by the display 36 is provided by the display controller 14, which will be described later.

The headphones 38 are provided near the ears of the user 110 and output the sound provided by the display controller 14. The headphones 38 cover both ears of the user 110. Therefore, when the user 110 wears the HMD 12, the sound generated in the surrounding environment cannot be heard or is difficult to hear.

The motion sensor 40 is a sensor that detects the motion of the HMD 12, and includes, for example, at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor. The detection value of the motion sensor 40 is output to the display controller 14. The display controller 14 calculates the movement of the head of the HMD 12 and, by extension, the user 110 based on the value detected by the motion sensor 40. Further, the HMD 12 has a communication I / F (not shown) and can communicate with the display controller 14 by wire or wirelessly.

The input device 16 receives an operation input from the user 110, and is generally referred to as a "remote controller". Since the field of view of the user 110 is obstructed by the HMD 12, the input device 16 is configured so that the user 110 can operate the input device 16 without visually recognizing the input device 16. For example, the input device 16 may have an operator that can recognize the position by tactile sensation, for example, a joystick or a physically pressable button. Further, the input device 16 may have a motion sensor (not shown) for detecting the motion of the input device 16, and may accept an instruction from the user 110 from the detection result of the motion sensor. For example, when the input device 16 is used to indicate the direction and amount of steering of the vehicle 100, the direction and amount of the steering may be indicated by the direction and amount of inclination of the input device 16 in the roll direction. .. In any case, the operation content for the input device 16 is sent to the display controller 14 by wire or wirelessly.

The display controller 14 controls the display of the HMD 12. The display controller 14 is, for example, a computer having a processor 22, a storage device 24, a communication I / F 26, and a video controller 28. This "computer" also includes a microcontroller that incorporates a computer system into an integrated circuit. In addition, a processor refers to a processor in a broad sense, and is a general-purpose processor (for example, CPU: Central Processing Unit, etc.) or a dedicated processor (for example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Fiber). It includes Gate Arrays, programmable logic devices, etc.).

The storage device 24 stores various data and programs. The storage device 24 does not have to be one physical element, and may be composed of a plurality of memories that are physically separated from each other. Further, the storage device 24 may be composed of a semiconductor memory (for example, RAM, ROM, solid state drive, etc.), a magnetic disk (for example, a hard disk drive, etc.), and a combination thereof.

The communication I / F 26 enables communication with various devices outside the display controller 14. The communication I / F 26 may correspond to a plurality of types of communication standards. For example, the communication I / F26 is, for example, wired communication such as USB, short-range wireless communication such as Bluetooth (registered trademark), wireless LAN such as Wi-Fi (registered trademark), mobile provided by a mobile phone company, or the like. It has communication equipment for performing at least one of data communication. The video controller 28 controls the display of the HMD 12 under the control of the processor 22.

Although the display controller 14 is shown as a single computer in FIG. 2, the display controller 14 may be composed of a plurality of physically separated computers. Therefore, the display controller 14 may have a plurality of processors 22. Further, although the display controller 14 is shown in FIG. 2 as a computer different from the vehicle ECU 120 described later, the vehicle ECU 120 may execute at least a part of the functions of the display controller 14.

The monitoring device 18 monitors the state of the driver 102 on behalf of the user 110. That is, as described above, when the HMD 12 is worn, the user 110 cannot grasp the surrounding state because the eyes and ears are covered. As a result, even if some abnormality occurs in the driver 102 during the period of using the HMD 12 and an emergency state in which proper operation cannot be continued occurs, the user 110 cannot notice such an abnormality. Therefore, the monitoring device 18 monitors the state of the driver 102 in place of the user 110 who uses the HMD 12. The monitoring device 18 has, for example, a monitoring camera 50 that captures an image of the vicinity of the driver 102's face, and outputs the captured image data to the display controller 14 as a monitoring result. Further, as another embodiment, the monitoring device 18 has a sensor for detecting a driving operation (for example, the amount of rotation of the steering wheel) performed by the driver 102 in place of or in addition to the monitoring camera 50, and the sensor of the sensor. The detection result is output to the display controller 14 as a monitoring result. Based on these monitoring results, the display controller 14 determines whether the driver 102 is in a normal state in which he / she can drive properly or in an emergency state in which he / she cannot drive properly.

The driving camera 20 is a camera provided near the driver's seat. The display controller 14 generates a driving environment image 62 (see FIG. 3), which is an image equivalent to the field of view seen from the driver's seat, based on the image captured by the driving camera 20. The driving environment image 62 is used when the driver 102 is in an emergency state, which will be described later. The number of such driving cameras 20 may be one or a plurality. Further, the driving camera 20 may be a camera mounted on an existing vehicle. Therefore, the driving camera 20 may be a camera for a drive recorder or a camera mounted for automatic driving. Further, the driving camera 20 may be a camera capable of capturing a wider range than the field of view of a person, for example, a hemispherical camera capable of capturing a 360-degree panoramic image.

The vehicle 100 has a vehicle ECU 120, a drive unit 122, and an environment sensor 124. The drive unit 122 has a plurality of devices for driving the vehicle 100. Specifically, the drive unit 122 includes a traveling device having wheels and the like for traveling the vehicle, a prime mover such as an engine and a motor, a power transmission device for transmitting the power of the prime mover to the traveling device, and a braking device for generating braking force. It has a suspension device that cushions the impact from the road surface, a steering device that transmits force and displacement to the traveling device for steering, and the like. The environment sensor 124 detects the surrounding environment of the vehicle 100 for driving support and automatic driving. Such an environment sensor 124 includes, for example, a camera, lidar, millimeter wave radar, sonar, a magnetic sensor, and a combination thereof. The camera mounted as the environment sensor 124 may be used as the above-mentioned surveillance camera 50 or driving camera 20.

The vehicle ECU 120 controls the drive of the entire vehicle including the drive unit 122. The vehicle ECU 120 is physically a computer having a processor and a storage device 24. Although the vehicle ECU 120 is shown as a single element in FIG. 2, the vehicle ECU 120 may be composed of a plurality of ECUs. For example, an ECU may be provided for each element constituting the drive unit 122. Therefore, the vehicle ECU 120 may have a motor ECU, a brake ECU, and the like.

Next, the operation of such a video apparatus 10 will be described. The user 110, who is a occupant other than the driver 102, can enjoy the entertainment image 60 (see FIG. 1) while the vehicle 100 is running by using the image device 10. Specifically, the user 110 wears the HMD 12 and operates the input device 16 to select the desired entertainment video 60. The display controller 14 outputs the image data and the audio data constituting the entertainment video 60 selected by the user 110 to the display 36 and the headphones 38. Here, the entertainment video 60 may be a simple video content that is not linked to the head of the user 110, or may be a virtual reality video in which the direction of the viewpoint in the video changes in conjunction with the movement of the head of the user 110. Further, the entertainment video 60 may be a passive video (for example, a movie) that does not require the operation input of the user 110 via the input device 16, or an active video that requires the operation input of the user 110 via the input device 16. It may be a typical image (for example, a game). Further, the content of the entertainment image 60 may change in conjunction with the movement of the vehicle 100. For example, when the entertainment image 60 is a virtual reality image in which the user 110 travels in space on a spaceship, when the vehicle 100 turns right, the image is displayed so that the spacecraft in the entertainment image 60 also turns right. It may be changed. In any case, when the entertainment video 60 is provided using the HMD 12, the user 110 may concentrate on the entertainment video 60 provided to the HMD 12 because the eyes and ears of the user 110 are covered with the HMD 12. You can get a high sense of presence.

On the other hand, when the HMD 12 is attached, the user 110 cannot grasp the surrounding situation, particularly the situation of the driver 102. Therefore, if some abnormality occurs in the driver 102 and an emergency state occurs in which proper operation cannot be continued, the user 110 may not be able to respond promptly. Therefore, in this example, a monitoring device 18 is provided, and instead of the user 110, the video device 10 monitors whether or not the driver 102 is in an emergency state.

Here, the emergency state is a state in which the driver 102 cannot continue proper driving. Such an emergency state is, for example, a state in which the driver 102 loses consciousness due to dozing or fainting, a state in which the driver 102 is suffering from strong pain due to some kind of seizure, a state in which the driver 102 is confused, or a state in which the driver 102 is confused. This includes a state in which at least a part of the body cannot be moved. Whether or not it is in such an emergency state is determined based on an image obtained by capturing the image of the driver 102, the content of the driving operation by the driver 102, and the like.

When the display controller 14 determines that the driver 102 is in an emergency state as a result of monitoring by the monitoring device 18, the display controller 14 replaces the driver 102 and prompts the user 110 to make an emergency stop of the vehicle 100. Execute the operation process. Specifically, when it is determined to be in an emergency state, the display controller 14 causes the HMD 12 to display the driving environment image 62 instead of the entertainment image 60. As described above, the driving environment image 62 is an image equivalent to the field of view seen from the driver's seat, and is a real-time image having almost no time difference from the reality. FIG. 3 is an image diagram of the driving environment image 62. As shown in FIG. 3, the driving environment image 62 includes not only an image of the front outside of the vehicle 100, but also an image of a rear-view mirror seen from the driver's seat, an image of various instruments, and an image of the side of the vehicle 100. It may be included.

The driving environment image 62 may be a virtual reality image in which the direction of the viewpoint in the image changes in conjunction with the movement of the head of the user 110. Therefore, when the head of the user 110 is turned to the right side, the driving environment image 62 may be changed to an image taken on the right side of the driver's seat. By converting the driving environment image 62 into a virtual reality image in this way, the user 110 can recognize himself / herself as if he / she is sitting in the driver's seat, and can perform an emergency stop operation with a more natural feeling. can.

Further, as another form, the driving environment image 62 may be an image of a fixed viewpoint. In this case, as shown in FIG. 4, the driving environment image 62 simultaneously displays an image 62a that images the front of the vehicle 100, an image 62b that images the side of the vehicle 100, and an image 62c that images the rear of the vehicle 100. It may be a video. In this way, when the driving environment image 62 is composed of a plurality of images from a fixed viewpoint, the calculation amount for generating the driving environment image 62 is calculated while providing the user 110 with the visual information necessary for the emergency stop operation. It can be significantly reduced compared to virtual reality images.

Further, during the period of displaying the driving environment image 62, the display controller 14 enables an emergency stop operation of the vehicle 100 by operating the input device 16. The emergency stop operation includes at least one operation of accelerating / decelerating, steering, and turning on the turn signal of the vehicle 100. The display controller 14 converts the operation received by the input device 16 into a signal for such an emergency stop operation, and outputs the signal to the vehicle ECU 120. The vehicle ECU 120 controls the drive unit 122 according to the received signal. As a result, the user 110 can make an emergency stop of the vehicle 100 on behalf of the driver 102. As a result, the driver 102 can quickly secure safety even when the driver cannot continue the operation. Hereinafter, the process of accepting the operation of the input device 16 as an emergency stop operation while providing the operating environment image 62 is referred to as an “alternative operation process”.

By the way, there is a high possibility that the user 110 cannot grasp which part of the input device 16 should be operated and how the operation of the vehicle 100 can be controlled only by providing the driving environment image 62. Therefore, prior to the provision of the driving environment video 62, or in parallel with the provision of the driving environment video 62, a guidance video 64 explaining the operation contents of the input device 16 necessary for the emergency stop is provided to the HMD 12. You may. FIG. 5 is a diagram showing an example of the guidance video 64. The guidance video 64 includes an image diagram of the input device 16 and its operation contents. In the example of FIG. 5, the input device 16 has a joystick 44 and a direction key 46. Further, the input device 16 has a built-in gravity sensor (not shown) and can detect the inclination of the input device 16 itself. In the guidance image 64 of FIG. 5, the vehicle 100 can be accelerated or decelerated by moving the joystick 44 up and down, the turn signal can be turned on by pressing the left and right direction keys 46, and the entire input device 16 is tilted in the roll direction. Explains that the steering of the vehicle 100 can be instructed. By displaying the guidance image 64, the user 110 can easily grasp what kind of operation should be performed for the emergency stop.

Further, even during the execution period of the alternative operation process, the driver 102 may perform some operation operation consciously or unconsciously. For example, when the driver 102 falls asleep while holding the steering wheel, the steering wheel may sway in conjunction with the movement of the body. In addition, if the driver cannot move freely due to some illness, the driver 102 may endeavor to continue the driving operation as much as possible. When such a driver operation of the driver 102 is enabled in an emergency state, instructions from both the user 110 and the driver 102 are mixed, which may hinder safe driving. Therefore, the operation of the driver 102 may be invalidated during the execution period of the alternative operation process. That is, during the execution period of the alternative operation process, the operation input via the steering wheel, the accelerator pedal, the brake pedal, or the like may be invalidated. With such a configuration, the vehicle 100 can be safely and urgently stopped in an emergency state.

FIG. 6 is a flowchart showing the flow of processing by the video apparatus 10. When the user 110 wearing the HMD 12 instructs to provide the entertainment image 60, the display controller 14 provides the entertainment image 60 to the HMD 12 (S10). Meanwhile, the display controller 14 determines whether or not the state of the driver 102 is an emergency state in which the driver cannot continue driving properly (S12). If it is not in an emergency state, the entertainment video 60 will continue to be provided until the user 110 inputs an end instruction. On the other hand, when it is determined that the driver 102 is in an emergency state (Yes in S12), the display controller 14 ends the provision of the entertainment video 60, and the user 110 needs to perform an emergency stop operation by the user 110. Notify that there is (S16). After that, the display controller 14 causes the HMD 12 to display the guidance image 64 and the operating environment image 62 (S18, 20). The guidance video 64 may be displayed at all times thereafter, or may be stopped after a certain period of time.

During the period during which the driving environment image 62 is provided, the display controller 14 outputs an instruction signal for invalidating the operation of the driver 102 to the vehicle ECU 120 (S22). Upon receiving this instruction signal, the vehicle ECU 120 invalidates the operation instruction by the driver 102.

On the other hand, the display controller 14 acquires the operation content for the input device 16, converts the operation content into a driving operation signal, and outputs the operation content to the vehicle ECU 120 (S24). For example, in the case of the input device 16 of FIG. 5, the display controller 14 generates a steering signal according to the direction and amount of tilt of the input device 16 and generates an acceleration / deceleration signal according to the direction and amount of tilt of the joystick 44. Then, in response to the pressing of the direction key 46, a lighting signal of the turn signal is generated and output to the vehicle ECU 120. The vehicle ECU 120 drives the drive unit 122 according to such a signal. Then, if the vehicle 100 can be stopped by such an operation of the input device 16 (Yes in S26), the processing of the video apparatus 10 is completed.

As described above, in this example, the state of the driver 102 is monitored during the period during which the entertainment image 60 is provided, and if the driver 102 becomes an emergency state in which the driver 102 cannot drive properly, the operation of emergency stop of the vehicle 100 is performed. Let the user 110 execute. As a result, safety can be ensured quickly even in an emergency situation.

Next, a modified example of the video apparatus 10 will be described with reference to FIG. 7. The imaging device 10 differs from the imaging device 10 of FIG. 2 in that it has a stimulus applying device 66. The stimulus applying device 66 is a device that attempts to restore the consciousness of the driver 102 by applying some kind of stimulus to the driver when the driver 102 loses consciousness. The configuration of the stimulus giving device 66 is not limited as long as it gives some kind of stimulus to the driver 102. Therefore, the stimulus applying device 66 is, for example, a speaker that outputs a relatively loud sound, a vibration generator that vibrates the driver's seat or the steering wheel, a motor that changes the reclining of the driver's seat, and a strong light for the driver 102. Lights that emit light, and combinations thereof may be included.

Further, in the video apparatus 10 of FIG. 7, prior to the provision of the entertainment video 60, a tutorial video for making the user 110 understand the emergency stop operation using the input device 16 is provided. That is, if the emergency stop operation is not understood in advance, it is difficult to properly execute the emergency stop operation even if the guidance image 64 is displayed in an emergency state. Therefore, in this example, a tutorial video for understanding the emergency stop operation when starting to use the video device 10 is provided.

The tutorial video may be a video video that simply explains the emergency stop operation. Further, as another form, the tutorial video may be an interactive video in which the user 110 is made to practice the operation of emergency stop while interacting with the user 110. Therefore, for example, in the tutorial video, after explaining the correspondence between the operation of the input device 16 and the movement of the vehicle 100, the output of the task and the evaluation of the operation performed by the user 110 for clearing the task are repeatedly performed. It may be a thing. Here, the "problem" is an operation content requested by the user, such as "turn the rudder to the right". When receiving such a problem, the user 110 actually operates the input device 16. If the operation performed by the user 110 is appropriate, the display controller 14 proceeds to the description of the next operation, and if not, outputs the same task again. In this way, by making the tutorial video a video for the user 110 to practice the operation, the user 110 can smoothly perform the emergency stop operation in the event of an emergency.

Next, the flow of processing of the video apparatus 10 of FIG. 7 will be described with reference to FIG. As shown in FIG. 8, when the user 110 starts using the video device 10, the video device 10 first provides the tutorial video to the user (S08). As described above, the tutorial video is a video that teaches the operation that the user 110 should perform when the driver is in an emergency state. By viewing such a tutorial video in advance, the user can smoothly perform an operation for emergency stop even when the driver 102 is in an emergency state.

When the tutorial video is completed, the video device 10 provides the entertainment video desired by the user 110 (S10). Then, when the driver 102 is in an emergency state (Yes in S12), the alternative operation process is executed (S16 to S26). Since the flow of this alternative operation process is the same as that of FIG. 6, detailed description here will be omitted.

Here, in the case of this example, the stimulation is applied to the driver 102 by the stimulation application device 66 in parallel with the alternative operation process (S28). Then, the state of the driver 102 that has been stimulated is monitored by the monitoring device 18 (S30). When the consciousness of the driver 102 is restored by the stimulus and the driver 102 returns to the normal state in which the driver 102 can drive properly (Yes in S30), the display controller 14 indicates that the driver 102 has returned. (S32). After that, the display controller 14 invalidates the operation by the input device 16 and enables the operation by the driver 102 (S34). As a result, the driver 102 will subsequently drive the vehicle 100. On the other hand, when the driver 102 does not return even if the stimulus is given (No in S30), the display controller 14 accepts the operation via the input device 16 (S24) as it is, and the driver by the stimulus giving device 66. The application of the stimulus to the 102 (S28) is repeated until the stop is completed or the driver 102 returns to the normal state.

As is clear from the above explanation, according to this example, since the tutorial video is provided to the user 110 in advance, when the driver 102 becomes an emergency state, the user 110 performs an operation for an emergency stop. It can be executed smoothly. Further, in this example, since the driver 102 is stimulated after the driver 102 is in an emergency state, the driver 102 can easily return to the normal state. Then, when the driver 102 returns to the normal state, the right to drive is returned to the driver 102, so that more stable running becomes possible.

The configuration described so far is an example, and at least in a video device using the HMD 12, the state of the driver 102 is monitored, and it is determined that the driver 102 is in an emergency state where the driver 102 cannot operate properly. Provides the user 110 with a driving environment image 62 equivalent to the field of view seen from the driver's seat via the HMD 12, and other configurations if the operation of the input device 16 is accepted as an operation for an emergency stop of the moving body. May be changed. For example, in the above description, the image device mounted on the vehicle 100 is taken as an example, but the image device is not limited to the vehicle 100 as long as it is a moving body driven by the driver 102, and other moving bodies. For example, it may be mounted on a ship, an aircraft, or the like. Further, if the driving environment video 62 is provided, the guidance video 64 and the tutorial video may not be provided. Further, the driving operation by the driver 102 may not be invalidated during the execution period of the alternative driving process.

10 Video equipment, 14 Display controller, 16 Input device, 18 Surveillance device, 20 Driving camera, 22 Processor, 24 Storage device, 26 Communication I / F, 28 Video controller, 36 Display, 38 Headphones, 40 Motion sensor, 44 Joystick, 46 direction keys, 50 surveillance cameras, 60 entertainment videos, 62 driving environment videos, 64 guidance videos, 66 stimulus givers, 100 vehicles, 102 drivers, 110 users, 120 vehicle ECUs, 122 drive units, 124 environment sensors.

Claims (1)

A head-mounted display that is attached to the head of a user who is a occupant other than the driver of the moving object and provides an image to the user while covering the user's field of view.
An input device that accepts operations from the user,
A display controller that controls the display of the head-mounted display, and
A monitoring device that monitors the driver's condition and
When the display controller is determined to be in an emergency state in which the driver cannot drive properly based on the monitoring result of the monitoring device, the display controller has a view from the driver's seat via the head-mounted display. An alternative operation process is executed, which provides the user with an image of the driving environment equivalent to that of the above, and accepts the operation of the input device as an operation for an emergency stop of the moving body.
A moving body-mounted video device that features this.

Images