JP2022086263A - Information processing apparatus and information processing method - Google Patents

Abstract

To display a captured image in a direction desired by a user in a position so as to reduce gaze moving amount of the user without user operation.SOLUTION: An information processing apparatus includes a controller which receives an input of data on a gaze direction of a user of a vehicle. The controller specifies one or more imaging apparatuses out of multiple imaging apparatuses installed to image the surroundings of the vehicle, specifies one or more display units out of multiple display units arranged in the interior of the vehicle on the basis of the gaze direction of the user, and causes the specified display units to display images captured by the specified imaging apparatuses.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing apparatus and an information processing method.

Patent Document 1 discloses a driving support image display control device for displaying an image for driving support on an in-vehicle monitor or the like. This driving support image display control device identifies the driving behavior of the own vehicle, refers to the data registered in advance, and displays the captured image of the gaze point at the time of the specified driving behavior on the display screen.

Japanese Unexamined Patent Publication No. 2007-215063

However, in the conventional driving support image display control device described above, the displayed captured image may differ from the captured image that the user wants to see. In this case, the user operates a hand switch to switch the captured image. There is a need to. Further, the display device is installed above the intermediate position of the center console in the vehicle interior, and the line-of-sight direction of the user may be different from the display position of the captured image. In this case, the user needs to move his / her line of sight significantly in order to confirm the captured image.

The present invention has been made in view of the above problems, and controls to display a captured image in a direction desired by the user at a position where the amount of movement of the user's line of sight is reduced without the user's operation. It is to provide an information processing apparatus and an information processing method.

In order to solve the above-mentioned problems, the information processing apparatus according to one aspect of the present invention includes a controller in which data regarding the line-of-sight direction of the user of the vehicle is input. The controller identifies one or more image pickup devices out of a plurality of image pickup devices installed so as to be able to take an image around the vehicle based on the user's line-of-sight direction, and enters the vehicle interior of the vehicle based on the user's line-of-sight direction. One or a plurality of display units are specified among the plurality of arranged display units, and the captured image captured by the specified image pickup device is displayed on the specified display unit.

According to the present invention, the captured image in the direction desired by the user can be displayed at a position where the amount of movement of the line of sight of the user is reduced without the user having to operate the image.

FIG. 1 is a block diagram showing a configuration example of an information processing apparatus according to an embodiment of the present invention. FIG. 2 is a diagram for explaining an example of a display device according to an embodiment of the present invention. FIG. 3 is a diagram for explaining an example of a method for specifying an image pickup device and a display unit by the information processing device according to the embodiment of the present invention. FIG. 4 is a flowchart showing a processing flow of the information processing apparatus according to the embodiment of the present invention.

Hereinafter, embodiments to which the present invention is applied will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[Configuration example of information processing device]
A configuration example of the information processing apparatus 1 according to the present embodiment will be described with reference to FIG. The information processing device 1 is mounted on the vehicle. As shown in FIG. 1, the information processing apparatus 1 includes a microphone 11, a vehicle speed sensor 12, a line-of-sight sensor 13, and a plurality of imaging devices (right camera 21, left camera 22, and left camera 22) for imaging the surroundings of the vehicle. It includes a front camera 23, a rear camera 24), a display device 30, and a controller 40. The information processing device 1 is composed of only a controller 40, and is configured as a microphone 11, a vehicle speed sensor 12, a line-of-sight sensor 13, a plurality of image pickup devices, and a display device 30 as separate devices different from the information processing device. You may.

The microphone 11 is installed in the vehicle interior, and voice data from the user is input to the microphone 11. The location of the microphone 11 is, for example, near the driver's seat. The microphone 11 outputs the input voice data to the controller 40.

The vehicle speed sensor 12 detects the traveling speed (vehicle speed) of the vehicle. The vehicle speed sensor 12 includes, for example, a sensor rotor that rotates together with wheels and has a protrusion (gear pulser) formed on the circumference, and a detection circuit having a pickup coil provided so as to face the protrusion of the sensor rotor. The vehicle speed sensor 12 outputs the detected vehicle speed to the controller 40.

The line-of-sight sensor 13 is a sensor that detects the direction of the line of sight of the user of the vehicle (in this case, the driver), and is not shown in particular, but is an appropriate place in the vehicle interior (for example, near the ceiling portion on the front side of the front seat). It is installed in. For example, the line-of-sight sensor 13 has a camera that captures an image including the driver's head and face, and from the image of the head and face, the positions of a plurality of feature points such as the inner corner of the eye, the corner of the eye, and the nostrils included in the image are determined. To detect. Then, based on the detected position, the direction in which the face is directed on the image is specified, and that direction is detected as the "line-of-sight direction" of the driver. The line-of-sight direction is, for example, the position of the eyes with respect to the vehicle coordinate system composed of the front-back (X-axis), left-right (Y-axis), and height (Z-axis) coordinate axes of the vehicle centered on the position of the center of gravity of the vehicle. It is specified by the angle with respect to each coordinate axis in the line-of-sight direction at this eye position. The line-of-sight sensor 13 outputs the detected information in the line-of-sight direction of the user to the controller 40.

The plurality of image pickup devices are a plurality of in-vehicle cameras installed in the vehicle so as to be able to take an image of the surroundings, and have an image pickup element such as a CCD (charge-coupled device) or a CMOS (complementary metal accessory semiconductor). The plurality of image pickup devices include at least a first image pickup device (right camera 21) that images the right side of the vehicle outside the vehicle and a second image pickup device (left camera 22) that images the left side of the vehicle outside the vehicle. ..

The plurality of image pickup devices include, for example, a right camera 21, a left camera 22, a front camera 23, and a rear camera 24, as shown in FIG. Although not particularly shown, the right camera 21 and the left camera 22 are arranged on the right side and the left side of the vehicle (for example, the lower part of the door mirror), respectively, and image the right side outside the vehicle and the left side outside the vehicle, respectively. The right camera 21 can capture the blind spot of the driver on the right side outside the vehicle of the vehicle, and the left camera 22 can capture the blind spot of the driver on the left side outside the vehicle of the vehicle. The front camera 23 is arranged at the front portion of the vehicle (for example, the front grill) and images the front outside the vehicle. The rear camera 24 is arranged at the rear of the vehicle (for example, the rear gate) and images the rear of the vehicle outside the vehicle. Each in-vehicle camera can rotate in a predetermined angle range (ideally, a wide-angle range close to 180 °) in the horizontal plane in a horizontal plane based on an instruction from the controller 40 to change the imaging direction (lens orientation). It has become like. As a result, each in-vehicle camera can cooperate to take an image of the entire circumference of the own vehicle.

The display device 30 includes a plurality of display units arranged in the vehicle interior of the vehicle, and displays the captured image captured by the image pickup device on the display unit. The plurality of display units include at least a first display unit 31 arranged on the front right side of the driver and a second display unit 32 arranged on the front left side of the driver. The display device 30 displays the captured image captured by the image pickup device specified by the image pickup device specifying unit 45 or the entire surrounding image of the vehicle on the display unit specified by the display unit identification unit 46.

An example of the display device 30 will be described with reference to FIG. The display device 30 includes, for example, a first display unit 31, a second display unit 32, a third display unit 33, a fourth display unit 34, and a fifth display unit 35 horizontally arranged on an instrument panel in the vehicle interior. For example, the first display unit 31 is arranged at the right end of the instrument panel, and the second display unit 32 is arranged at the left end of the instrument panel. The third display unit 33 is provided near the center of the instrument panel. The fourth display unit 34 is provided between the first display unit 31 and the third display unit 33 in the front direction of the driver's seat. The fifth display unit 35 is provided between the second display unit 32 and the third display unit 33 in the front direction of the passenger seat. The plurality of display units of the display device 30 are not limited to the number and arrangement shown in FIG. The plurality of display units may be, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and the plurality of display units may be a touch panel.

The controller 40 is a general-purpose computer including a CPU (Central Processing Unit), a memory, a storage device, an input / output unit, and the like. The controller 40 processes the data acquired from the microphone 11, the vehicle speed sensor 12, the line-of-sight sensor 13, and each vehicle-mounted camera, and controls each vehicle-mounted camera and the display device 30. Specifically, the CPU reads various programs stored in the storage device into the memory and executes various instructions included in the programs. As a result, the controller 40 functions as a plurality of information processing circuits included in the information processing device 1. The controller 40 includes a voice recognition unit 41, a vehicle speed determination unit 42, a storage unit 43, a line-of-sight direction determination unit 44, an image pickup device identification unit 45, and a display unit identification unit 46 as a plurality of information processing circuits.

The voice recognition unit 41 acquires and recognizes the user's voice data input to the microphone 11 from the microphone 11. Well-known techniques such as frequency analysis are used for speech recognition. The voice recognition unit 41 recognizes an instruction from the user by recognizing the voice data of the user. For example, when the voice recognition unit 41 recognizes that the user's voice data is a "camera", the voice recognition unit 41 recognizes that an instruction to display a captured image around the vehicle on the display device 30 has been input.

The vehicle speed determination unit 42 acquires the vehicle speed from the vehicle speed sensor 12. The vehicle speed determination unit 42 determines whether or not the acquired vehicle speed is equal to or less than a preset predetermined value. The predetermined value at this time is set to a value having a somewhat slow speed, for example, 30 km / h.

The storage unit 43 acquires and stores information on the driver's line-of-sight direction from the line-of-sight sensor 13. The storage unit 43 includes an image pickup device that captures an image of a position on or near the driver's line-of-sight direction axis associated with the driver's line-of-sight direction, and a display unit located on or near the driver's line-of-sight axis. It is stored in advance. For example, it is stored that the image pickup device and the display unit corresponding to the case where the line-of-sight direction of the driver is determined to be the right side with respect to the line-of-sight direction directly in front of the driver are the right camera 21 and the first display unit 31. ing. Then, it is stored that the image pickup device and the display unit corresponding to the case where the line-of-sight direction of the driver is determined to be the left side with respect to the line-of-sight direction directly in front of the driver are the left camera 22 and the second display unit 32. ing.

The line-of-sight direction determination unit 44 acquires information on the driver's line-of-sight direction from the storage unit 43, and determines which direction the driver's line-of-sight direction is based on the line-of-sight direction directly in front of the driver. The line-of-sight direction determination unit 44 determines that, for example, when the line-of-sight direction of the driver is within a preset range on the right side with respect to the line-of-sight direction directly in front of the driver, the line-of-sight direction of the driver is the right side of the vehicle. judge. Then, when the driver's line-of-sight direction is within a preset range on the left side with respect to the line-of-sight direction directly in front of the driver, it is determined that the driver's line-of-sight direction is on the left side of the vehicle. The driver's line-of-sight direction determined by the line-of-sight direction determination unit 44 is not limited to the right side and the left side of the vehicle, and the driver's line-of-sight direction is set in advance with reference to the line-of-sight direction directly in front of the driver. The driver's line-of-sight direction may be determined by determining which range is within the set three or more ranges.

The line-of-sight direction determination unit 44 determines the line-of-sight direction of the driver when the line-of-sight direction of the driver is within the same range for a predetermined time or longer. The line-of-sight direction determination unit 44 may determine the line-of-sight direction of the driver when the line-of-sight direction of the driver is within the same range a predetermined number of times or more within a predetermined time.

The image pickup device identification unit 45 identifies one or a plurality of image pickup devices that image a position on or near the axis of the driver's line-of-sight direction among the plurality of image pickup devices based on the driver's line-of-sight direction. For example, when the driver's line-of-sight direction is either the right side or the left side of the vehicle, the image pickup device specifying unit 45 takes an image of one side of a plurality of image pickup devices that is the same as the driver's line-of-sight direction. To identify. Specifically, the image pickup device specifying unit 45 acquires the information of the image pickup device associated with the line-of-sight direction of the driver determined by the line-of-sight direction determination unit 44 from the storage unit 43, and identifies the image pickup device. The image pickup device identification unit 45 is based on the captured images captured by the right camera 21, the left camera 22, the front camera 23, and the rear camera 24 when the driver's line-of-sight direction is neither the right side nor the left side of the vehicle. , Generates an omnidirectional image of the vehicle. The image pickup device specifying unit 45 outputs the information of the specified image pickup device and the information of the generated omnidirectional image to the display device 30.

The display unit specifying unit 46 identifies one or a plurality of display units located on or near the axis of the driver's line-of-sight direction among the plurality of display units based on the driver's line-of-sight direction. For example, when the driver's line-of-sight direction is either the right side or the left side of the vehicle, the display unit identification unit 46 is one side of a plurality of display units in front of the driver, which is the same as the driver's line-of-sight direction. Specify the display unit placed at the position of. Specifically, the display unit specifying unit 46 acquires the information of the display unit associated with the line-of-sight direction of the driver determined by the line-of-sight direction determination unit 44 from the storage unit 43, and identifies the display unit. The display unit specifying unit 46 identifies a predetermined predetermined display unit among the plurality of display units when the line-of-sight direction of the driver is neither the right side nor the left side of the vehicle. The display unit specifying unit 46 outputs the information of the specified display unit to the display device 30.

An example of a method for specifying an image pickup apparatus and a display unit will be described with reference to FIG. FIG. 3 shows a scene in which the driver confirms the left side of the vehicle outside the vehicle when the vehicle travels on a narrow road avoiding the left side gutter. In the scene shown in FIG. 3, the line-of-sight direction determination unit 44 determines that the line-of-sight direction of the driver is on the left side. In this case, the image pickup device specifying unit 45 is stored in association with the direction of the line of sight to the left side of the driver among the right camera 21, the left camera 22, the front camera 23, and the rear camera 24. A left-hand camera 22 that captures an image on the left side outside the vehicle, which is the same as the line-of-sight direction, is specified. Further, the display unit specifying unit 46 is stored in association with the line-of-sight direction toward the left side of the driver among the first display units 31 to the fifth display unit 35, and is stored in front of the driver in the driver's line-of-sight direction. The second display unit 32 located at the same left position as the above is specified.

Then, the display device 30 causes the specified second display unit 32 to display the captured image captured by the specified left camera 22 on the left side of the outside of the vehicle. As a result, the information processing apparatus 1 displays the captured image on the left side of the outside of the vehicle, which is the direction the user wants to see in the scene of FIG. 3, on the display unit on the front left side near the user's line of sight. Can be done. Therefore, the user can check the captured image in the direction desired by the user without significantly moving the line of sight without switching the captured image or the display position with a switch or the like.

Next, an operation example of the information processing apparatus 1 according to the present embodiment will be described with reference to FIG.

In step S101 of FIG. 4, voice data from the user is input to the microphone 11. The microphone 11 outputs the acquired voice data to the controller 40. The voice recognition unit 41 recognizes the acquired voice data (for example, a “camera”) and recognizes that an instruction to display a captured image around the vehicle on the display device 30 has been input.

The process proceeds to step S102, and the vehicle speed sensor 12 outputs the detected vehicle speed to the controller 40. The vehicle speed determination unit 42 acquires the vehicle speed from the vehicle speed sensor 12. The process proceeds to step S103, and the vehicle speed determination unit 42 determines whether or not the acquired vehicle speed is equal to or less than a preset predetermined value. If the vehicle speed is equal to or less than a predetermined value (YES in step S103), the process proceeds to step S104. On the other hand, when the vehicle speed is higher than the predetermined value (NO in step S103), the information processing apparatus 1 ends the process of FIG.

In step S104, the line-of-sight sensor 13 detects the line-of-sight direction of the driver and outputs the detected line-of-sight direction information to the controller 40. The storage unit 43 acquires and stores information on the driver's line-of-sight direction from the line-of-sight sensor 13.

The line-of-sight direction determination unit 44 determines which direction the driver's line-of-sight direction is based on the line-of-sight direction directly in front of the driver. The line-of-sight direction determination unit 44 determines that the driver's line-of-sight direction is the right side of the vehicle when the driver's line-of-sight direction is within a preset range on the right side with respect to the line-of-sight direction directly in front of the driver. .. Then, when the driver's line-of-sight direction is within a preset range on the left side with respect to the line-of-sight direction directly in front of the driver, it is determined that the driver's line-of-sight direction is on the left side of the vehicle.

The process proceeds to step S105, and the line-of-sight direction determination unit 44 determines whether or not the line-of-sight direction of the driver is on the right side with respect to the line-of-sight direction directly in front of the driver. If the driver's line of sight is to the right (YES in step S105), the process proceeds to step S106. On the other hand, when the line-of-sight direction of the driver is not to the right (NO in step S105), the process proceeds to step S108.

In step S106, the image pickup device specifying unit 45 acquires the information of the image pickup device associated with the line-of-sight direction to the right side of the driver from the storage unit 43. Then, among the plurality of in-vehicle devices, an image pickup device (right camera 21) that images the right side of the outside of the vehicle, which is associated with the direction of the line of sight toward the right side of the driver, is specified. The image pickup device specifying unit 45 outputs the information of the specified image pickup device to the display device 30.

The process proceeds to step S107, and the display unit specifying unit 46 acquires the information of the display unit associated with the line-of-sight direction to the right side of the driver from the storage unit 43. Then, among the plurality of display units, the display unit (first display unit 31) arranged at the position on the front right side of the user, which is associated with the line-of-sight direction toward the right side of the driver, is specified. The display unit specifying unit 46 outputs the information of the specified display unit to the display device 30.

In step S108, the line-of-sight direction determination unit 44 determines whether or not the line-of-sight direction of the driver is on the left side. If the driver's line of sight is to the left (YES in step S108), the process proceeds to step S109. On the other hand, when the line-of-sight direction of the driver is not to the left (NO in step S108), the process proceeds to step S111.

In step S109, the image pickup device specifying unit 45 acquires the information of the image pickup device associated with the line-of-sight direction to the left side of the driver from the storage unit 43. Then, among the plurality of in-vehicle devices, an image pickup device (left camera 22) that images the left side of the outside of the vehicle, which is associated with the direction of the line of sight toward the left side of the driver, is specified. The image pickup device specifying unit 45 outputs the information of the specified image pickup device to the display device 30.

The process proceeds to step S110, and the display unit specifying unit 46 acquires the information of the display unit associated with the line-of-sight direction toward the left side of the driver from the storage unit 43. Then, among the plurality of display units, the display unit (second display unit 32) arranged at the position on the front left side of the user, which is associated with the driver's line-of-sight direction to the right side, is specified. The display unit specifying unit 46 outputs the information of the specified display unit to the display device 30.

In step S111, the image pickup device specifying unit 45 generates an omnidirectional image of the vehicle from the images captured by the right camera 21, the left camera 22, the front camera 23, and the rear camera 24. The image pickup device identification unit 45 outputs the information of the generated omnidirectional image to the display device 30.

The process proceeds to step S112, and the display unit specifying unit 46 identifies a predetermined display unit (for example, a third display unit 33) set in advance from among the plurality of display units. The display unit specifying unit 46 outputs the information of the specified display unit to the display device 30.

The process proceeds to step S113, and the display device 30 causes the display unit specified by the display unit identification unit 46 to display the image captured by the image pickup device specified by the image pickup device identification unit 45 or the omnidirectional image. The display device 30 displays an image captured by the image pickup device specified by the image pickup device specifying unit 45 or an all-around image for a predetermined time (for example, 5 seconds).

The process proceeds to step S114, and the line-of-sight sensor 13 detects the line-of-sight direction of the driver and outputs the detected line-of-sight direction information to the controller 40. The storage unit 43 acquires and stores information on the driver's line-of-sight direction from the line-of-sight sensor 13. The line-of-sight direction determination unit 44 determines which direction the driver's line-of-sight direction is based on the line-of-sight direction directly in front of the driver.

The process proceeds to step S115, and the line-of-sight direction determination unit 44 determines whether or not the line-of-sight direction of the driver determined in step S114 has changed from the line-of-sight direction determined in step S105. When the line-of-sight direction of the driver changes from the line-of-sight direction determined in step S105 (YES in step S115), the process returns to step S105. When the line-of-sight direction of the driver does not change from the line-of-sight direction determined in step S105 (NO in step S115), the information processing apparatus 1 ends the process of FIG.

[Effect of this embodiment]
As described above, the information processing apparatus according to the present embodiment includes a controller for inputting data regarding the line-of-sight direction of the user of the vehicle. The controller identifies one or more image pickup devices out of a plurality of image pickup devices installed so as to be able to take an image around the vehicle based on the user's line-of-sight direction, and enters the vehicle interior of the vehicle based on the user's line-of-sight direction. One or a plurality of display units are specified among the plurality of arranged display units, and the captured image captured by the specified image pickup device is displayed on the specified display unit.

Thereby, it is possible to specify the image pickup device that captures the user's line-of-sight direction among the plurality of image pickup devices, and it is possible to specify the display unit that is in the user's line-of-sight direction among the plurality of display units. Therefore, the captured image captured in the direction desired by the user can be displayed in the direction of the user's line of sight, and the captured image in the direction desired by the user can be displayed in the direction desired by the user without the user performing a switching operation with a switch or the like. It can be displayed at a position where the amount of movement of the line of sight is reduced.

Further, the information processing apparatus according to the present embodiment specifies one or a plurality of imaging devices that image a position on or near the axis in the line-of-sight direction of the user among the plurality of imaging devices. Thereby, among the plurality of image pickup devices, it is possible to specify the image pickup device that captures the vicinity position in the line-of-sight direction of the user, and it is possible to specify the image pickup device that captures the vicinity in the direction that the user wants to see. As a result, the captured image in the vicinity in the direction desired by the user can be displayed without the user performing a switching operation with a switch or the like.

The information processing apparatus according to the present embodiment identifies one or a plurality of display units located at or near the axis in the line-of-sight direction of the user among the plurality of display units. Thereby, among the plurality of display units, the display unit located near the position in the line-of-sight direction of the user can be specified. As a result, the captured image can be displayed in the vicinity of the user's line-of-sight direction without the user performing a switching operation with a switch or the like. Therefore, it is possible to reduce the amount of movement of the line of sight when the user confirms the captured image.

The plurality of display units of the information processing apparatus according to the present embodiment include at least a first display unit arranged on the front right side of the user and a second display unit arranged on the front left side of the user. As a result, when the user's line-of-sight direction is on the right side of the vehicle, the captured image is displayed on the display unit on the front right side of the user, and when the user's line-of-sight direction is on the left side of the vehicle, the front left side of the user. The captured image can be displayed on the display unit. Therefore, it is possible to reduce the amount of movement of the line of sight when the user confirms the captured image.

The plurality of image pickup devices of the information processing device according to the present embodiment include at least a first image pickup device that images the right side of the vehicle and a second image pickup device that images the left side of the vehicle. As a result, when the user's line of sight is on the right side of the vehicle, an image pickup device that images the right side of the vehicle is specified, and when the user's line of sight is on the left side of the vehicle, the left side of the vehicle is imaged. It is possible to specify the image pickup device to be used. Therefore, it is possible to specify an image pickup device that captures an image in the direction that the user wants to see.

The first imaging device captures the blind spot on the right side of the vehicle, and the second imaging device captures the blind spot on the left side of the vehicle. As a result, when the user's line of sight is on the right side of the vehicle, an imaging device that captures the blind spot on the right side of the vehicle is specified, and when the user's line of sight is on the left side of the vehicle, the left side of the vehicle is identified. It is possible to identify an imaging device that captures the blind spot of.

The information processing apparatus according to the present embodiment captures an image of one side of a plurality of imaging devices that is the same as the user's line-of-sight direction when the user's line-of-sight direction is either the right side or the left side of the vehicle. The device is specified, and among the plurality of display units, the display unit located in front of the user and on the same side as the user's line-of-sight direction is specified. As a result, when the user's line-of-sight direction is on the right side of the vehicle, the captured image on the right side of the vehicle can be displayed on the display unit on the front right side of the user, and when the user's line-of-sight direction is on the left side of the vehicle. , The captured image on the left side of the vehicle can be displayed on the display unit on the front left side of the user. Therefore, the captured image in the direction that the user wants to see can be displayed at a position that reduces the amount of movement of the user's line of sight, without the user performing a switching operation with a switch or the like.

The plurality of image pickup devices of the information processing device according to the present embodiment further include a third image pickup device that images the front of the vehicle and a fourth image pickup device that images the rear of the vehicle, and the information processing device is a user. When the line-of-sight direction of the vehicle is neither the right side nor the left side of the vehicle, an omnidirectional image is generated from the images captured by the first to fourth image pickup devices, and a predetermined display unit among the plurality of display units is generated. It is specified, and the omnidirectional image is displayed on the specified predetermined display unit. This allows the user to check the captured image outside the vehicle even when the user's line-of-sight direction cannot be determined.

The information processing apparatus according to the present embodiment displays a captured image when it recognizes a voice instruction from a user. As a result, at the timing when the user wants to confirm the captured image outside the vehicle, the captured image in the direction the user wants to see can be displayed at a position where the amount of movement of the user's line of sight is reduced. Further, since the processing is not executed when the user does not need to confirm the captured image, the processing load of the information processing apparatus can be reduced.

The information processing apparatus according to the present embodiment displays a captured image when the speed of the vehicle is equal to or less than a predetermined value. This can improve the safety when the user confirms the captured image.

As mentioned above, embodiments of the invention have been described, but the statements and drawings that form part of this disclosure should not be understood to limit the invention. This disclosure will reveal to those skilled in the art various alternative embodiments, examples and operational techniques.

For example, the controller 40 may use map information data or weather information when determining the line-of-sight direction of the driver. For example, when there is a dangerous object outside the vehicle, the direction of the user's line of sight may be determined when the direction of the driver's line of sight matches the direction of the dangerous object outside the vehicle, and when fog or the like is generated outside the vehicle. , The driver's line-of-sight direction may be determined.

The microphone 11 is not limited to the one installed in the vehicle. A microphone of a mobile terminal (for example, a smartphone) may be substituted. In this case, the mobile terminal is connected so as to be able to communicate with the controller 40.

40 Controller 21-24 Left camera, right camera, front camera, rear camera (multiple imaging devices)
31-35 1st display unit, 2nd display unit, 3rd display unit, 4th display unit, 5th display unit (multiple display units)
1 Information processing equipment

Claims (11)

It has a controller that inputs data about the direction of the vehicle user's line of sight.
The controller
Based on the line-of-sight direction of the user, one or a plurality of image pickup devices among a plurality of image pickup devices installed so as to be able to image the surroundings of the vehicle are identified.
Based on the line-of-sight direction of the user, one or a plurality of display units among the plurality of display units arranged in the vehicle interior of the vehicle are specified.
An information processing device characterized in that an image captured by the specified image pickup device is displayed on the specified display unit. The information processing device according to claim 1, wherein the controller identifies one or a plurality of image pickup devices that image a position on or near the axis in the line-of-sight direction of the user among the plurality of image pickup devices. .. The information processing according to claim 1 or 2, wherein the controller identifies one or a plurality of display units located at or near the axis in the line-of-sight direction of the user among the plurality of display units. Device. The plurality of display units according to claim 1 to 3, wherein the plurality of display units include at least a first display unit arranged on the front right side of the user and a second display unit arranged on the front left side of the user. The information processing apparatus according to any one of the following items. The plurality of image pickup devices according to claim 1 to 4, wherein the plurality of image pickup devices include at least a first image pickup device that images the right side of the vehicle and a second image pickup device that images the left side of the vehicle. The information processing device according to any one of the items. The information processing according to claim 5, wherein the first image pickup apparatus images a blind spot on the right side of the vehicle, and the second image pickup device images a blind spot on the left side of the vehicle. Device. The controller captures an image pickup device that captures one side of the plurality of image pickup devices that is the same as the line-of-sight direction of the user when the line-of-sight direction of the user is either the right side or the left side of the vehicle. The fifth or sixth aspect of the present invention is characterized in that, among the plurality of display units, a display unit arranged in front of the user at a position on the same side as the line-of-sight direction of the user is specified. Information processing equipment. The plurality of image pickup devices further include a third image pickup device that images the front of the vehicle and a fourth image pickup device that images the rear of the vehicle.
The controller generates an omnidirectional image from the captured images captured by the first to fourth image pickup devices when the line-of-sight direction of the user is neither the right side nor the left side of the vehicle, and the plurality of peripheral images are generated. Identify the predetermined display part of the display part,
The information processing apparatus according to claim 7, wherein the omnidirectional image is displayed on the specified predetermined display unit. The information processing apparatus according to any one of claims 1 to 8, wherein the controller displays a captured image when it recognizes an instruction by voice from the user. The information processing device according to any one of claims 1 to 9, wherein the controller displays the captured image when the speed of the vehicle is equal to or less than a predetermined value. Detects the direction of the vehicle user's line of sight and
Based on the line-of-sight direction of the user of the vehicle, one or a plurality of image pickup devices among a plurality of image pickup devices installed so as to be able to image the surroundings of the vehicle are identified.
Based on the line-of-sight direction of the user, one or a plurality of display units of the plurality of display units arranged in the vehicle interior of the vehicle are specified.
An information processing method characterized by displaying an image captured by the specified image pickup device on the specified display unit.

Images