WO2021053843A1

WO2021053843A1 - Image display system, image display method, and computer program

Info

Publication number: WO2021053843A1
Application number: PCT/JP2020/002884
Authority: WO
Inventors: 祥生石黒
Original assignee: 国立大学法人東海国立大学機構
Priority date: 2019-09-18
Filing date: 2020-01-28
Publication date: 2021-03-25
Also published as: JPWO2021053843A1

Abstract

The present invention improves the convenience of an image display system. This image display system is provided with: an image display device; a support device that supports the image display device and makes it possible to change the orientation of the image display device; and a control device for controlling the image display device and the support device. The control device includes a space information acquisition unit, an orientation control unit, and an image display control unit. The space information acquisition unit acquires space information including first position information indicating the position of the image display device and second position information indicating the position of an object in the vicinity of the image display device. The orientation control unit causes the support device to change the orientation of the image display device on the basis of the space information. The image display control unit causes the image display device to display an image corresponding to the orientation of the image display device.

Description

Image display system, image display method, and computer program

The technology disclosed in this specification relates to an image display system.

In recent years, the development of automatic driving technology for vehicles such as automobiles is rapidly progressing. The automatic driving technology of a vehicle is aimed at a vehicle by recognizing the position of the own vehicle and the surrounding environment by using various sensors (camera, radar, LIDAR, ultrasonic sensor, GPS, etc.) and a high-precision 3D map, for example. It is a technology that autonomously travels to the ground.

The automatic driving technology of the vehicle has been sufficiently improved, and the system performs all driving operations such as acceleration, steering, and braking under specific circumstances (for example, while driving on the highway) or under all circumstances. When the driver no longer needs to perform the driving operation, the driver can devote time as a passenger to something other than the driving operation. For example, the passenger is an image display system (infotainment system) including an in-vehicle display used for a car navigation system or the like, a smartphone, a tablet terminal, an image display device such as an HMD (Head Mounted Display) brought by the passenger. ) Can be used to watch a moving image or play a game (see, for example, Patent Documents 1 to 3).

JP-A-2017-49901 Japanese Unexamined Patent Publication No. 2017-97921 JP-A-2017-81256

In an image display system for passengers of an autonomous vehicle, for example, when introducing a tourist attraction seen from a vehicle window, an image corresponding to the direction (for example, an object located in the direction (for example, an object (for example, for example)) while indicating the direction is shown. , A historic building) may be desired to be displayed on an image display device (for example, an image showing the name, size, year of construction, etc. of the building). In such a case, an image (for example, an image of an arrow pointing to the right) may be displayed on the screen of an in-vehicle display, a smartphone, or a tablet terminal, or a voice (for example, "Please look at your right hand") may be heard from a speaker. Even if it is output, it is difficult for the passenger to tell the exact direction to see. Further, although it is possible to convey an accurate direction by using a wearable image display device such as an HMD, the wearable image display device is troublesome to wear and has a strange appearance when worn. As described above, in an image display system including a conventional image display device, it is difficult to display an image corresponding to the direction on the image display device while showing an accurate direction without using the image display device on the mounting side. There is room for improvement in terms of convenience.

It should be noted that such a problem is not limited to the image display system mounted on the autonomous driving vehicle, but is a common problem in general for the image display system including the image display device.

This specification discloses a technique capable of solving the above-mentioned problems.

The technology disclosed in the present specification can be realized, for example, in the following form.

(1) The image display system disclosed in the present specification is an image display system, and includes an image display device and a support device that supports the image display device and can change the orientation of the image display device. The control device includes a control device for controlling the image display device and the support device, and the control device includes first position information indicating the position of the image display device and the position of an object around the image display device. A spatial information acquisition unit that acquires spatial information including a second position information indicating the above, an orientation control unit that causes the support device to change the orientation of the image display device based on the spatial information, and the image display device. It has an image display control unit for displaying an image according to the orientation of the image on the image display device. In this image display system, the orientation of the image display device is based on spatial information including a first position information indicating the position of the image display device and a second position information indicating the position of an object around the image display device. Is changed, and an image corresponding to the orientation of the image display device is displayed on the image display device. Therefore, according to this image display system, it is possible to display an image corresponding to the direction on the image display device while showing the user an accurate direction depending on the orientation of the image display device without using the image display device on the mounting side. This makes it possible to improve the convenience of the image display system.

(2) In the image display system, the control device further has a target setting unit that sets one of the peripheral objects as a target object based on the spatial information, and the orientation control unit is the image. The support device may be configured to change the orientation of the image display device so that the display device is oriented to point to the target object. According to this image display system, the direction of the user's line of sight can be guided to a target object, which is one of the surrounding objects, depending on the orientation of the image display device, further improving the convenience of the image display system. Can be done.

(3) In the image display system, the control device further has a target information acquisition unit that acquires target information that is information about the target object, and the image display control unit displays an image showing the target information. It may be configured to be displayed on the image display device. According to this image display system, the direction of the user's line of sight can be guided to a target object, which is one of the surrounding objects, depending on the orientation of the image display device, and an image showing information about the target object is imaged. Since it can be displayed on the display device, the convenience of the image display system can be further improved.

(4) In the image display system, the target information may include information indicating at least one of the name, size, and year of construction of the target object. According to this image display system, an image showing information indicating at least one of the name, size, and year of construction of the target object can be displayed on the image display device, and useful information is presented to the user. be able to.

(5) In the image display system, the support device supports the image display device so that the orientation of the image display device can be manually changed, and the target setting unit manually changes the orientation of the image display device. When the orientation is changed, an object located in the direction indicated by the image display device may be set as the target object. According to this image display system, when the user points the image display device in the direction of a desired object, the object is set as the target object, so that the target object can be set by the user's intuitive operation. , The convenience of the image display system can be further improved.

(6) In the image display system, the image display device has a camera that captures a subject located in the direction indicated by the image display device, and the image display control unit manually orients the image display device. When the change is made, the image captured by the camera may be displayed on the image display device. According to this image display system, the orientation of the current image display device can be fed back to the user by the image displayed on the image display device, so that the user can easily perform the image display device accurately on the desired object. The orientation of the image display device can be manually changed so that the image display device is in the pointing state, and the convenience of the image display system can be further improved.

(7) In the image display system, the image display device has a flat image display surface, the direction indicated by the image display device is orthogonal to the image display surface, and the image display surface is the same as the image display surface. The configuration may be such that the direction is toward the opposite side. According to this image display system, the user can more accurately grasp the direction pointed by the image display device, and the convenience of the image display system can be further improved.

(8) In the image display system, the support device may have a configuration having three or more rotation axes. According to this image display system, the orientation of the image display device can be adjusted more flexibly, so that the convenience of the image display system can be further improved.

(9) In the image display system, the image display system is mounted on a vehicle capable of autonomous driving, the support device supports the image display device on the vehicle, and the spatial information acquisition unit is mounted on the vehicle. The spatial information obtained by the installed sensor may be acquired. According to this image display system, by changing the direction of the image display device using the information used for automatic driving, it is related to the running of the vehicle for the passengers of the vehicle capable of automatic driving and the people outside the vehicle. It is possible to display an image corresponding to the direction on the image display device while indicating the exact direction to be performed, and it is possible to improve the convenience of the image display system.

The techniques disclosed in the present specification can be realized in various forms, for example, an image display device, an image display system, an image display method, a computer program that realizes those methods, and a computer program thereof. It can be realized in the form of a non-temporary recording medium or the like in which the above is recorded.

Explanatory drawing which shows the schematic structure of the image display system 10 in this embodiment. A flowchart showing the contents of the image display control process in the present embodiment Explanatory drawing schematically showing the state of the image display device 150 and the like at the time of the image display control processing in this embodiment. Explanatory drawing schematically showing the state of the image display device 150 and the like at the time of the image display control processing in this embodiment. Explanatory drawing schematically showing the state of the image display device 150 and the like at the time of the image display control processing in this embodiment. Explanatory drawing schematically showing the image display device 150 and the like in this embodiment.

A. Embodiment:
A-1. Configuration of image display system 10:
FIG. 1 is an explanatory diagram showing a schematic configuration of an image display system 10 according to the present embodiment. The image display system 10 of the present embodiment is an infotainment system that is mounted on a vehicle VH capable of autonomous driving and can provide infotainment such as videos, games, and tourist information to the passenger PA of the vehicle VH.

As shown in FIG. 1, the image display system 10 includes an image display device 150, a support device 160, and a control device 100. The image display device 150 includes a display such as a liquid crystal display or an organic EL display, and displays an image such as a still image or a moving image. The shape of the image display device 150 is a substantially flat plate having a substantially flat front surface FS (that is, an image display surface) and a surface opposite to the front surface FS (hereinafter, referred to as “back surface BS”). The image display device 150 has a camera (rear camera) 152 in which a lens is arranged on the rear BS. The camera 152 captures a subject located in the back BS direction of the image display device 150 and generates an image. The image display device 150 may include other components such as a front camera and a speaker in which a lens is arranged on the front FS. Further, the image display device 150 may be provided with a touch panel, a microphone, or the like, and may function as an input device.

The support device 160 is a device that is fixed to a predetermined position (dashboard in this embodiment) of the vehicle VH and supports the image display device 150. In this embodiment, the support device 160 is composed of a robot arm having a large number of joints (three or more) and a drive motor 162 for driving each joint. The support device 160 can change the position and orientation of the image display device 150 by driving each joint by each drive motor 162 (see FIGS. 3 and 4). Hereinafter, such a change in the position and orientation of the image display device 150 will be referred to as an "automatic change" in the position and orientation of the image display device 150.

Further, the support device 160 of the present embodiment sets the position and orientation of the image display device 150 by the action of a force on the image display device 150 or the support device 160 from the outside by making the torque of each drive motor 162 zero. The image display device 150 is configured to be changeable and the position and orientation of the image display device 150 to be fixed when the action of the force disappears. Therefore, for example, the passenger PA can change the position and orientation of the image display device 150 to a desired position and orientation by grasping the image display device 150 and moving or rotating the image display device 150. (See FIG. 5). Hereinafter, such a change in the position and orientation of the image display device 150 is referred to as a "manual change" in the position and orientation of the image display device 150.

The control device 100 is a computer for controlling each component of the vehicle VH, and includes a control unit 110 and a storage unit 170. Each of these parts is communicably connected to each other via a bus 190.

The storage unit 170 of the control device 100 is composed of, for example, a ROM, a RAM, a hard disk drive (HDD), or the like, and temporarily stores a work area or data when storing various programs or data or executing various programs. It is used as a storage area. For example, the storage unit 170 includes an automatic driving control program 172, which is a computer program for executing automatic driving of the vehicle VH, an image display control program 174, which is a computer program for executing image display control processing described later, and the storage unit 170. The support control program 176 is stored. These programs are provided in a state of being stored in a computer-readable recording medium (not shown) such as a CD-ROM, a DVD-ROM, or a USB memory, and can be installed in the control device 100 in the storage unit 170. It is stored.

Further, a three-dimensional map database MD is constructed in the storage unit 170 of the control device 100. The three-dimensional map database MD stores a high-precision three-dimensional map used for automatic driving of the vehicle VH. This three-dimensional map includes, for example, information on the position and shape of traffic lights, road signs, roadside trees, guardrails, buildings, etc., in addition to information on the position and shape of roads.

Further, an infotainment database ID is constructed in the storage unit 170 of the control device 100. The infotainment database ID stores information for providing infotainment such as videos, games, and tourist information to the passenger PA of the vehicle VH. In the present embodiment, in order to provide tourist information to the passenger PA of the vehicle VH, information indicating the position of an object (more specifically, a tourist attraction such as a building or a natural object) in the infotainment database ID (more specifically, information indicating the position of an object (more specifically, a tourist attraction such as a building or a natural object) ( Hereinafter, "tourist attraction location information I2") and information indicating specifications such as the appearance, name, size, and year of construction of the tourist attraction (hereinafter referred to as "specification information I3") are stored. ing.

The control unit 110 of the control device 100 is composed of, for example, a CPU or the like, and controls the operation of each unit of the vehicle VH by executing a computer program read from the storage unit 170. For example, the control unit 110 controls the automatic driving of the vehicle VH by reading the automatic driving control program 172 from the storage unit 170 and executing it. At this time, the control unit 110 functions as the automatic operation control unit 120. The automatic operation control unit 120 includes a position / orientation detection unit 122, a peripheral situation detection unit 124, and a route setting unit 126. The functions of each of these parts will be described in accordance with the description of the automatic operation control process described later.

Further, the control unit 110 controls the image display device 150 and the support device 160 by reading and executing the image display control program 174 and the support control program 176 from the storage unit 170. At this time, the control unit 110 functions as an image display control unit 130 and a support control unit 140. The image display control unit 130 includes a spatial information acquisition unit 132, a target setting unit 134, and a target information acquisition unit 136. Further, the support control unit 140 includes an orientation control unit 142. The functions of each of these parts will be described in accordance with the description of the image display control process described later.

A-2. Automatic operation control processing:
Next, the automatic operation control process by the automatic operation control unit 120 of the control device 100 will be described. The automatic driving control process is a process for controlling the automatic driving of the vehicle VH. The content of the automatic operation control process in the present embodiment is the same as the known content, and is approximately as follows.

First, the route setting unit 126 of the automatic driving control unit 120 sets a movement route (hereinafter referred to as "route") from the departure point to the destination according to the instruction of the passenger PA. The route can be set by various methods. For example, the route can be set by referring to the road network database in which the road is represented by nodes and links and searching the route based on the reference.

When the route is set, the automatic driving control unit 120 autonomously drives the vehicle VH by controlling the acceleration, steering, and braking of the vehicle VH so as to travel on the set route. At this time, the position / orientation detection unit 122 of the automatic operation control unit 120 is based on signals from various sensors 102 (GPS, geomagnetic sensors, etc.) mounted on the vehicle VH, or in the signal and the three-dimensional map database MD. Based on the stored high-precision 3D map, the position (latitude, longitude, altitude, etc.) and orientation (direction of travel direction, azimuth, etc.) of the vehicle VH are detected. Further, the peripheral situation detection unit 124 of the automatic driving control unit 120 is based on or is based on a signal from various sensors 102 (camera, radar, LIDAR, ultrasonic sensor, GPS, etc.) mounted on the vehicle VH, or is 3 Based on the high-precision 3D map stored in the dimensional map database MD, the situation around the vehicle VH (position and shape of objects (other vehicles, pedestrians, street trees, lanes, etc.) around the vehicle VH) Is detected. The automatic driving control unit 120 sets a destination along a set route based on the position and orientation of the vehicle VH detected by the position / orientation detection unit 122 and the surrounding condition detected by the peripheral condition detection unit 124. The vehicle VH is driven autonomously up to.

A-3. Image display control processing:
Next, the image display control process executed by the image display system 10 of the present embodiment will be described. FIG. 2 is a flowchart showing the contents of the image display control process in the present embodiment. 3 to 5 are explanatory views schematically showing a state of the image display device 150 and the like during the image display control process in the present embodiment. The image display control process in the present embodiment is information that introduces tourist attractions that can be seen from the vehicle window as infotainment using the image display device 150 to the passenger PA of the vehicle VH traveling toward the destination by automatic driving. This is a process for presenting, for example, when the passenger PA gives an instruction to start the process. FIG. 3 shows the state of the image display device 150 at the start of the image display control process. As shown in FIG. 3, at the start of the image display control process, the image display device 150 is located in front of the passenger PA sitting facing forward by the support device 160, and the front FS is positive to the passenger PA. The position and orientation are such that they face each other.

When the image display control process is started, the spatial information acquisition unit 132 of the image display control unit 130 acquires the own vehicle position information I1 indicating the position of the vehicle VH (S110). In the present embodiment, the own vehicle position information I1 includes information indicating the direction of the vehicle VH in addition to the position of the vehicle VH. Further, in the present embodiment, the spatial information acquisition unit 132 determines the position and orientation of the vehicle VH detected by the position / orientation detection unit 122 of the automatic driving control unit 120 based on the signals from various sensors 102 in the above-mentioned automatic driving control process. Information is acquired as own vehicle position information I1. Since the image display system 10 including the image display device 150 is mounted on the vehicle VH, the own vehicle position information I1 indicating the position and orientation of the vehicle VH is information indicating the position and orientation of the image display device 150. It can be said that there is. The position and orientation of the image display device 150 referred to here is not the "relative" position and orientation with respect to the vehicle VH, but the "absolute" position and orientation with respect to features such as roads. For example, even if the relative position and orientation of the image display device 150 with respect to the vehicle VH are fixed, if the position and orientation of the vehicle VH fluctuates as the vehicle VH travels, the image display device 150 with respect to the feature said. Absolute position and orientation also fluctuate. The own vehicle position information I1 is an example of the first position information and spatial information in the claims.

Next, the image display control unit 130 cooperates with the support control unit 140 to determine whether or not the orientation of the image display device 150 described above has been manually changed (S120). Whether or not the orientation of the image display device 150 has been manually changed can be determined, for example, by detecting with a sensor whether or not each joint of the support device 160 has moved without being driven by the drive motor 162.

When it is determined that the orientation of the image display device 150 has not been manually changed (S120: NO), the target setting unit 134 of the image display control unit 130 has an object (sightseeing spot) that can be introduced around the vehicle VH. Whether or not it is determined (S130). More specifically, as described above, the tourist attraction location information I2 is stored in the infotainment database ID, and the tourist attraction location information I2 is acquired by the spatial information acquisition unit 132. The target setting unit 134 determines the position of the vehicle VH at the current position based on the relationship between the position of the vehicle VH specified by the own vehicle position information I1 and the position of each tourist attraction specified by the acquired tourist attraction position information I2. Determine if there is a tourist attraction that can be seen from the train window. In addition, the correspondence relationship between each position on the road or the parking lot and the tourist attraction that can be seen from the corresponding relationship is set in advance, the correspondence relationship is stored in the infotainment database ID, and the target setting unit 134 performs the correspondence. The determination of S130 may be performed with reference to the correspondence. Tourist attraction location information I2 is an example of the second location information and spatial information in the claims.

In the above S130, when it is determined that there is an object (sightseeing spot) that can be introduced around the vehicle VH (S130: YES), the target setting unit 134 sets one of such tourist spots as the target object TA. The target information acquisition unit 136 acquires the specification information I3 of the target object TA from the infotainment database ID (S140). Specification information I3 of the target object TA is information indicating the specifications (appearance, name, size, year of construction, etc.) of the target object TA. The specification information I3 of the target object TA is an example of the target information in the claims.

Next, the orientation control unit 142 of the support control unit 140 is oriented so that the image display device 150 points to the target object TA based on the own vehicle position information I1 and the tourist attraction position information I2 of the target object TA. , The support device 160 is made to change the orientation of the image display device 150 (S150). Here, in the present specification, the direction pointed to by the image display device 150 is a direction orthogonal to the front surface (image display surface) FS of the image display device 150 and toward the side opposite to the front surface FS (rear surface BS side). Is. That is, the state in which the image display device 150 is oriented so as to point to the target object TA means an arbitrary point (for example, the center of the screen) on the front surface (image display surface) FS of the image display device 150, as shown in FIG. It is a state in which a virtual straight line VL that passes through the point P0) and is orthogonal to the front surface FS intersects the target object TA. The orientation control unit 142 specifies the position and orientation of the vehicle VH (that is, the position and orientation of the image display device 150) based on the own vehicle position information I1 and the position of the target object TA based on the tourist attraction position information I2. Then, the direction toward the target object TA (the direction of the virtual straight line VL) is calculated from the image display device 150, and the drive motor 162 is controlled to drive each joint of the support device 160 based on the direction to drive the image. The orientation of the display device 150 is changed. FIG. 4 shows a state in which the image display device 150 is changed in an orientation so as to point to the first target object TA1. In response to the change in the orientation of the image display device 150, the passenger PA shifts the line of sight in the direction pointed to by the image display device 150 (that is, the direction in which the target object TA is located). In the present embodiment, the support device 160 is composed of a robot arm having a large number of joints, and the position can be changed in addition to the orientation of the image display device 150. Therefore, the orientation of the image display device 150 In addition, the position of the image display device 150 has also been changed. The changed position of the image display device 150 is a position where the virtual straight line VL intersects the passenger PA (that is, a position sandwiched between the target object TA and the passenger PA). At this position, the front FS of the image display device 150 is substantially orthogonal to the line-of-sight direction of the passenger PA looking at the target object TA, so that the visibility of the image displayed on the image display device 150 by the passenger PA is improved. improves.

Further, the image display control unit 130 causes the image display device 150 to display an image showing the specification information I3 based on the specification information I3 of the target object TA acquired in S140 (S160). FIG. 4 shows a state in which an image showing the specifications (appearance, name, size, etc.) of the first target object TA1 is displayed on the image display device 150 as the specification information I3 of the first target object TA1. It is shown. A part of the specification information I3 may be output as sound from a speaker (not shown) included in the image display device 150. By displaying the image showing the specification information I3 on the image display device 150, the passenger PA has the appearance of the actual first target object TA1 and the first displayed on the image display device 150 without moving the line of sight. The contents of the specification information I3 of the target object TA1 of 1 can be visually recognized.

If it is determined in S130 that there is no object (sightseeing spot) that can be introduced around the vehicle VH (S130: NO), the processes of S140 to S160 are skipped.

After that, the image display control unit 130 determines whether or not the user has arrived at the destination (S200), and if it is determined that the image display control unit 130 has not yet arrived at the destination (S200: NO), the above-mentioned S110 or later Repeat the process.

As the vehicle VH travels, the position and orientation of the vehicle VH change from moment to moment. Therefore, if the relative position and orientation of the image display device 150 with respect to the vehicle VH is fixed, the image display device for features such as roads will be changed according to the change in the position and orientation of the vehicle VH as the vehicle VH travels. The absolute position and orientation of the 150 also changes from moment to moment. Therefore, even after the target object TA is set in S140 and the image display device 150 is changed to the direction pointing to the target object TA in S150, the position and orientation of the vehicle VH will be changed according to the traveling of the vehicle VH. The relative position and orientation of the image display device 150 with respect to the vehicle VH is adjusted so that the image display device 150 continues to point to the target object TA.

Also, as the vehicle VH travels, the tourist attractions that can be seen from the vehicle window of the vehicle VH change from moment to moment. Therefore, the target object TA is set in S140, the orientation of the image display device 150 is changed in S150, and an image showing the specification information I3 of the target object TA is displayed on the image display device 150 in S160, and then the vehicle VH. When the target object TA cannot be seen from the vehicle window in response to the change in the position and orientation of the vehicle VH due to the traveling of the vehicle, the processing after S130 is executed again.

Further, in S130, when it is determined that there are a plurality of objects (sightseeing spots) that can be introduced around the vehicle VH, one tourist attraction is set as the target object TA in S140. The target object TA may be switched every time the time elapses or every time there is an instruction from the passenger PA. That is, in S140, another tourist attraction is newly set as the target object TA, the direction of the image display device 150 is changed to the direction pointing to the new target object TA in S150, and the direction of the new target object TA is changed in S160. An image showing the specification information I3 may be displayed on the image display device 150.

If the orientation of the image display device 150 is manually changed during the image display control process, it is determined in S120 that the orientation of the image display device 150 is manually changed (S120: YES). In FIG. 5, during the image display control process, the passenger PA grabs the image display device 150 and points to the orientation (and position) of the image display device 150, and the image display device 150 points to the second target object TA2. It is shown that the orientation (and position) has been changed. When it is determined that the orientation of the image display device 150 has been manually changed (S120: YES), the target setting unit 134 of the image display control unit 130 can introduce an object (introducable object) in the direction indicated by the image display device 150 (S120: YES). It is determined whether or not there is a tourist attraction) (S170). More specifically, as described above, the tourist attraction location information I2 is stored in the infotainment database ID, and the tourist attraction location information I2 is acquired by the spatial information acquisition unit 132. The target setting unit 134 uses the virtual straight line VL (image) described above based on the position of the vehicle VH specified by the own vehicle position information I1 and the orientation of the image display device 150 detected by the orientation control unit 142 of the support control unit 140. A virtual straight line that passes through a point on the front FS of the display device 150 and is orthogonal to the front FS) is calculated, and is on the virtual straight line VL (or a position where the distance from the virtual straight line VL is equal to or less than a predetermined threshold value). Determine if there is a tourist attraction located in. In addition, the combination of each position on the road or parking lot and the position and orientation of the image display device 150 and the correspondence relationship with the tourist attraction are set in advance, and the correspondence relationship is stored in the infotainment database ID. The target setting unit 134 may make the determination of S170 with reference to the correspondence.

In the above S170, when it is determined that there is an object (sightseeing spot) that can be introduced in the direction indicated by the image display device 150 (S170: YES), the target setting unit 134 selects one of such tourist spots. It is set as the target object TA, and the target information acquisition unit 136 acquires the specification information I3 of the target object TA from the infotainment database ID (S180). Specification information I3 of the target object TA is an image showing the specifications (appearance, name, size, year of construction, etc.) of the target object TA.

Further, the image display control unit 130 causes the image display device 150 to display an image showing the specification information I3 of the target object TA (S190). FIG. 5 shows a state in which an image showing the specifications (appearance, name, size, etc.) of the second target object TA2 is displayed on the image display device 150 as the specification information I3 of the second target object TA2. It is shown. By displaying the image showing the specification information I3 on the image display device 150, the passenger PA can see the appearance of the second target object TA2 that he / she wants to see by himself / herself without moving his / her line of sight, and the image display device 150. The contents of the specification information I3 of the displayed second target object TA2 can be visually recognized.

In the present embodiment, when the orientation (and position) of the image display device 150 is manually changed, the image display control unit 130 displays the image captured by the camera 152 included in the image display device 150 as an image display device. It is supposed to be displayed on 150. Therefore, the passenger PA can confirm the object located in the direction indicated by the image display device 150 by referring to the image displayed on the image display device 150, and the image display device 150 can accurately identify the desired object. The position and orientation of the image display device 150 can be changed so as to be in the state indicated by.

If it is determined in S170 that there is no object (sightseeing spot) that can be introduced around the vehicle VH (S170: NO), the processes of S180 to S190 are skipped. Alternatively, the passenger PA is notified that there are no objects (sightseeing spots) that can be introduced around the vehicle VH, and the manual change (and position) of the image display device 150 is urged to be redone (S120). It may be a thing.

When the above-mentioned various processes are repeatedly executed and it is determined in S200 that the destination has arrived (S200: YES), the image display control unit 130 ends the image display control process.

A-4. Effect of this embodiment:
As described above, the image display system 10 of the present embodiment supports the image display device 150, the image display device 150, and the support device 160 capable of changing the orientation of the image display device 150, and the image display device. A control device 100 for controlling the 150 and the support device 160 is provided. The control device 100 includes a spatial information acquisition unit 132, an orientation control unit 142, and an image display control unit 130. The space information acquisition unit 132 provides the vehicle position information I1 indicating the position of the vehicle VH (that is, the position of the image display device 150) and the tourist attraction position information I2 indicating the position of an object around the image display device 150. Acquire the spatial information including. The orientation control unit 142 causes the support device 160 to change the orientation of the image display device 150 based on the above spatial information. The image display control unit 130 causes the image display device 150 to display an image according to the orientation of the image display device 150.

As described above, the image display system 10 of the present embodiment includes the own vehicle position information I1 indicating the position of the image display device 150 and the tourist attraction position information I2 indicating the position of an object around the image display device 150. The orientation of the image display device 150 is changed based on the spatial information, and an image corresponding to the orientation of the image display device 150 is displayed on the image display device 150. Therefore, according to the image display system 10 of the present embodiment, the image display device 150 is oriented according to the direction of the image display device 150 while indicating the correct direction to the passenger PA (user) without using the image display device on the mounting side. The image can be displayed on the image display device 150, and the convenience of the image display system 10 can be improved.

Further, in the image display system 10 of the present embodiment, the control device 100 further has a target setting unit 134 that sets one of the surrounding objects as the target object TA based on the above spatial information, and the orientation control unit 142. Causes the support device 160 to change the orientation of the image display device 150 so that the image display device 150 is oriented to point to the target object TA. Therefore, according to the image display system 10 of the present embodiment, the direction of the line of sight of the occupant PA can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150. The convenience of the display system 10 can be further improved.

Further, in the image display system 10 of the present embodiment, the control device 100 further has a target information acquisition unit 136 for acquiring specification information I3 which is information about the target object TA, and the image display control unit 130 includes various An image showing the original information I3 is displayed on the image display device 150. Therefore, according to the image display system 10 of the present embodiment, the direction of the line of sight of the occupant PA can be guided to the target object TA, which is one of the surrounding objects, depending on the orientation of the image display device 150. Since the image showing the specification information I3 regarding the target object TA can be displayed on the image display device 150, the convenience of the image display system 10 can be further improved.

Further, in the image display system 10 of the present embodiment, the specification information I3 of the target object TA includes information indicating at least one of the name, size, and year of construction of the target object TA. Therefore, according to the image display system 10 of the present embodiment, the image display device 150 can display an image showing information indicating at least one of the name, size, and year of construction of the target object TA. It is possible to present useful information to the passenger PA.

Further, in the image display system 10 of the present embodiment, the support device 160 supports the image display device 150 so that the orientation of the image display device 150 can be manually changed, and the target setting unit 134 of the image display control unit 130 When the orientation of the image display device 150 is manually changed, an object located in the direction indicated by the image display device 150 after the orientation is changed is set as the target object TA. That is, in the image display system 10 of the present embodiment, the image display device 150 functions as a 3D pointing device. Therefore, according to the image display system 10 of the present embodiment, when the occupant PA points the image display device 150 in the direction of a desired object, the object is set as the target object TA, so that the occupant PA is intuitive. The target object TA can be set by various operations, and the convenience of the image display system 10 can be further improved.

Further, in the image display system 10 of the present embodiment, the image display device 150 has a camera 152 that captures a subject located in the direction indicated by the image display device 150, and the image display control unit 130 manually displays an image. When the orientation of the device 150 is changed, the image captured by the camera 152 is displayed on the image display device 150. Therefore, according to the image display system 10 of the present embodiment, the current orientation of the image display device 150 can be fed back to the passenger PA by the image displayed on the image display device 150, so that the passenger PA can easily do so. In addition, the orientation of the image display device 150 can be manually changed so that the image display device 150 accurately points to a desired object, and the convenience of the image display system 10 can be further improved.

Further, in the image display system 10 of the present embodiment, the image display device 150 has a flat front surface (image display surface) FS, and the direction pointed to by the image display device 150 is orthogonal to the front surface FS and is front surface. It is a direction toward the opposite side (rear BS side) from the direction in which the FS faces. Therefore, according to the image display system 10 of the present embodiment, the passenger PA can more accurately grasp the direction indicated by the image display device 150, and the convenience of the image display system 10 can be further improved.

Further, in the image display system 10 of the present embodiment, the support device 160 has three or more rotation axes. Therefore, according to the image display system 10 of the present embodiment, the orientation of the image display device 150 can be adjusted more flexibly, so that the convenience of the image display system 10 can be further improved.

Further, the image display system 10 of the present embodiment is mounted on the vehicle VH capable of automatic driving, the support device 160 supports the image display device 150 on the vehicle VH, and the spatial information acquisition unit 132 is installed on the vehicle VH. The spatial information obtained by the sensor 102 is acquired. Therefore, according to the image display system 10 of the present embodiment, by changing the direction of the image display device 150 using the information used for automatic driving, the passenger PA of the vehicle VH capable of automatic driving and the person outside the vehicle On the other hand, while indicating an accurate direction related to the traveling of the vehicle VH, an image corresponding to the direction can be displayed on the image display device 150, and the convenience of the image display system 10 can be improved.

B. Modification example:
The technique disclosed in the present specification is not limited to the above-described embodiment, and can be transformed into various forms without departing from the gist thereof, and for example, the following modifications are also possible.

The configuration of the image display system 10 in the above embodiment is merely an example and can be variously modified. For example, in the above embodiment, the image display device 150 is supported by the support device 160 on the dashboard of the vehicle VH, but as in the modified example shown in FIG. 6, the image display device 150 is supported by the support device 160 on the vehicle VH. It may be supported by the ceiling or floor. Further, in the above embodiment, the support device 160 is configured so that the position and orientation of the image display device 150 can be manually changed, but the position and orientation of the image display device 150 can always be manually changed. It does not have to be configured to be.

Further, in the above embodiment, the support device 160 is composed of a robot arm having a large number of joints, and the position and orientation of the image display device 150 can be changed. However, the support device 160 is an image display device 150. The image display device 150 may be configured by another device (for example, a gimbal) as long as the orientation of the image display device 150 can be changed while supporting the image display device 150. When the support device 160 has three or more axes, for example, the pan, tilt, and roll of the image display device 150 can be adjusted along each axis, and the orientation of the image display device 150 can be adjusted more flexibly. It is preferable because it can be used.

Further, in the above embodiment, in the initial state, the image display device 150 is located in front of the passenger PA, and the front FS of the image display device 150 is oriented so as to face the passenger PA. The position and orientation of the image display device 150 in the initial state is not limited to this. For example, in the initial state, the front FS of the image display device 150 may face upward to form a table shape so that one or a plurality of passenger PAs can easily see the screen. Alternatively, in the initial state, the orientation of the image display device 150 may be such that the touch input operation of the screen by the passenger PA is easy to perform.

Further, in the above embodiment, the case where one image display device 150 is used by one passenger PA (user) is illustrated, but the technique disclosed in the present specification is that one image display device 150 is used. The same applies to the case where it is shared by a plurality of users. That is, since the orientation of one image display device 150 can be recognized almost accurately by each of the plurality of users, the orientation of the image display device indicates the orientation to the plurality of users and corresponds to the direction. The image can be displayed on the image display device.

Further, in the above embodiment, the vehicle-mounted image display device is used as the image display device 150, but the image display device 150 is another image display device such as a smartphone or tablet terminal brought by the passenger PA. May be used. In such a case, for example, the support device 160 is provided with a holder for holding the image display device, and by having the holder hold the image display device, the support and the position and orientation of the image display device can be changed. Will be done. Further, as the image display device 150, a projection type image display device such as a projector may be used.

Further, in the above embodiment, the control device 100 may be configured to be divided into a plurality of devices. For example, the vehicle VH may be provided with a control device for automatic driving control, a control device for image display control, and a control device for support control, which are independent of each other. Further, in each of the above embodiments, a part of the configuration realized by the hardware may be replaced with software, and conversely, a part of the configuration realized by the software may be replaced with the hardware. May be good.

Further, the content of the image display control process in the above embodiment is just an example and can be variously modified. For example, in the image display control process according to the above embodiment, the spatial information acquisition unit 132 acquires information on the position and orientation of the vehicle VH detected based on signals from various sensors 102 installed on the vehicle VH as spatial information. However, the spatial information acquisition unit 132 detects the position and orientation of the image display device 150 based on a signal from a sensor other than the sensor 102 installed in the vehicle VH (for example, a sensor installed in the image display device 150). Information may be acquired as spatial information. The spatial information may not include information indicating the orientation of the vehicle VH or the image display device 150.

Further, the image display control process in the present embodiment is a process of presenting information introducing tourist attractions seen from the vehicle window as infotainment to the passenger PA of the vehicle VH traveling toward the destination by automatic driving. If so, the techniques disclosed herein are also applicable to other image display control processes. For example, the techniques disclosed herein are also applicable to image display control processing for providing other infotainment (games, moving images, etc.). Further, in the technique disclosed in the present specification, for example, the spatial information acquisition unit 132 may acquire spatial information, and the orientation control unit 142 may direct the front FS of the image display device 150 to a peripheral object. Is changed in the direction of the image display device 150, and the image display control unit 130 displays an image to be shown to surrounding objects (for example, an image that calls attention to the progress of the vehicle VH) on the image display device 150. It can also be applied to an image display control process that causes the image to be displayed.

Further, in the above embodiment, an example of the image display system 10 installed in the vehicle VH capable of automatically driving has been described, but the technique disclosed in the present specification is an image display system used for other purposes (for example, for example. Image display system installed on vehicles that are not self-driving, image display system installed on moving objects other than vehicles, image display system installed on fixed objects (for example, buildings), portable image display The same applies to systems, etc.).

10: Image display system 100: Control device 102: Sensor 110: Control unit 120: Automatic operation control unit 122: Position / orientation detection unit 124: Peripheral situation detection unit 126: Route setting unit 130: Image display control unit 132: Spatial information Acquisition unit 134: Target setting unit 136: Target information acquisition unit 140: Support control unit 142: Direction control unit 150: Image display device 152: Camera 160: Support device 162: Drive motor 170: Storage unit 172: Automatic operation control program 174 : Image display control program 176: Support control program 190: Bus BS: Rear FS: Front I1: Own vehicle position information I2: Tourist attraction location information I3: Specification information ID: Infotainment database MD: 3D map database PA: Passenger TA: Target object VH: Vehicle VL: Virtual straight line

Claims

It is an image display system
Image display device and
A support device that supports the image display device and can change the orientation of the image display device,
A control device that controls the image display device and the support device,
With
The control device is
A spatial information acquisition unit that acquires spatial information including a first position information indicating the position of the image display device and a second position information indicating the position of an object around the image display device.
An orientation control unit that causes the support device to change the orientation of the image display device based on the spatial information.
An image display control unit that displays an image according to the orientation of the image display device on the image display device,
Have,
Image display system.
The image display system according to claim 1.
The control device further has a target setting unit that sets one of the peripheral objects as a target object based on the spatial information.
The orientation control unit causes the support device to change the orientation of the image display device so that the image display device is oriented to point to the target object.
Image display system.
The image display system according to claim 2.
The control device further has a target information acquisition unit that acquires target information that is information about the target object.
The image display control unit causes the image display device to display an image indicating the target information.
Image display system.
The image display system according to claim 3.
The target information includes information indicating at least one of the name, size, and year of construction of the target object.
Image display system.
The image display system according to any one of claims 2 to 4.
The support device supports the image display device so that the orientation of the image display device can be manually changed.
When the orientation of the image display device is manually changed, the target setting unit sets an object located in the direction indicated by the image display device as the target object.
Image display system.
The image display system according to claim 5.
The image display device has a camera that captures a subject located in the direction indicated by the image display device.
The image display control unit causes the image display device to display an image captured by the camera when the orientation of the image display device is manually changed.
Image display system.
The image display system according to any one of claims 2 to 6.
The image display device has a flat image display surface and has a flat image display surface.
The direction pointed to by the image display device is a direction orthogonal to the image display surface and toward the side opposite to the image display surface.
Image display system.
The image display system according to any one of claims 1 to 7.
The support device has three or more axes of rotation.
Image display system.
The image display system according to any one of claims 1 to 8.
The image display system is mounted on a vehicle capable of autonomous driving.
The support device supports the image display device on the vehicle.
The spatial information acquisition unit acquires the spatial information obtained by the sensor installed in the vehicle.
Image display system.
An image display method for displaying an image using an image display device and a support device that supports the image display device and can change the orientation of the image display device.
A step of acquiring spatial information including a first position information indicating the position of the image display device and a second position information indicating the position of an object around the image display device.
A step of causing the support device to change the orientation of the image display device based on the spatial information, and
A process of displaying an image according to the orientation of the image display device on the image display device, and
To prepare
Image display method.
A computer program for displaying an image using an image display device and a support device that supports the image display device and can change the orientation of the image display device.
On the computer
A process of acquiring spatial information including a first position information indicating the position of the image display device and a second position information indicating the position of an object around the image display device.
A process of causing the support device to change the orientation of the image display device based on the spatial information.
A process of displaying an image according to the orientation of the image display device on the image display device, and
To execute,
Computer program.