WO2017018844A1

WO2017018844A1 - Autonomous vehicle and operation method of same

Info

Publication number: WO2017018844A1
Application number: PCT/KR2016/008328
Authority: WO
Inventors: 크로닌존; 멜빈 크로닌세쓰
Original assignee: 삼성전자 주식회사
Priority date: 2015-07-30
Filing date: 2016-07-29
Publication date: 2017-02-02

Abstract

The present invention provides an autonomous vehicle and an operation method of same, the autonomous vehicle enabling displaying of a virtual driving environment image, which substitutes the actual driving environment of the autonomous vehicle, by means of a display device formed on an area of a window of the autonomous vehicle, thereby providing a passenger with a more realistic experience with respect to the virtual driving environment.

Description

[Correction by rule 26.08.2016] Autonomous vehicle and its operation method

The present disclosure relates to an autonomous vehicle and a method of operation thereof.

Recently, interest in autonomous vehicles is increasing. In particular, various additional functions related to autonomous driving have been continuously developed to alleviate traffic congestion caused by increasing demand for automobiles and to safely avoid obstacles such as people and other vehicles. For example, there are many algorithms associated with lane keeping systems.

In addition, as internet connectivity expands, the amount of data generated from various devices and automobiles is rapidly increasing, and various services using the same are emerging.

Therefore, there is a demand for a method and an autonomous vehicle that can provide a familiar autonomous driving experience to passengers using various data.

An autonomous vehicle and its operation method are provided. The technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and further technical problems can be inferred from the following embodiments.

According to an aspect, an autonomous vehicle includes a display device formed in an area of a car window of an autonomous vehicle; And a processor that controls the display device to display a virtual driving environment image that replaces the actual driving environment of the autonomous vehicle.

According to another aspect, an operating method of an autonomous vehicle includes: obtaining a virtual driving environment image in place of an actual driving environment of the autonomous vehicle; And controlling a display device formed in an area of a car window of an autonomous vehicle to display a virtual driving environment image.

According to another aspect, there is provided a computer readable recording medium having recorded thereon a program for implementing the method.

According to another aspect, an autonomous vehicle may include an input device configured to receive a virtual driving environment from a user; And a windshield displaying the selected virtual driving environment.

According to the present embodiment, the virtual driving environment image, which replaces the actual driving environment, is provided to the occupant through a display device formed in the area of the window of the autonomous vehicle, and thus the passenger can experience the virtual driving environment more realistically. Can provide.

1 is a diagram illustrating an autonomous vehicle according to an embodiment.

2 is a block diagram illustrating detailed hardware configurations of an autonomous vehicle according to an embodiment.

3 is a block diagram of an autonomous vehicle according to an embodiment.

4 is a diagram illustrating a windshield of an autonomous vehicle according to an exemplary embodiment.

5 is a diagram for describing a display apparatus, according to an exemplary embodiment.

6 is a diagram for describing a display apparatus, according to another exemplary embodiment.

7 is a diagram illustrating a UI for determining a driving route, according to an exemplary embodiment.

8 is a diagram illustrating a UI for setting a virtual reality, according to an exemplary embodiment.

9 is a diagram for describing generating a virtual driving environment image.

FIG. 10 is a diagram illustrating an embodiment of generating virtual driving environment images corresponding to points at which an autonomous vehicle travels straight.

FIG. 11 is a diagram illustrating an embodiment of generating virtual driving environment images corresponding to points at which an autonomous vehicle travels in a right turn.

12 and 13 illustrate an embodiment of generating a plurality of virtual driving environment images corresponding to a point on a driving route.

14 is a diagram illustrating an embodiment of a camera of an autonomous vehicle.

15 is a diagram illustrating an embodiment in which a processor generates a virtual driving environment image based on an image of an actual driving environment.

FIG. 16 is a diagram illustrating a UI for selecting an area of a window to display a virtual driving environment according to an exemplary embodiment.

FIG. 17 is a diagram illustrating an embodiment in which a virtual driving environment image is displayed on an area of a car window corresponding to a line of sight of a passenger.

18 is a diagram illustrating a UI for selecting content to display on a display apparatus, according to an exemplary embodiment.

19 is a diagram illustrating an embodiment of displaying a movie on a display device.

20 is a diagram illustrating a UI for setting an event, according to an exemplary embodiment.

FIG. 21 is a diagram for one embodiment of providing a passenger with information on an event when a predetermined event occurs.

FIG. 22 is a diagram for one embodiment of providing a passenger with information about an event when a predetermined event occurs.

FIG. 23 is a diagram for one embodiment of providing a passenger with information about an event when a predetermined event occurs.

24 is a flowchart of a method of operating an autonomous vehicle, according to an exemplary embodiment.

FIG. 25 is a detailed flowchart embodying step 2420 of FIG. 24.

26 is a detailed flowchart of a method of operating an autonomous vehicle according to an embodiment.

The virtual driving environment image may be an image showing a virtual driving environment outside of the autonomous vehicle viewed from the viewpoint of the inside of the autonomous vehicle toward the area of the vehicle window.

In addition, the processor may obtain information about a driving route from a current position of the autonomous vehicle to a destination, and generate virtual driving environment images corresponding to each of the points on the driving route.

The apparatus may further include a motion sensing device that senses a movement of the autonomous vehicle, and the processor may control the display device to display the virtual driving environment images based on the sensed movement.

In addition, the motion sensing device senses the traveling speed of the autonomous vehicle, and the processor may control the rate of change of the image between the virtual driving environment images displayed on the display device based on the sensed driving speed.

In addition, when there are a plurality of display apparatuses, the processor may control an image change rate between virtual driving environment images displayed on each of the plurality of display apparatuses based on the sensed movement.

The apparatus may further include an image sensor configured to capture an image of an actual driving environment, and the processor may generate a virtual driving environment image based on the captured image of the actual driving environment.

In addition, the processor may generate a virtual driving environment image reflecting the appearance of the object shown in the image of the actual driving environment.

In addition, the processor may generate a virtual driving environment image based on a virtual reality selected from a passenger of the autonomous vehicle among a plurality of virtual realitys.

In addition, the processor may determine whether a predetermined event occurs, and when an event occurs, the processor of the autonomous vehicle may control the actual driving environment corresponding to the event through the display device.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. The following examples are only for the purpose of embodying the spirit of the present invention, and of course, it does not limit or limit the scope of rights. From the detailed description and examples, what can be easily inferred by those skilled in the art is interpreted as belonging to the scope of rights.

As used herein, the term “consisting of” or “comprising” should not be construed as including all of the various elements, or steps, described in the specification, and some or some of them may be included. Should not be included, or should be construed to further include additional components or steps. In addition, the terms "... unit", "module", etc. described in the specification mean a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software. .

In addition, terms including ordinal numbers such as “first” or “second” as used herein may be used to describe various components, but these terms may distinguish one component from another or It can be used for convenience.

Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an autonomous vehicle according to an embodiment.

The autonomous vehicle 1 may refer to a vehicle that can drive itself without intervention of a passenger.

The autonomous vehicle 1 may display a virtual driving environment image that replaces the actual driving environment of the autonomous vehicle 1. In other words, the autonomous vehicle 1 may display an image representing a virtual driving environment different from the actual driving environment around the autonomous vehicle 1. For example, if the autonomous vehicle 1 is traveling in the city center, there may be many buildings in the actual driving environment around the autonomous vehicle 1, but the autonomous vehicle 1 represents a forest. The virtual driving environment image may be displayed. Therefore, the passenger of the autonomous vehicle 1 may experience that the autonomous vehicle 1 is driving in the forest, not in the city center, through the virtual driving environment image displayed by the autonomous vehicle 1.

The autonomous vehicle 1 may display a virtual driving environment image through a display device formed in an area of a car window of the autonomous vehicle 1. Therefore, when the occupant looks at the area of the window of the autonomous vehicle 1, the occupant may view the virtual driving environment image displayed on the display device formed in the area of the window, so that the actual driving around the autonomous vehicle 1 is performed. Experience a virtual driving environment rather than an environment.

In addition, the autonomous vehicle 1 may display a virtual driving environment image through a display device in conjunction with the movement of the autonomous vehicle 1, and thus, the autonomous vehicle 1 may be virtually realized. Experience the driving environment.

The autonomous vehicle 1 includes a propulsion device 210, a power supply device 299, a communication device 250, an input device 260, an output device 280, a storage device 270, and a travel device 220. , The sensing device 230, the peripheral device 240, and the control device 290. However, the autonomous vehicle 1 may further include other general-purpose components in addition to the components illustrated in FIG. 2, or the autonomous vehicle 1 may not include some of the components illustrated in FIG. 2. It may be understood by those of ordinary skill in the art related to the present embodiment.

The propulsion device 210 may include an engine / motor 211, an energy source 212, a transmission 213 and a wheel / tire 214.

The engine / motor 211 may be any combination between an internal combustion engine, an electric motor, a steam engine, and a stirling engine. For example, when the autonomous vehicle 1 is a gas-electric hybrid car, the engine / motor 211 may be a gasoline engine and an electric motor.

Energy source 212 may be a source of energy that powers the engine / motor 211 in whole or in part. That is, engine / motor 211 may be configured to convert energy source 212 into mechanical energy. Examples of energy sources 212 may be at least one of gasoline, diesel, propane, other compressed gas based fuels, ethanol, solar panels, batteries, and other electrical power sources. Alternatively, the energy source 212 may be at least one of a fuel tank, a battery, a capacitor, and a flywheel. The energy source 212 can provide energy to the systems and devices of the autonomous vehicle 1.

Transmission 213 may be configured to transfer mechanical power from engine / motor 211 to wheel / tire 214. For example, the transmission 213 may include at least one of a gearbox, a clutch, a differential, and a drive shaft. When the transmission 213 includes drive shafts, the drive shafts may include one or more axles configured to be coupled to the wheel / tire 214.

The wheel / tire 214 may be configured in a variety of formats, including a unicycle, a bike / motorcycle, a tricycle, or a four wheel type of a car / truck. For example, other wheel / tire types may be possible, such as including six or more wheels. The wheel / tire 214 includes at least one wheel fixedly attached to the transmission 213 and at least one tire coupled to a rim of the wheel that can contact the driving surface. can do.

The traveling device 220 may include a brake unit 221, a steering unit 222, and a throttle 223.

The steering unit 222 may be a combination of mechanisms configured to adjust the direction of the autonomous vehicle 1.

The throttle 223 may be a combination of mechanisms configured to control the speed of operation of the engine / motor 211 to control the speed of the autonomous vehicle 1. In addition, the throttle 223 may adjust the amount of throttle opening to adjust the amount of mixed gas of fuel air flowing into the engine / motor 211, and may control power and thrust by adjusting the throttle opening.

The brake unit 221 may be a combination of mechanisms configured to decelerate the autonomous vehicle 1. For example, the brake unit 221 may use friction to reduce the speed of the wheel / tire 214.

The sensing device 230 may include a plurality of sensors configured to sense information about the environment in which the autonomous vehicle 1 is located, as well as one or more actuators configured to modify the position and / or orientation of the sensors. Can include them. For example, the sensing device 230 includes a Global Positioning System (GPS) 224, an Inertial Measurement Unit (IMU) 225, a RADAR unit 226, a LIDAR unit 227, and an image sensor 228. can do. In addition, the sensing device 230 may include at least one of a temperature / humidity sensor 232, an infrared sensor 233, a barometric pressure sensor 235, a proximity sensor 236, and an RGB sensor illuminance sensor 237. It may be, but is not limited thereto. Since functions of the respective sensors can be intuitively deduced by those skilled in the art from the names, detailed descriptions thereof will be omitted.

In addition, the sensing device 230 may include a motion sensing device 238 capable of sensing the movement of the autonomous vehicle 1. The motion sensing device 238 may include a geomagnetic sensor 229, an acceleration sensor 231, and a gyroscope sensor 234.

The GPS 224 may be a sensor configured to estimate the geographic location of the autonomous vehicle 1. That is, the GPS 224 may include a transceiver configured to estimate the position of the autonomous vehicle 1 with respect to the earth.

The IMU 225 may be a combination of sensors configured to detect positional and orientation changes of the autonomous vehicle 1 based on inertial acceleration. For example, the combination of sensors may include accelerometers and gyroscopes.

The RADAR unit 226 may be a sensor configured to detect objects in the environment in which the autonomous vehicle 1 is located using a wireless signal. In addition, the RADAR unit 226 can be configured to sense the speed and / or direction of the objects.

The LIDAR unit 227 may be a sensor configured to detect objects in the environment in which the autonomous vehicle 1 is located using a laser. More specifically, LIDAR unit 227 may include a laser light source and / or laser scanner configured to emit a laser, and a detector configured to detect reflection of the laser. The LIDAR unit 227 may be configured to operate in coherent (eg, using hetirodyne detection) or noncoherent detection mode.

The image sensor 228 may be a still camera or a video camera configured to record three-dimensional images of the interior of the autonomous vehicle 1. For example, the image sensor 228 may include a number of cameras, which may be disposed at a number of locations on the inside and outside of the autonomous vehicle 1.

The peripheral device 240 may include a navigation 241, a light 242, a turn signal 243, a wiper 244, an interior light 245, a heater 246, and an air conditioner 247.

The navigation 241 may be a system configured to determine a travel route for the autonomous vehicle 1. The navigation 241 may be configured to dynamically update the travel route while the autonomous vehicle 1 is traveling. For example, the navigation 241 may use data from the GPS 224 and maps to determine the route of travel for the autonomous vehicle 1.

The storage device 270 may include a magnetic disk drive, an optical disk drive, and a flash memory. Alternatively, the storage device 270 may be a portable USB data storage device. Storage device 270 may store system software for executing examples related to the present disclosure. System software for carrying out the examples relating to the present disclosure may be stored on a portable storage medium.

The communication device 250 may include at least one antenna for wirelessly communicating with another device. For example, communication device 250 may be used to communicate with a cellular network or other wireless protocols and systems wirelessly via Wi-Fi or Bluetooth. The communication device 250 controlled by the control device 290 may transmit and receive a wireless signal. For example, the control device 290 may execute a program included in the storage device 270 in order for the communication device 250 to transmit and receive a wireless signal with the cellular network.

The input device 260 means a means for inputting data for controlling the autonomous vehicle 1. For example, the input device 260 may include a key pad, a dome switch, a touch pad (contact capacitive type, pressure resistive type, infrared sensing type, surface ultrasonic conduction type, and integral type). Tension measurement method, piezo effect method, etc.), a jog wheel, a jog switch, and the like, but are not limited thereto. In addition, the input device 260 may include a microphone, which may be configured to receive audio (eg, voice commands) from the occupant of the autonomous vehicle 1.

The output device 280 may output an audio signal or a video signal, and the output device 280 may include a display 281 and a sound output unit 282.

The display unit 281 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. 3D display, an electrophoretic display. In addition, depending on the implementation form of the output device 280, the output device 280 may include two or more display units 281.

The sound output unit 282 outputs audio data received from the communication device 250 or stored in the storage device 270. In addition, the sound output unit 282 may include a speaker, a buzzer, and the like.

The input device 260 and the output device 280 may include a network interface, and may be implemented as a touch screen.

The control device 290 typically controls the overall operation of the autonomous vehicle 1. For example, the control device 290 executes the programs stored in the storage device 270, such that the propulsion device 210, the traveling device 220, the sensing device 230, the peripheral device 240, and the communication device ( 250, the input device 260, the storage device 270, the output device 280, and the power supply 299 may be controlled overall. In addition, the control device 290 may control the movement of the autonomous vehicle 1.

The power supply 299 may be configured to provide power to some or all of the components of the autonomous vehicle 1. For example, power supply 299 may comprise a rechargeable lithium ion or lead-acid battery.

3 is a block diagram of an autonomous vehicle according to an embodiment.

The autonomous vehicle 1 may include a display device 110 and a processor 120. In the autonomous vehicle 1 shown in FIG. 3, only components related to the present embodiment are shown. Therefore, it will be understood by those skilled in the art that other general purpose components other than the components shown in FIG. 3 may be further included.

The display device 110 may include the display 281 of FIG. 2, and the processor 120 may correspond to the control device 290 of FIG. 2.

The display apparatus 110 may be formed in an area of a vehicle window of the autonomous vehicle 1.

The window of the autonomous vehicle 1 is a window 401 corresponding to the front surface of the autonomous vehicle 1, a car window 402 corresponding to the right side of the autonomous vehicle 1, and a left side of the autonomous vehicle 1. The window 403 corresponding to the surface, the window 404 corresponding to the rear of the autonomous vehicle 1, and the window 405 corresponding to the ceiling of the autonomous vehicle 1 may be used. Accordingly, the autonomous vehicle 1 may include a display device formed in at least one area of the

vehicle windows

401, 402, 403, 404, 405.

In FIG. 4, according to an example, the autonomous vehicle 1 includes windows corresponding to five regions, but the present invention is not limited thereto, and the autonomous vehicle 1 may have different positions, different sizes, and different positions from those of FIG. 4. The windows may be provided in a form or the like.

Referring back to FIG. 3, according to an example, the display apparatus 110 may be a transparent display formed in an area of a vehicle window. According to another example, the display device 110 may be a transparent display to replace the windshield. That is, the display device 110 may be a transparent display having a function for a display and a function as a window. For example, the display device 110 may be configured as a transparent electrode. When a voltage is applied to the display device 110, the display device 110 may have a function for display, and the display device 110 may be used. If no voltage is applied), it may have a function as a windshield. According to another example, the display device 110 may have a size of an area of the window and may be formed on the surface of the window. According to another example, the display device 110 may be slidably coupled to the window.

The display apparatus 110 may be a transparent display formed in an area of the windshield 501 of the autonomous vehicle 1. That is, the display device 110 may be the transparent display 502 closely bonded to one surface of the window 501. According to an example, the display apparatus 110 may be a flexible thin film type and may be configured as an element that may transmit light and output an image of high brightness. Such a device may be any one of an LCD, an LED, and a transparent organic light emission diode (TOLED).

In FIG. 5, the front window of the autonomous vehicle 1 is illustrated according to an example. However, the display device 110 as a transparent display may be formed in an area of another vehicle window of the autonomous vehicle 1.

The display apparatus 110 may have a size of the window 601 of the autonomous vehicle 1 and may be slidably coupled to the window 601. That is, the display device 110 may slide in one direction and overlap the area of the window 601 entirely, and may not exist in the other direction by sliding in the other direction and overlapping the area of the window 601.

6 illustrates the front window of the autonomous vehicle 1 according to an example, the slidable display apparatus 110 may be formed in an area of another vehicle window of the autonomous vehicle 1.

Referring back to FIG. 3, the processor 120 may generate a virtual driving environment image that replaces the actual driving environment of the autonomous vehicle 1. The virtual driving environment image refers to an image showing a virtual driving environment outside the autonomous vehicle 1 as viewed toward an area of a vehicle window from an internal viewpoint of the autonomous vehicle 1. In other words, the virtual driving environment image refers to an image representing the virtual driving environment outside the autonomous vehicle 1 that the occupant of the autonomous vehicle 1 may look toward the area of the vehicle window. The virtual driving environment may be a driving environment in virtual reality in which the actual driving environment is partially reflected. For example, a real driving environment may be a road in a rainy city, but a virtual driving environment may be a road in a city with bright sunlight. Therefore, the virtual driving environment image may represent a virtual driving environment that the occupant can recognize as the actual driving environment when the passenger looks out of the autonomous vehicle 1 toward the area of the vehicle window.

The processor 120 may generate a virtual driving environment image based on the information about the actual driving environment and the information about the virtual reality around the autonomous driving vehicle 1. The information on the actual driving environment may include information on a driving route for the autonomous vehicle 1 to travel to a destination, and may include an image of the actual driving environment. In addition, the processor 120 may obtain information about the virtual reality from the storage device 270 of FIG. 2, or may obtain information about the virtual reality from an external network. In addition, the virtual reality may be determined by a passenger's selection of the plurality of virtual realitys.

The processor 120 may generate a virtual driving environment image based on the information on the driving route from the current position of the autonomous vehicle 1 to the destination. More specifically, the processor 120 may acquire information about a driving route from the current position of the autonomous driving vehicle 1 to the destination, and reflect the driving route acquired in the preset virtual reality, and thus the virtual driving environment image. Can be generated. For example, the processor 120 may generate a virtual driving environment image by reflecting a state of a road corresponding to a driving route in a virtual reality representing a coast. According to an example, the processor 120 may obtain information about a destination from the occupant, and determine a driving route from the current position of the autonomous vehicle 1 to the destination. According to another example, the navigation 241 of FIG. 2 may determine the driving route from the current position of the autonomous vehicle 1 to the destination, and the processor 120 obtains information about the driving route from the navigation 241. can do.

The processor 120 may generate a virtual driving environment image corresponding to a point on the driving route. In other words, based on the point on the travel path where the autonomous vehicle 1 may be located, the processor 120 may display the autonomous vehicle 1 that the occupant of the autonomous vehicle 1 may look toward the area of the vehicle window. An image representing an external virtual driving environment may be generated. Similarly, the processor 120 may generate virtual driving environment images corresponding to each of the points based on the points on the driving route of the autonomous vehicle 1.

The processor 120 may provide a passenger with a UI 710 for determining a driving route. According to an example, the processor 120 may display the UI 710 on the display device 110 or on a separate display.

The occupant may input information on a desired destination through the input device 260 in the area 701 for inputting destination information in the UI 710. Accordingly, the occupant may enter a destination '1600 Pennsylvania Ave, D.C' in the area 701. The occupant may then select a travel route to the destination via the additional setting area 702. In other words, as shown in FIG. 7, the occupant may select a driving route via the freeway among several driving routes to the destination. Therefore, the processor 120 may determine the driving route via the highway selected by the occupant as the driving route to the destination of the autonomous vehicle 1.

The processor 120 may provide a passenger with a UI 810 for setting up virtual reality. The occupant may select one of the plurality of virtual reality through the UI 810. In other words, the occupants can access the UI 810 via the Rocky Mountains, Amazon Rainforest, Saharan Safari, Grand Canyon, Hawaii Volcanoes, and Big Sur. Virtual reality corresponding to any one of Big Sur, California) and a rolling irish hill. In addition, the passenger may download another virtual reality from the external network by selecting the download menu 801.

According to one example, the processor 120 may first provide the occupant with the UI 710 of FIG. 7 to determine the driving route to the destination of the autonomous vehicle 1, and then the UI 810 of FIG. 8. May be provided to the occupant to determine the virtual reality. Therefore, the processor 120 may generate a virtual driving environment image by using the determined driving route and the virtual reality.

9 is a diagram for describing generating a virtual driving environment image.

The processor 120 generates a virtual driving environment image 930 corresponding to the partial section 910 based on the partial section 910 of the driving path of the autonomous vehicle 1 and the partial region 920 of the virtual reality. can do. That is, the processor 120 may generate a virtual driving environment image 930, which is a virtual driving environment that can be viewed by the occupant at the point 915 based on the point 915 where the autonomous vehicle 1 will travel in the future. Can be. More specifically, the processor 120 may recognize the road appearance of the section 910 based on the point 915, and reflect the recognized road appearance in the partial region 920 of the virtual reality, thereby providing a virtual driving environment. An image 930 may be generated. In other words, since the road shape of the section 910 is a section that turns left after going straight a certain distance, the processor 120 reflects the road form that turns left after going a certain distance in the partial region 920 of the virtual reality, the virtual driving environment An image 930 may be generated. Similarly, the processor 120 may recognize a road view for each of the remaining sections of the driving route of the autonomous vehicle 1, and reflect the road view recognized for each section to other regions of the virtual reality, thereby autonomously driving the vehicle 1. A plurality of virtual driving environment images corresponding to the entire driving route may be generated.

The processor 120 may generate virtual

driving environment images

1020 and 1030 based on the

points

1010 and 1015 on the driving route. When the driving path of the autonomous vehicle 1 passes straight through the point 1010 and the point 1015, the processor 120 determines that the autonomous vehicle 1 is located at the point 1010 based on the fact that the autonomous vehicle 1 is located at the point 1010. The driving environment image 1020 may be generated, and then the virtual driving environment image 1030 may be generated based on the position of the autonomous vehicle 1 at the point 1015.

The virtual driving environment image 1020 may represent a virtual driving environment outside the autonomous driving vehicle 1 that the occupant may look toward the area of the vehicle window when the autonomous driving vehicle 1 is located at the point 1010. The virtual driving environment image 1030 may represent a virtual driving environment outside the autonomous driving vehicle 1 that the occupant may look toward the area of the vehicle window when the autonomous driving vehicle 1 is located at the point 1015. have. Accordingly, some objects 1026 of the virtual driving environment of the virtual driving environment image 1020 may disappear from the virtual driving environment image 1030, and some objects 1022, of the virtual driving environment of the virtual driving environment image 1020. The size and shape of the image 1024 may be expressed in a virtual driving environment image 1030 by being changed to look closer, such as some

objects

1032 and 1034. Accordingly, the processor 120 continuously provides virtual

passenger environment images

1020 and 1030 to the occupant so that the autonomous vehicle 1 travels to the

occupant

1010 and 1015 in a straight line. Can be provided. In addition, the actual driving environment is a road in the city, but since the virtual driving environment shown in the virtual

driving environment images

1020 and 1030 is a seaside road, the processor 120 continuously occupies the virtual

driving environment images

1020 and 1030. By providing to the passenger, it is possible to provide the occupant the experience that the autonomous vehicle 1 travels straight along the waterfront road.

10 illustrates an example in which the processor 120 generates a virtual driving environment image 1020 and a virtual driving environment image 1030 corresponding to a point 1010 and a point 1015 on a driving route, but to a passenger. In order to provide a more realistic driving experience, the processor 120 may generate virtual driving environment images corresponding to many points on the driving route.

The processor 120 may generate virtual

driving environment images

1120 and 1130 based on the

points

1110 and 1115 on the driving route. When the driving path of the autonomous vehicle 1 passes through the

points

1110 and 1115 in a right turn, the processor 120 may determine that the autonomous vehicle 1 is located at the point 1110. The driving environment image 1120 may be generated, and then the virtual driving environment image 1130 may be generated based on the position of the autonomous vehicle 1 at the point 1115.

The virtual driving environment image 1120 may represent a virtual driving environment outside the autonomous driving vehicle 1 that the occupant may look toward the area of the vehicle window when the autonomous driving vehicle 1 is located at the point 1110. The virtual driving environment image 1130 may represent a virtual driving environment outside the autonomous driving vehicle 1 that the occupant may look toward the area of the vehicle window when the autonomous driving vehicle 1 is located at the point 1115. have. Accordingly, the processor 120 continuously provides the virtual

driving environment images

1120 and 1130 to the occupant so that the autonomous vehicle 1 travels to the

occupant

1110 and 1115 on the driving route to the right. Can be provided. In addition, although the actual driving environment is a road in the city, the virtual driving environment shown in the virtual

driving environment images

1020 and 1030 is a road surrounded by trees, so that the processor 120 continuously processes the virtual

driving environment images

1120 and 1130. By providing it to the occupant, it is possible to provide the occupant the experience that the autonomous vehicle 1 travels on the road surrounded by trees to the right.

First, referring to FIG. 12, the processor 120 may generate a plurality of virtual

driving environment images

1210, 1220, and 1230 corresponding to a point 1205 on a driving route. More specifically, the processor 120 is a virtual driving environment outside the autonomous vehicle 1 that the occupant can look toward the area of the front windshield when the autonomous vehicle 1 is located at the point 1205. A driving environment image 1210 may be generated, and a virtual driving environment image 1220 may be generated, which is a virtual driving environment outside the autonomous vehicle 1 which the occupant may look toward the area of the left side window pane. A virtual driving environment image 1230, which is a virtual driving environment outside the autonomous vehicle 1, which the occupant can look toward the area of the right side vehicle window, may be generated. Therefore, when the plurality of virtual

driving environment images

1210, 1220, 1230 are simultaneously displayed on the display device 110 formed in the area of the front window, the area of the left side window, and the area of the right side window, the passenger feels more. You can experience a virtual driving environment.

Subsequently, referring to FIG. 13, the processor 120 may generate a plurality of virtual

driving environment images

1310, 1320, and 1330 corresponding to the point 1305 on the driving route. More specifically, the processor 120 is a virtual driving environment outside the autonomous vehicle 1 that the occupant can view toward the area of the front window when the autonomous vehicle 1 is located at the point 1305. The driving environment image 1310 may be generated, and a virtual driving environment image 1320 may be generated, which is a virtual driving environment outside the autonomous driving vehicle 1 that the occupant may look toward the area of the left side window pane. A virtual driving environment image 1330, which is a virtual driving environment outside the autonomous vehicle 1, which the occupant may look toward the area of the right side vehicle window, may be generated.

Accordingly, in the display apparatus 110 formed in the area of the front window, the area of the left side window, and the area of the right side window, a plurality of virtual

driving environment images

1210, 1220, 1230 corresponding to the point 1205, and As the plurality of virtual

driving environment images

1310, 1320, and 1330 corresponding to the point 1205 are continuously displayed, the occupant may realize the autonomous vehicle 1 as a result of the

points

1205 and 1305 on the driving route. You can experience driving straight ahead. In addition, although the actual driving environment is a rainy road, the virtual driving environment shown in the plurality of virtual

driving environment images

1210, 1220, 1230, 1310, 1320, and 1330 is a road with sunlight, and thus, the processor 120 includes a plurality of roads. By providing the passenger with the virtual

driving environment images

1210, 1220, 1230, and the plurality of virtual

driving environment images

1310, 1320, 1330 corresponding to the point 1205, the autonomous driving vehicle 1 is provided to the passenger. ) Can provide the experience of driving straight on a sunny road.

12 and 13 illustrate the front windshield, the left windshield and the right windshield as an example, the processor 120 may provide an external virtual driving environment that the occupant can see through other windows within the autonomous vehicle 1. A virtual driving environment image may be generated.

Referring back to FIG. 3, the processor 120 may generate a virtual driving environment image based on an image of the actual driving environment of the autonomous vehicle 1. More specifically, the processor 120 may generate a virtual driving environment image reflecting the appearance of an object appearing in the image of the actual driving environment. For example, the processor 120 may generate a virtual driving environment image reflecting the appearance of the road shown in the image of the actual driving environment. In addition, the processor 120 may generate a virtual driving environment image in which the movement trajectory or the change rate of the object included in the image of the actual driving environment is reflected. For example, the processor 120 may generate a virtual driving environment image that reflects the movement trajectory and the speed of the vehicle that appear in the image of the actual driving environment.

According to an example, the image sensor 228 of FIG. 2 may capture an image of the actual driving environment of the autonomous vehicle 1, and the processor 120 may capture an image of the actual driving environment captured by the image sensor 228. Based on the image, the virtual driving environment image may be generated. According to another example, the processor 120 may obtain an image of the actual driving environment of the autonomous vehicle 1 from an external network.

Referring to FIG. 14, as an example of the image sensor 228, the

cameras

1410, 1420, 1430, and 1440 may be installed on the outer surfaces of the

window windows

401, 402, 403, 404 of the autonomous vehicle 1. That is, the

cameras

1410, 1420, 1430, and 1440 may include a car window 401 corresponding to the front side of the autonomous vehicle 1, a car window 403 corresponding to the left side, a car window 402 corresponding to the right side, and a rear side. It may be installed on the outer surface of each of the corresponding window 404.

Accordingly, the

cameras

1410, 1420, 1430, and 1440 may photograph an actual driving environment of the autonomous vehicle 1, which the occupant can look toward the area of the vehicle window, and acquire an image of the actual driving environment. .

The image sensor 228 may be installed in the front windshield of the autonomous vehicle 1, and the image sensor 228 may photograph an actual driving environment that a passenger can see through the front windshield of the autonomous vehicle 1. . The processor 120 may acquire an actual driving environment image 1510 captured by the image sensor 228. In addition, the processor 120 may acquire the virtual reality 1520 selected by the occupant. Therefore, the processor 120 may generate the virtual driving environment image 1530 based on the actual driving environment image 1510 and the virtual reality 1520.

According to an example, the processor 120 may recognize the road image through the actual driving environment image 1510, and generate the virtual driving environment image 1530 by reflecting the recognized road image in the virtual reality 1520. can do. In other words, since the road shape of the actual driving environment image 1510 is a section in which the user turns left after going straight for a predetermined distance, the processor 120 reflects the road shape in which the user turns left after going straight in the virtual reality 1520, the virtual driving environment image. 1530 may be generated. Therefore, when the virtual driving environment image 1530 is displayed on the display device 110 formed in the area of the front window of the autonomous vehicle 1, the occupant may recognize the virtual driving environment image 1530 as the actual driving environment. have. In addition, the processor 120 may recognize an object appearing in the actual driving environment image 1510 and determine whether to reflect the recognized object in the virtual driving environment image 1530. For example, the processor 120 may determine to reflect an object such as a traffic light and a crosswalk that appear in the actual driving environment image 1510 in the virtual driving environment image 1530. In addition, as shown in FIG. 15, the processor 120 may recognize the

vehicles

1511, 1512, and 1513 on the road of the actual driving environment image 1510, and recognize the

vehicles

1511, 1512, and 1513. It may be determined that the virtual driving environment image 1530 is not displayed.

According to another example, the processor 120 may recognize the road area through the actual driving environment image 1510, and replace the area other than the road area with the virtual reality 1520 within the actual driving environment image 1510. Can be. That is, when the actual driving environment image 1510 is a road area surrounded by buildings, and the virtual reality 1520 is in a forest with trees, the processor 120 is an area corresponding to the buildings in the actual driving environment image 1510. By replacing with a forest area, a virtual driving environment image 1530 can be generated.

According to another example, the processor 120 may recognize the driving route of the autonomous vehicle 1 through the road area shown in the actual driving environment image 1510, and not only the virtual driving environment image 1530 but also the driving route. Other virtual driving environment images corresponding to each of the points on the image may be generated.

In FIG. 15, an example of generating a virtual driving environment image 1530 is generated based on a camera installed in a front window of the autonomous vehicle 1, but the processor 120 is similarly installed in another window of the autonomous vehicle 1. Based on the camera, another virtual driving environment image may be generated. That is, the processor 120 may use the actual driving environment image acquired through the camera installed in the other vehicle window of the autonomous vehicle 1 to display another virtual driving environment to be displayed on the display device 110 formed in the area of the other vehicle window. An image can be generated.

Referring to FIG. 3 again, the processor 120 may control the display apparatus 110 formed in the area of the vehicle window of the autonomous vehicle 1 to display a virtual driving environment image. Therefore, the occupant can experience the virtual driving environment as if he / she looks at the area of the car window inside the autonomous vehicle 1, as if experiencing the actual driving environment. That is, the processor 120 may intend the occupant to mistake the actual driving environment through the virtual driving environment shown in the virtual driving environment image.

The processor 120 may control the display apparatus 110 to continuously display virtual driving environment images corresponding to each of the points on the driving route of the autonomous driving vehicle 1. That is, the processor 120 may generate virtual driving environment images corresponding to each of the points on the driving route, and control the display apparatus 110 to continuously display the generated virtual driving environment images.

For example, referring to FIG. 10, the processor 120 may control the display apparatus 110 formed in the area of the vehicle window to continuously display the virtual

driving environment images

1020 and 1030. Accordingly, the occupant may view the virtual

driving environment images

1020 and 1030 continuously displayed through the area of the car window, and thus the occupant may experience the virtual driving environment shown in the virtual

driving environment images

1020 and 1030. The occupant may recognize that the autonomous vehicle 1 is going straight in the virtual driving environment.

Similarly, referring to FIG. 11, the processor 120 may control the display apparatus 110 formed in the area of the vehicle window to continuously display the virtual

driving environment images

1120 and 1130. Accordingly, the occupant may view the virtual

driving environment images

1120 and 1130 continuously displayed through the area of the vehicle window, so that the occupant may experience the virtual driving environment shown in the virtual

driving environment images

1120 and 1130. The occupant may recognize that the autonomous vehicle 1 turns right in the virtual driving environment.

The processor 120 may control the display apparatus 110 to display a virtual driving environment image in association with the movement of the autonomous vehicle 1. The processor 120 acquires a virtual driving environment image corresponding to an operation in which the autonomous vehicle 1 travels straight, and a virtual driving environment image corresponding to an operation in which the autonomous vehicle 1 rotates left or right, respectively, as a video. can do. According to an example, the processor 120 may obtain a virtual driving environment image corresponding to the driving operation of the autonomous vehicle 1 from an external network. According to another example, the processor 120 may determine the autonomous vehicle 1. The virtual driving environment image corresponding to the driving operation may be generated. Therefore, when the autonomous vehicle 1 travels straight, the processor 120 may control the display apparatus 110 to reproduce the virtual driving environment image corresponding to the straight driving as a moving image. In addition, when the autonomous vehicle 1 travels left or right, the processor 120 may control the display apparatus 110 to reproduce a virtual driving environment image corresponding to the left turn driving or the right turn driving as a moving image. Therefore, the processor 120 may display the virtual driving environment image through the display apparatus 110 in association with the movement of the autonomous vehicle 1, and thus the passenger may realize the autonomous vehicle 1 more realistically. You can experience that you are driving in a virtual driving environment.

The motion sensing device 238 of FIG. 2 may sense the motion of the autonomous vehicle 1, and the processor 120 may be based on the motion of the autonomous vehicle 1 sensed by the motion sensing device 238. The display apparatus 110 may control to display a virtual driving environment image. The movement of the autonomous vehicle 1 may include at least one of speed, acceleration, deceleration, roll, pitch, yaw, and a change amount thereof, of the autonomous vehicle, and the movement sensing device. 238 may sense at least one of an acceleration, a deceleration, a roll, a pitch, a yaw, and a change amount thereof of the autonomous vehicle 1. In addition, the motion sensing device 238 may sense a traveling speed, a position change, and a direction change of the autonomous vehicle 1. In addition, the motion sensing device 238 may sense a driving state or a stopped state of the autonomous vehicle 1.

In addition, the windshield of the autonomous vehicle 1 may display a virtual driving environment so as to correspond to the movement control of the autonomous vehicle 1 by the control device 290. In other words, the controller 290 may control the movement of the autonomous vehicle 1, and the windshield of the autonomous vehicle 1 may display an image representing the virtual driving environment corresponding to the movement of the autonomous vehicle. have.

In addition, the autonomous vehicle 1 may further include a reproducing apparatus, and the reproducing apparatus may reproduce the virtual driving environment according to the movement control of the autonomous vehicle 1 by the controller 290, and may drive the autonomous driving. The windshield of the vehicle 1 can display the playback result of the playback device. In other words, the controller 290 may control the movement of the autonomous vehicle 1, the reproduction apparatus may reproduce an image representing the virtual driving environment corresponding to the movement of the autonomous vehicle, and the vehicle window may reproduce the image. Can be displayed. According to an example, the virtual driving environment may be 3D graphic data, and the playback device may be a graphics processing unit (GPU).

The processor 120 may control the display apparatus 110 to display virtual driving environment images corresponding to each of the points on the driving route of the autonomous vehicle 1 based on the movement of the autonomous vehicle 1. . While the display device 110 continuously displays the virtual driving environment images, when the motion sensing device 238 senses the stationary state of the autonomous vehicle 1, the processor 120 continuously displays the virtual driving environment images. Can be paused.

The processor 120 may control an image change rate of the virtual driving environment images displayed on the display apparatus 110 based on the movement of the autonomous vehicle 1. The image change rate may refer to a temporal change rate of the virtual driving environment images displayed on the display apparatus 100. That is, the image change rate may be a speed at which the virtual driving environment images are developed in the display apparatus 110. According to an example, when the motion sensing device 238 senses the driving speed of the autonomous vehicle 1, the processor 120 may display an image of the virtual driving environment displayed on the display device 110 based on the sensed speed. It is possible to control the rate of image change between them. For example, when the speed of the autonomous vehicle 1 is faster than before, the processor 120 may control the deployment speed of the virtual driving images displayed on the display apparatus 110 faster than before, and the autonomous vehicle When the speed of (1) is slower than before, the processor 120 may control the development speed of the virtual driving environment images displayed on the display apparatus 110 to be slower than before. Referring to FIG. 10, when the driving speed of the autonomous vehicle 1 is increased, the processor 120 controls the deployment speed of the virtual

driving environment images

1020 and 1030 to be displayed on the display device 110 faster. The passenger may be provided with a more realistic driving experience through the virtual

driving environment images

1020 and 1030 that are rapidly developed.

In addition, when there are a plurality of display apparatuses 110, the processor 120 is displayed on each of the plurality of display apparatuses 110 based on the movement of the autonomous vehicle 1 sensed by the movement apparatus 238. An image change rate between the virtual driving environment images may be controlled. That is, for example, when the display device 110 is formed in each of the area of the front window, the area of the right side window, and the area of the left side window of the autonomous vehicle 1, the processor 120 autonomously travels right. Based on the movement of the driving vehicle 1, the image change rate of the virtual driving environment images displayed on the display device 110 formed in the area of the left side window and the virtual display displayed on the display device 110 formed in the area of the right side window The image change rate of the driving environment images may be controlled differently. That is, in order to provide a rider with a more realistic driving experience, the processor 120 may display a virtual driving environment that is displayed on the display device 110 formed in the area of the left side window pane when the autonomous vehicle 1 runs in a right turn. The deployment speed of the images may be controlled to be faster than the deployment speed of the virtual driving environment images displayed on the display apparatus 110 formed in the area of the right side window.

When the display apparatus 110 is formed in the areas of the plurality of window windows, the processor 120 may determine an area of the window window to display the virtual driving environment image among the areas of the plurality of window windows. According to an example, the processor 120 may determine an area of the window window to display the virtual driving environment image among the areas of the plurality of window windows based on the passenger's selection.

In addition, according to another example, the processor 120 may determine an area of the car window corresponding to the line of sight of the passenger among the areas of the plurality of car windows as an area of the car window to display the virtual driving environment image. For example, the image sensor 228 of FIG. 2 may detect a driver's gaze, and the processor 120 may detect an area of the car window corresponding to the detected driver's gaze among the areas of the plurality of car windows, in a virtual driving environment. The area of the window to display the image may be determined. According to an example, when there are a plurality of occupants in the autonomous driving vehicle 1, a virtual driving environment image may be displayed on an area of a vehicle window corresponding to the eyes of the occupant based on a predetermined occupant among the plurality of occupants. Can be determined by the area of the windshield. According to another example, when there are a plurality of occupants in the autonomous vehicle 1, the processor 120 stops detecting the eyes of the occupants, and sets the area of the preset window as an area of the window to display the virtual driving environment image. You can decide.

When the display apparatus 110 is formed in the areas of the plurality of window windows, the processor 120 may provide a passenger with a UI 1610 that selects an area of the window window to display the virtual driving environment image among the areas of the plurality of window windows. Can be. That is, as shown in FIG. 16, the display apparatus 110 includes an area of the front window 401 of the autonomous vehicle 1, an area of the left window 403, an area of the right window 402, and a rear window ( When formed in the area of 404 and in the area of the ceiling window 405, the processor 120 includes an area of the front window 401, an area of the left side window 402, an area of the right side window 402, a rear window 404. The passenger 16 may be provided with a UI 1610 that selects which area of the vehicle window and the area of the ceiling window 405 to display the virtual driving environment image. Accordingly, the occupant may select an area of the vehicle window to display the virtual driving environment through the UI 1610.

The processor 120 may determine the

areas

401 and 403 of the car window corresponding to the line of sight of the occupant 1710 among the

areas

401, 402, 403 and 405 of the plurality of car windows as the area of the car window to display the virtual driving environment image. More specifically, the image sensor 228 may detect the gaze of the occupant 1710, and the processor 120 may virtually identify the

areas

401 and 403 of the windshield located within a specific angle range with the gaze direction of the occupant 1710. It may be determined as an area of a vehicle window to display the driving environment image.

If the occupant 1710 turns his head to the right, the processor 120 may determine the

areas

401 and 402 of the car window corresponding to the gaze of the occupant 1710 as the area of the car window to display the virtual driving environment image. .

Referring back to FIG. 3, the processor 120 may control the display apparatus 110 to display content selectable by the occupant. The content may be a content such as an image or a picture provided through the Internet or computer communication, or may be an image provided by the autonomous vehicle 1 itself. The processor 120 may provide a passenger to select a content to the passenger and control the display device 110 to display the content selected by the passenger.

The processor 120 may provide a passenger with a UI 1810 for selecting content to be displayed on the display apparatus 110. In other words, the processor 120 may provide a passenger with a UI 1810 capable of selecting a YouTube, a movie library, or Netflix. In addition, the processor 120 may provide a passenger with a UI 1810 capable of selecting an image captured by the image sensor 228 installed in the vehicle window, and provide a passenger with a UI 1810 capable of selecting a virtual driving environment image. Can be provided to

The occupant 1910 may select a movie as content to be displayed on the display apparatus 110 through the UI 1810 of FIG. 18. Subsequently, the occupant 1910 may lie in the autonomous vehicle 1 and look at the area of the car window 405 corresponding to the ceiling.

Therefore, the processor 120 may control to display a movie on the display device 110 formed in the area of the window 405 corresponding to the ceiling of the occupant 1910.

Referring back to FIG. 3, the processor 120 may determine whether a preset event occurs. When a preset event occurs, the processor 120 may provide the passenger with information associated with the preset event. According to an example, when a preset event occurs, the processor 120 may control an image of an actual driving environment associated with the event to be displayed on the display apparatus 110. That is, when a preset event occurs, the processor 120 may control the passenger of the autonomous vehicle 1 to view the actual driving environment corresponding to the event through the display device 110. When the preset event occurs while the display device 110 displays the virtual driving environment image, the processor 120 may control the display device 110 to display the changed driving image as an image of the actual driving environment associated with the event. Can be. In addition, the processor 120 may control the display apparatus 110 to simultaneously display the virtual driving environment image and the image of the actual driving environment associated with the event. That is, since the autonomous vehicle 1 displays the virtual driving environment image or the content through the display device 110 formed in the vehicle window, the occupant cannot see the actual driving environment around the autonomous vehicle 1. Information on the set event can be provided to the passenger separately.

The preset event may be a situation in which the autonomous vehicle 1 is stopped for a preset time. For example, when the autonomous vehicle 1 is stopped for 30 seconds or more due to traffic jam, the processor 120 may determine that a predetermined event has occurred. Subsequently, the processor 120 may control the traffic congestion situation, which is an actual driving environment associated with a preset event, to be displayed on the display apparatus 110.

The preset event may be a situation in which the weather around the autonomous vehicle 1 changes. For example, when the weather of the autonomous vehicle 1 changes from a clear weather to a rainy weather, the processor 120 may determine that a predetermined event has occurred. Subsequently, the processor 120 may control the display apparatus 110 to display a rainy image captured by the image sensor 228 which is an actual driving environment associated with a preset event.

The preset event may be a situation in which a change in the physical state of the occupant of the autonomous vehicle 1 exists. For example, when the occupant changes from the waking state to the sleeping state, the processor 120 may determine that a predetermined event has occurred. More specifically, the image sensor 228 may photograph the occupant's eyes, and the processor 120 may close the occupant's eyes more than a reference ratio compared to the normal state, or when the occupant's eyes are closed for more than the reference time. The passenger may be determined to be in a sleep state at. Subsequently, the processor 120 may stop displaying the virtual driving environment image and turn off the internal lighting 245 of the autonomous vehicle 1 in order not to disturb the sleep of the occupant.

The processor 120 may provide a UI 2010 for setting an event for the occupant. The occupant is responsible for a number of events (e.g., the vehicle suddenly changes speed, the vehicle enters a freeway, the vehicle is located near a landmark, the vehicle arrives at its destination, the weather changes, The UI 2010 may preset whether to be provided with information about future events based on which event of a road condition is dangerous and a situation where an emergency vehicle is around. Therefore, when the event selected in the UI 2010 occurs, the processor 120 may provide information about the selected event to the occupant.

The processor 120 may control the virtual driving environment image 2110 to be displayed on the display device 110. The autonomous vehicle 1 may detect that a wild animal has suddenly appeared while driving, and the autonomous vehicle 1 may suddenly change speed. Subsequently, the processor 120 may determine that a predetermined event, which is a situation in which the speed suddenly changes, occurs. Subsequently, the processor 120 controls the display apparatus 110 displaying the virtual driving environment image 2110 to display an image 2120 of a wild animal, which is a real driving environment associated with a preset event, in a partial region. Can be.

The processor 120 may control the virtual driving environment image 2210 to be displayed on the display device 110. In this case, the autonomous vehicle 1 may recognize that the current location is around the landmark while driving, and the processor 120 may determine that a preset event, which is a situation that is located around the landmark, has occurred. Subsequently, the processor 120 may control the display apparatus 110 to change and display a landmark, which is an actual driving environment associated with a preset event, in the virtual driving environment image 2210 as a captured image 2220.

According to another example, when the display device 110 is a transparent display, in a situation in which the autonomous vehicle 1 is positioned around a landmark, the processor 120 may display a display device that displays a virtual driving environment image 2210. By transparently processing 110, the passenger may view the actual driving environment 2220 through the transparently processed display device 110.

The processor 120 may control the virtual driving environment image 2310 to be displayed on the display device 110. The processor 120 may recognize that the surrounding weather is raining while the autonomous vehicle 1 is driving, and determine that a preset event has occurred. Subsequently, the processor 120 may provide information to the occupant via the sound output unit 282 that rain comes.

The method shown in FIG. 24 may be a method performed in time series by the autonomous vehicle 1 described in the drawings.

In operation 2410, the autonomous vehicle 1 may acquire a virtual driving environment image that replaces the actual driving environment of the autonomous vehicle 1.

The autonomous vehicle 1 may generate a virtual driving environment image based on the information on the driving route from the current position of the autonomous vehicle 1 to the destination. More specifically, the autonomous vehicle 1 may obtain information about a driving route from the current position of the autonomous vehicle 1 to the destination, and reflect the driving route obtained in the preset virtual reality, thereby driving virtually. An environment image can be generated. In addition, the autonomous vehicle 1 may generate virtual driving environment images corresponding to each of the points based on the points on the driving route of the autonomous vehicle 1.

The autonomous vehicle 1 may generate a virtual driving environment image based on an image of the actual driving environment of the autonomous vehicle 1. The autonomous vehicle 1 may acquire an image of the actual driving environment of the autonomous vehicle 1, and the autonomous vehicle 1 may generate a virtual driving environment image based on the acquired image of the actual driving environment. Can be generated. More specifically, the autonomous vehicle 1 may recognize a road image through an image of an actual driving environment, and generate a virtual driving environment image by reflecting the recognized road image in a virtual reality.

The autonomous vehicle 1 may acquire a virtual driving environment image from an external network. The autonomous vehicle 1 is a moving image of the virtual driving environment corresponding to the operation of driving the autonomous driving vehicle 1, and the virtual driving environment image corresponding to the operation of driving the left or right turn of the autonomous vehicle 1, respectively, the video Can be obtained.

In addition, the autonomous vehicle 1 may acquire a virtual driving environment image through the input device 260. More specifically, the input device 260 of the autonomous vehicle 1 may receive a selection of a virtual driving environment from the user, and acquire an image indicating the virtual driving environment selected from the user.

In operation 2420, the autonomous vehicle 1 may control the display device formed in the area of the window of the autonomous vehicle 1 to display the virtual driving environment image. The autonomous vehicle 1 may control the display device to continuously display virtual driving environment images corresponding to each of the points on the driving route of the autonomous vehicle 1. In addition, the autonomous vehicle 1 may include a virtual driving environment image corresponding to an operation in which the autonomous vehicle 1 travels straight, and a virtual driving environment image corresponding to an operation in which the autonomous vehicle 1 travels left or right. The display device can be controlled to play back as a moving picture.

In addition, the windshield of the autonomous vehicle 1 may display the virtual driving environment selected by the user in s2410. That is, the vehicle window of the autonomous vehicle 1 may display an image representing the selected virtual driving environment.

FIG. 25 is a detailed flowchart embodying step 2420 of FIG. 24.

In operation 2510, the autonomous vehicle 1 may sense a movement of the autonomous vehicle 1. The autonomous vehicle 1 may sense a traveling speed, a position change, and a direction change of the autonomous vehicle 1. In addition, the autonomous vehicle 1 may sense a driving state or a stationary state of the autonomous vehicle 1.

In operation 2520, the autonomous vehicle 1 may control the display device to display the virtual driving environment image based on the sensed movement.

The autonomous vehicle 1 may control the display device to display virtual driving environment images corresponding to each of the points on the driving route of the autonomous vehicle 1 based on the sensed movement. While the display device continuously displays the virtual driving environment images, when the stationary state of the autonomous vehicle 1 is sensed, the autonomous vehicle 1 may pause the continuous display of the virtual driving environment images.

The autonomous vehicle 1 may control an image change rate of the virtual driving environment images displayed on the display device based on the sensed movement. The image change rate may be a speed at which the virtual driving environment images are developed on the display device. Therefore, when the driving speed of the autonomous vehicle 1 is sensed, the autonomous vehicle 1 may control the rate of change of the image between the virtual driving environment images displayed on the display device based on the sensed speed.

When the autonomous vehicle 1 travels straight, the autonomous vehicle 1 may control the display apparatus to reproduce a virtual driving environment image corresponding to the straight driving as a moving image. In addition, the autonomous vehicle 1 may control the display apparatus 110 to reproduce a virtual driving environment image corresponding to the left turn driving or the right turn driving as a moving image when the autonomous driving vehicle 1 travels left or right. .

The method shown in FIG. 26 may be a method performed in time series by the autonomous vehicle 1 described above in the figures.

In operation 2610, the autonomous vehicle 1 may acquire a virtual driving environment image that replaces the actual driving environment of the autonomous vehicle 1. Step 2610 may correspond to step 2410 of FIG. 24.

In operation 2620, the autonomous vehicle 1 may control the display device formed in the area of the window of the autonomous vehicle 1 to display the virtual driving environment image. Step 2620 may correspond to step 2420 of FIG. 24.

In operation 2630, the autonomous vehicle 1 may determine whether a preset event occurs. The preset event may be a situation in which a vehicle suddenly changes speed, a vehicle enters a highway, a vehicle is located near a landmark, a vehicle arrives at a destination, a weather is changed, and a surrounding The road condition may be at least one of a dangerous situation and an emergency vehicle in the vicinity.

As a result of the determination in operation 2630, when the preset event occurs, the autonomous vehicle 1 may control the passenger of the autonomous vehicle 1 to view the actual driving environment corresponding to the event through the display device. According to an example, when a preset event occurs while the display device displays the virtual driving environment image, the autonomous vehicle 1 controls the display device to change and display the image of the actual driving environment associated with the event. can do. In addition, the autonomous vehicle 1 may control the display device to simultaneously display the virtual driving environment image and the image of the actual driving environment associated with the event.

The device according to the above-described embodiments includes a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, a touch panel, a key, a button And a user interface device such as the like. Methods implemented by software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. The computer-readable recording medium may be a magnetic storage medium (eg, read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optical reading medium (eg, CD-ROM). ) And DVD (Digital Versatile Disc). The computer readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium is readable by the computer, stored in the memory, and can be executed by the processor.

This embodiment can be represented by functional block configurations and various processing steps. Such functional blocks may be implemented in various numbers of hardware or / and software configurations that perform particular functions. For example, an embodiment may include an integrated circuit configuration such as memory, processing, logic, look-up table, etc. that may execute various functions by the control of one or more microprocessors or other control devices. You can employ them. Similar to the components that may be implemented as software programming or software elements, the present embodiment includes various algorithms implemented in C, C ++, Java (data structures, processes, routines or other combinations of programming constructs). It may be implemented in a programming or scripting language such as Java), an assembler, or the like. The functional aspects may be implemented with an algorithm running on one or more processors. In addition, the present embodiment may employ the prior art for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism", "element", "means", "configuration" can be used widely and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in conjunction with a processor or the like.

Specific implementations described in this embodiment are examples, and do not limit the technical scope in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of the lines between the components shown in the drawings by way of example shows a functional connection and / or physical or circuit connections, in the actual device replaceable or additional various functional connections, physical It may be represented as a connection, or circuit connections.

In this specification (particularly in the claims), the use of the term “above” and similar indicating terminology may correspond to both the singular and the plural. In addition, when a range is described, it includes the individual values which belong to the said range (if there is no description contrary to it), and it is the same as describing each individual value which comprises the said range in detailed description. Finally, if there is no explicit order or contrary to the steps constituting the method, the steps may be performed in a suitable order. It is not necessarily limited to the order of description of the above steps. The use of all examples or exemplary terms (eg, etc.) is for the purpose of describing technical concepts in detail and is not to be limited in scope by the examples or exemplary terms unless defined by the claims. In addition, one of ordinary skill in the art appreciates that various modifications, combinations and changes can be made depending on design conditions and factors within the scope of the appended claims or equivalents thereof.

Claims

In an autonomous vehicle,

A display device formed in an area of a car window of the autonomous vehicle; And

And a processor configured to control the display device to display a virtual driving environment image that replaces the actual driving environment of the autonomous driving vehicle.
The method of claim 1,

The virtual driving environment image,

And an image representing a virtual driving environment outside the autonomous vehicle as viewed toward an area of the vehicle window from a viewpoint inside the autonomous vehicle.
The method of claim 1,

The processor,

And obtaining information on a driving route from a current position of the autonomous driving vehicle to a destination, and generating virtual driving environment images corresponding to each of the points on the driving route.
The method of claim 3, wherein

Further comprising: a motion sensing device for sensing the movement of the autonomous vehicle,

The processor,

And the display apparatus controls to display the virtual driving environment images based on the sensed movement.
The method of claim 4, wherein

The motion sensing device,

Sensing the driving speed of the autonomous vehicle,

The processor,

And controlling an image change rate between the virtual driving environment images displayed on the display device based on the sensed driving speed.
The method of claim 4, wherein

The processor,

And a plurality of display devices, based on the sensed movement, controlling an image change rate between the virtual driving environment images displayed on each of the plurality of display devices.
The method of claim 1,

And an image sensor for capturing an image of the actual driving environment.

The processor,

And generating the virtual driving environment image based on the captured image of the actual driving environment.
The method of claim 7, wherein

The processor,

And generating the virtual driving environment image reflecting the appearance of the object shown in the image of the actual driving environment.
The method of claim 1,

The processor,

The virtual driving environment image is generated based on a virtual reality selected from a passenger of the autonomous vehicle among a plurality of virtual reality.
The method of claim 1,

The processor,

And determining whether or not a predetermined event occurs, and when the event occurs, controlling the passenger of the autonomous vehicle to view the actual driving environment corresponding to the event through the display device.
In a method of operating an autonomous vehicle,

Acquiring a virtual driving environment image in place of an actual driving environment of the autonomous vehicle; And

And controlling a display device formed in an area of a car window of the autonomous vehicle to display the virtual driving environment image.
The method of claim 11,

The virtual driving environment image,

And an image representing a virtual driving environment outside of the autonomous vehicle viewed toward the area of the vehicle window from a viewpoint inside the autonomous vehicle.
The method of claim 11,

Acquiring the virtual driving environment image,

Obtaining information about a driving route from a current position of the autonomous vehicle to a destination; And

Generating virtual driving environment images corresponding to each of the points on the driving route.
The method of claim 13,

Sensing the movement of the autonomous vehicle;

The controlling step,

And controlling the display apparatus to display the virtual driving environment images based on the sensed movement.
The method of claim 14,

The sensing step,

Sensing the driving speed of the autonomous vehicle,

The controlling step,

And controlling an image change rate between the virtual driving environment images displayed on the display device based on the sensed driving speed.
The method of claim 14,

The controlling step,

And controlling the image change rate between the virtual driving environment images displayed on each of the plurality of display apparatuses based on the sensed movement when there are a plurality of display apparatuses.
The method of claim 11,

Photographing an image of the actual driving environment;

Acquiring the virtual driving environment image,

And generating the virtual driving environment image based on the captured image of the actual driving environment.
The method of claim 17,

Acquiring the virtual driving environment image,

And generating the virtual driving environment image reflecting the appearance of the object shown in the image of the actual driving environment.
The method of claim 11,

Acquiring the virtual driving environment image,

The virtual driving environment image is generated based on a virtual reality selected from a passenger of the autonomous vehicle among a plurality of virtual reality.
The method of claim 11,

Determining whether a preset event occurs; And

And determining that the event occurs, the occupant of the autonomous vehicle viewing the actual driving environment corresponding to the event through the display device. .
A computer-readable recording medium having recorded thereon a program for executing the method of claim 11 on a computer.
An input device for selecting a virtual driving environment from a user; And

And a windshield for displaying the selected virtual driving environment.
The method of claim 22,

The vehicle window displays the virtual driving environment to correspond to the movement of the autonomous vehicle.
The method of claim 23,

Wherein the movement comprises at least one of speed, acceleration, deceleration, roll, pitch, yaw and variations of the autonomous vehicle.
The method of claim 22,

Further comprising a control device for controlling the movement of the autonomous vehicle,

And the vehicle window displays the virtual driving environment to correspond to the control of the control device.
The method of claim 25,

Reproducing apparatus for reproducing the virtual driving environment under the control of the control device,

And the vehicle window displays a result of the regeneration of the reproducing apparatus.