WO2018058361A1 - Method, system, display device for displaying virtual reality in vehicle, and computer program product - Google Patents

Method, system, display device for displaying virtual reality in vehicle, and computer program product Download PDF

Info

Publication number
WO2018058361A1
WO2018058361A1 PCT/CN2016/100542 CN2016100542W WO2018058361A1 WO 2018058361 A1 WO2018058361 A1 WO 2018058361A1 CN 2016100542 W CN2016100542 W CN 2016100542W WO 2018058361 A1 WO2018058361 A1 WO 2018058361A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
display device
altitude
movement
information
Prior art date
Application number
PCT/CN2016/100542
Other languages
French (fr)
Inventor
Lu Chen
Carsten Isert
Tao Xu
Original Assignee
Bayerische Motoren Werke Aktiengesellschaft
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bayerische Motoren Werke Aktiengesellschaft filed Critical Bayerische Motoren Werke Aktiengesellschaft
Priority to PCT/CN2016/100542 priority Critical patent/WO2018058361A1/en
Publication of WO2018058361A1 publication Critical patent/WO2018058361A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics

Definitions

  • the present invention relates to a method for displaying virtual reality in a vehicle, a system for displaying virtual reality in the vehicle, a display device for displaying virtual reality in a vehicle, and a computer program product.
  • VR Virtual Reality
  • a plurality of fields including, for example, entertainment, real estate development, education and training, industrial stimulation, and the like.
  • VR devices have been applied in vehicles only very limitedly.
  • a VR device is combined with a vehicle stimulator to stimulate the traffic on the vehicle stimulator in order to help training the operation skills of vehicle drivers.
  • the task of the present invention lies in providing immersive experience for a person in a vehicle.
  • a method for displaying VR in a vehicle comprising: acquiring information about the position, orientation and movement of the vehicle; constructing VR based on map data according to the position, orientation and movement of the vehicle; and outputting the VR on VR output means.
  • vehicles include land vehicles, aircrafts, or watercrafts, wherein land vehicles particularly refer to automotive vehicles, rail cars, motors, etc., aircrafts particularly refer to airplanes, airships, etc., and watercrafts particularly refer to ships, yachts, etc.
  • the VR output means may be a VR head-mounted display, such as VR glasses, VR helmets, or a stereoscopic display system including a stereoscopic image display and 3D glasses, or a cave type stereoscopic display system (CAVE system) using the surface of the vehicle as the projection surface, and alternatively, it may be a naked eye stereoscopic display system.
  • the present invention is not limited to the VR output means as listed above, and it is self-evident that other VR output means may be also included.
  • the VR is preferably rendered very realistic.
  • the VR may also be rendered to be relatively abstractive such that the viewer would pay more attention to the most important information.
  • the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for the person in the vehicle.
  • a three-dimensional dynamic visual scene with respect to the exterior environment of the vehicle can be provided for the person in the vehicle, wherein the three-dimensional dynamic visual scene can move along with movement of the vehicle in the real world.
  • the VR according to the present invention firstly visualizes map data, and such VR is also fused with the actual movement of the vehicle in which a person is located.
  • the person in the vehicle can, by means of the VR, in particular through the casing of the vehicle, without any visual blocking, the exterior environment of the vehicle. Therefore, the present invention can provide immersive experience for the person in the vehicle with the movement of the vehicle.
  • the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
  • those in a land vehicle can view the exterior environment of the vehicle with an eagle perspective to experience like floating in the air.
  • those viewing the VR with an eagle perspective in the dynamic three-dimensional map view are located above the actual position of the vehicle.
  • the position of the VR viewed by the person is equivalent to a current car position (CCP) in a traditional navigation system.
  • CCP current car position
  • the VR viewed by the person is based on an altitude different from the actual position of the vehicle.
  • the altitude at which the perspective is can be adjusted.
  • the person in the vehicle can adjust his/her perspective to obtain a better eyeshot.
  • the person can, for example, adjust upwardly the altitude so as to view the map content at a more distant place from a higher altitude, or to view beyond high buildings so as to avoid blocking of the eyeshot by the buildings.
  • This is especially advantageous for looking for a geographic location by VR.
  • the vehicle is an underground rail car
  • passengers can view the landscapes on the ground by the VR output means.
  • the altitude is configured to be extremely high, such as hundreds of meters, a broad range of region where the vehicle is located can be viewed from an overhead viewpoint.
  • the entertainment of the viewing of the VR can be also enhanced.
  • the person can, for example, downwardly adjust the altitude so as to view the VR scene on the ground surface where the aircraft sweeps over a place more close to the ground surface.
  • a novel way of entertainment is provided for those taking the aircraft.
  • virtual vehicles different from said vehicle can be presented, such as planes, rockets, flying carpets or airships.
  • the candidate virtual vehicles include, for example, planes, rockets, flying carpets or airships.
  • the flying carpet is also called a magic carpet, which is an imaginary carpet that can carry people to fly in the air, and it is a vehicle in many mythologies, legends, fictions, cartoons or other works.
  • the carpet located below is additionally displayed such that the person viewing the VR feels like being on the flying carpet.
  • the person in the vehicle can experience a feeling of driving or taking another vehicle, thereby obtaining a novel and exciting driving or taking experience.
  • a particularly preferred embodiment of the present invention is that in the VR, the person in the vehicle can select the function of flying carpet (or magic carpet) .
  • flying carpet or magic carpet
  • the altitude can be automatically adjusted so that the viewer can avoid “collision” with such building structures in the VR.
  • information about current route, course or surrounding environment can be displayed in the VR.
  • the information is, for example, a starting point, an end point, a predetermined route planning, travel data, names of the streets and buildings in the surrounding environment about the current route. In this way, those in the vehicle can obviously feel continuously approaching of the end point of the route.
  • the so-called Point of Interest can be displayed in the VR.
  • the POI may relate to scenic spots, such as rivers, places of historic interests like castles, museums or monuments, and/or to infrastructures such as bridges, traffic lights, telephone booths, shops, parking lots, etc.
  • the names of the corresponding POIs can be stored in the navigation database.
  • the viewer of the VR may view the scenic spots in the initial vehicle or rapidly look for the desired infrastructure.
  • those in the vehicle can scan the surrounding scenic spots at different altitudes; for example, he/she can view the trend of the whole river, the trend of the mountain or the arrangement of the landscape, and can also view the details of the scenic spot or infrastructure locally.
  • the information stored in the address book of the navigation system can be displayed in the VR.
  • the addresses of family members and friends are stored in the address book, and in the VR according to the present invention, such places are marked additionally.
  • a balloon symbol of “Anna's Home” is marked above the home of Anna.
  • the embodiments of the present invention are based on the following consideration: in the VR technique, a technical problem that cannot be ignored is that the VR user often has motion sickness. The main reason for the motion sickness lies in the incoordination between the pictures visually observed by the user and the real state of the body.
  • the vehicle has alternative acceleration and deceleration and at the same time jolts up and down
  • organs in charge of balance such as the inner ear deliver signals to the brain indicating the human body is in motion, such that the visual signals contradicts with the somatosensory signals in the brain, and for this, the brain produces a dizzy feeling to warn the user to escape from the current state, thereby causing motion sickness.
  • the present invention in order to avoid such motion sickness, it is provided that all actual movements of the vehicle can be reproduced in the VR.
  • the speed, acceleration and deceleration, jolts up and down, deflection left and right and the like of the vehicle are all realistically reflected in the VR scenes presented to those in the vehicle.
  • those in the vehicle can have a feeling directly connected with the motion, and on the other hand, the visual pictures received by those in the vehicle when viewing the VR coordinates with the actually felt motion, so that no motion sickness would occur.
  • the motion state of the head of the person wearing the display device is detected, and all the motions of the head of the person can be reproduced in the VR.
  • the motion of the human body, especially the head can be very accurately reflected in the VR, so that the visual signals received by the brain is consistent with the somatosensory signals, thereby avoiding as far as possible the occurrence of motion sickness.
  • moving or zooming operations can be performed in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling.
  • the direction is maintained to be the same as in the real world such that the VR viewer would feel comfortable without occurring motion sickness.
  • the interactivity between the VR and the viewer can be enhanced.
  • the viewer can view any place in the VR. In this way, for example, it is possible to conveniently look for a target and see the environment in the proximity of the target.
  • such an embodiment also gives a possibility of manipulating the travelling in the VR. For example, those in the vehicle can set the destination, add POIs, change the travelling route, and change the travelling mode (such as energy-saving type, sport type and comfortable type) .
  • movement and zoom operations as well as manipulation of the information may be performed by a data glove or a manipulation handle provided with the display device, or by tracking motion of the eyeball of the viewer.
  • hazard situations and navigation instructions can be displayed in the VR.
  • the content of augmented reality (AR) is additionally displayed in the VR.
  • actual hazard situations such as pedestrians
  • navigation instructions may be superimposed on the VR pictures.
  • the VR can be output on the VR output means with perspectives from different positions in the vehicle. Hence, those sit at the rear part of a vehicle can see the scenes that can be seen from the front part of the vehicle.
  • the other users can be displayed and communications with the other users can be performed.
  • the other users switching on the function of displaying the VR are visible to one another simultaneously in the same VR world and can interact with one another, such as sending messages to one another.
  • the other users may relate to persons located within the same vehicle, persons located in other vehicles and persons located at fixed positions. Under the condition that the other users are persons located within the same vehicle, persons taking the same vehicle can be seen in the VR such that the persons viewing the VR would not feel isolated, thereby enhancing conformity to the real scene. Under the condition that the other users are persons located in other vehicles or persons located at fixed positions, one the one hand, their positions are visible to each other.
  • said embodiment of the present invention may be based on communication techniques like V2V and V2X, and a sequence of traffic information including real-time traffic, road information, pedestrian information and the like can be obtained, thereby enhancing the driving security, relieving the traffic jam, increasing the traffic efficiency and providing on-board entertainment information.
  • the vehicle can be positioned by using the position service of the Global Positioning System (GPS) , the GLONASS, the Galileo, the COMPASS, a laser radar or the position service of a wireless communication network of a telecommunication mobile operator.
  • GPS Global Positioning System
  • GLONASS Global Positioning System
  • Galileo Galileo
  • COMPASS COMPASS
  • laser radar a laser radar
  • the orientation according to the present invention may be a space orientation of the vehicle, i.e., the orientation includes not only a two-dimensional position orientation directed by the head part of the vehicle but also a pitch of the vehicle with respect to the horizontal plane.
  • the orientation can be determined by, for example, a direction sensor.
  • the movement of the vehicle is detected by a speed sensor, a gyroscope, an acceleration sensor, a steering angle sensor, or the like.
  • information about the movement of the vehicle includes acceleration, speed, and deflection.
  • the acceleration, speed and deflection can be obtained by the controller, the speed sensor, the acceleration sensor, the steering angle sensor, the gyroscope or the like of the vehicle per se.
  • a system for displaying VR in a vehicle comprising:
  • -a receiving unit for acquiring information about the position, orientation and movement of the vehicle
  • -a construction unit for constructing VR based on map data according to the position, orientation and movement of the vehicle
  • the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle.
  • the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
  • the system for displaying VR in a vehicle further comprises a perspective adjusting unit for adjusting the altitude at which the perspective is such that the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
  • a display device for displaying VR in a vehicle comprising:
  • a memory configured to store computer executable instructions
  • a processor configured to execute the computer executable instructions, wherein when the computer executable instructions are executed by the processor, the display device is enable to:
  • the map data can be provided by the vehicle via a interface of the display device, and/or the map data can be provided by the memory of the display device, and/or the map data can be downloaded from a back-end server.
  • the map data can be obtained from a plurality of sources.
  • the map data according to the present invention may relate to real scene three-dimensional maps or stimulated three-dimensional maps.
  • the map data can be stored or downloaded to the vehicle and/or the memory of the display device in the form of a map database. If the map data are only stored in the vehicle, the desired map data is transmitted in real time or in advance to the display device via the interface of the display device.
  • transmission of data can relate to wired connection (such as vehicle bus, like Local Interconnect Network (LIN) bus, Controller Area Network (CAN) bus or FlexRay bus) or to wireless connection.
  • the wireless connection may be a WLAN for example, in accordance with IEEE802.11a, IEEE802.11b, IEEE802.11ac, IEEE802.11ad, IEEE802.11g, IEEE802.11h, or IEEE802.11n or Bluetooth, in particular Bluetooth V2.0, Bluetooth V3.0 or Bluetooth V4.0.
  • the display device comprises an detector configured to detect the motion state of the head of the person wearing the display device, and to reproduce all the motions of the head of the person in the VR by the display device.
  • the motion of the human body, especially the head can be very accurately reflected in the VR, so that the visual signals received is consistent with the somatosensory signals by the brain, thereby avoiding as far as possible the occurrence of motion sickness.
  • the display device when the computer executable instructions are executed by the processor, the display device is enabled to execute the methods as described in the various embodiments of the present invention.
  • the features in the method according to the present invention can be applied to the display device according to the present invention.
  • the display device is further equipped with or connected to a data glove, an operation handle or means for tracking the eyeball movement of the viewer, such that the user may perform moving or zooming operations in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling.
  • a data glove an operation handle or means for tracking the eyeball movement of the viewer
  • the user may perform moving or zooming operations in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling.
  • a computer program product which can be loaded to an internal memory of a computer and comprises software code segments, when the product is executed on the computer, the methods according to various embodiments of the present application are implemented by using the software code segments.
  • Fig. 1 is a schematic diagram illustrating an example according to the present invention.
  • Fig. 2 is a flow chart illustrating a method for displaying VR in a vehicle according to the present invention.
  • Fig. 3 is a block diagram illustrating one example of a hierarchy for displaying VR in a vehicle according to the present invention.
  • Fig. 4 is a block diagram illustrating another example of a hierarchy for displaying VR in a vehicle according to the present invention.
  • Fig. 1 illustrates an example of displaying VR in a vehicle according to the present invention.
  • the vehicle relates to a motor vehicle 3.
  • Persons 1 in the motor vehicle wear a display device 2 for display VR, respectively.
  • the display device 2 is a VR head-mounted display.
  • the persons wearing the display device in the motor vehicle usually include a person taking the motor vehicle, such as a person on the co-pilot seat, or a person on a rear seat.
  • the persons wearing the display device in the motor vehicle also include the driver on the pilot seat.
  • the motor vehicle can be automatically driven according to parameters including the destination, the course, etc. preset by the driver, and the travel substantially requires no participation of the driver.
  • the driver may be notified to take off the display device to manipulate the motor vehicle.
  • Fig. 2 is a flow chart illustrating a method for displaying VR in a vehicle according to the present invention.
  • the present invention firstly, information about the position, orientation and movement of the vehicle is acquired, and VR is constructed according to the position, orientation and movement of the vehicle and based on map data so as to output the VR on the VR output means.
  • VR is constructed according to the position, orientation and movement of the vehicle and based on map data so as to output the VR on the VR output means.
  • construction of the VR may be performed on a processor integrated in the display device, or in data processing means disposed in the vehicle.
  • the constructed VR can be transmitted to the display device via an interface exchanging data with the vehicle so as to output the VR on the VR output means.
  • the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle.
  • the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
  • the person wearing the display device can see VR scenes at an altitude different from the actual position of the vehicle.
  • the person can view the exterior environment of the motor vehicle with an eagle perspective.
  • the VR scenes seen by the person are just like scenes that can be observed at a position above the vehicle, particularly the motor vehicle, such that the person can experience like floating in the air.
  • the altitude at which the perspective is can be adjusted, in particular by the display device.
  • the person in the motor vehicle for example, can see VR scenes 10 m, 30 m, 50 m, 100 m, 200 m and the like above the motor vehicle.
  • map contents at a more distant place can be viewed at a higher altitude, and at the same time, can be viewed beyond high buildings so as to avoid blocking of the eyeshot by the buildings.
  • the altitude is adjustable, the entertainment of viewing of the VR can be enhanced.
  • VR virtual vehicles different from the current vehicle
  • a particularly preferred virtual vehicle is a flying carpet or magic carpet, since a vehicle like a flying carpet has not yet been developed, and in practice, people are not yet able to have an experience of sitting on a flying carpet.
  • information about current route, course or surrounding environment can be displayed in the VR.
  • the route to be traveled can be highlighted.
  • the ground to be passed along the planned route can be colored with green, and a flag label can be added to the position of destination.
  • information about the current course can be displayed, including, for example, remaining mileages, expected remaining time, expected arrival time, etc.
  • names of the buildings around the current position can be displayed.
  • identifiers or descriptions may be added to the POIs; for example, balloon identifiers can be added above shops, parking lots, and scenery spots.
  • Fig. 3 is a block diagram illustrating one example of a hierarchy 4 for displaying VR in a vehicle according to the present invention, wherein the hierarchy 4 according to the present invention comprises a display device for displaying VR in the vehicle and a vehicle.
  • motor vehicle 3 as shown in Fig. 1 is taken as an example of the vehicle.
  • the motor vehicle 3 includes a sensor 7, configured to detect information about the position, orientation and movement of the vehicle.
  • the motor vehicle 3 is further provided with an interface 6 for exchanging data with the display device 2.
  • Fig. 3 is exemplarily illustrates a display device 2 according to the present invention which is used for displaying VR in the vehicle.
  • the display device 2 includes VR output means 10; a memory 9 configured to store computer executable instructions; and a processor 8 configured to execute the computer executable instructions, wherein when the computer executable instructions are executed by the processor, the display device is enable to:
  • the display device may further include an interface 5 exchanging data with the vehicle (which is the motor vehicle 3 here) , through which information about the position, orientation and movement of the vehicle can be acquired.
  • the processor may construct VR based on map data according to the position, orientation and movement of the vehicle (which is the motor vehicle 3 here) .
  • the memory 9 included in the display device 2 stores map data, and construction of VR can be performed by the processor 8 of the display device 2.
  • the display device further includes VR output means 10, on which the constructed VR is output.
  • the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle.
  • VR can be output with a perspective at an altitude different from the actual altitude of the vehicle, such that the display device 2 can display VR scenes at an altitude different from the actual altitude of the vehicle (which is the motor vehicle 3 here) .
  • the altitude can be adjusted by the display device 2.
  • the display device 2 can present virtual vehicles different from said vehicle (which is the motor vehicle 3 here) , such as planes, rockets, flying carpets or airships.
  • the map data can be provided by the display device 2.
  • the map data can be provided by the vehicle via the interface 5 of the display device 2, and/or the map data can be downloaded from a back-end server, in particular to the memory 9 of the display device 2.
  • the display device 2 can also acquire and display information about current route, course or surrounding environment.
  • the information about current route, course or surrounding environment can be acquired from the vehicle (which is the motor vehicle 3) via the interface 5 of the display device 2.
  • all actual movements of the vehicle can be reproduced. All the actual movements of the vehicle can be detected by the sensor of the vehicle (which is the motor vehicle 3 here) and transmitted to the display device 2 via an interface 6 at the vehicle side and the interface 5 of the display device 2. Hence, all the actual movements of the vehicle can be reproduced faithfully by the processor 8 in the output VR.
  • the actual movements are, for example, speed, acceleration and deceleration, jolts up and down, deflections left and right, etc.
  • those in the vehicle can have a feeling directly connected with the motion, and on the other hand, the visual pictures received by those in the vehicle when viewing the VR coordinate with the actually felt motion, so that no motion sickness would occur.
  • hazard situations and navigation instructions can be displayed by fusing augmented reality with the virtual reality.
  • the sensor 7 can detect the dangerous condition, and in particular collect the position and picture of the pedestrian.
  • the position and picture of the pedestrian are transmitted to the display device 2 via interfaces 5 and 6, where the dangerous condition is displayed by fusing augmented reality with the virtual reality.
  • the picture of the pedestrian that has been detected is superimposed on the VR picture for highlighted display.
  • navigation instructions may be displayed in the VR. For example, the text “to turn right 100 m ahead” is displayed in the picture.
  • Fig. 4 is a block diagram illustrating another example of a hierarchy 4 for displaying VR in a vehicle according to the present invention. This example is partially identical with the example according to Fig. 3, so the above descriptions shall be consulted, and only differences therebetween are described here.
  • display device 2 comprises, in addition to the processor 8, the memory 9 and the VR output means 10, a detector 11.
  • the detector 11 can detect the motion status of the head of the person wearing the display device.
  • all the movements of the head of the person can be reproduced in the VR by the display device.
  • the motion of the human body, especially the head can be very accurately reflected in the VR, so that the visual signals received by the brain is consistent with the somatosensory signals, thereby avoiding as far as possible the occurrence of motion sickness.
  • Fig. 4 also shows a manipulation device 12 connected to the display device 2.
  • the manipulation device may be a data glove or an operation handle provided on the display device 2, or a device for tracking the eyeball movement of the viewer.
  • moving or zooming operations can be performed in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle (which is the motor vehicle 3 here) for influencing the route, POI or other information associated with the travelling.

Abstract

The present invention relates to a method for displaying VR in a vehicle, the method comprising: acquiring information about the position, orientation and movement of the vehicle； constructing VR based on map data according to the position, orientation and movement of the vehicle； and outputting the VR on a VR output means. The present invention further relates to a system for displaying virtual reality in a vehicle, a display device for displaying virtual reality in a vehicle, and a computer program product.

Description

METHOD, SYSTEM, DISPLAY DEVICE FOR DISPLAYING VIRTUAL REALITY IN VEHICLE, AND COMPUTER PROGRAM PRODUCT TECHNICAL FIELD
The present invention relates to a method for displaying virtual reality in a vehicle, a system for displaying virtual reality in the vehicle, a display device for displaying virtual reality in a vehicle, and a computer program product.
BACKGROUND ART
At present, Virtual Reality ( “VR” ) has been applied in a plurality of fields, including, for example, entertainment, real estate development, education and training, industrial stimulation, and the like. However, up till now, VR devices have been applied in vehicles only very limitedly.
Known is a virtual driving system, in which a VR device is combined with a vehicle stimulator to stimulate the traffic on the vehicle stimulator in order to help training the operation skills of vehicle drivers.
CONTENTS OF THE INVENTION
The task of the present invention lies in providing immersive experience for a person in a vehicle.
According to an aspect of the present invention, described is a method for displaying VR in a vehicle, the method comprising: acquiring information about the position, orientation and movement of the vehicle； constructing VR based on map data according to the position, orientation and movement of the vehicle； and outputting the VR on VR output means.
In the present invention, vehicles include land vehicles, aircrafts, or  watercrafts, wherein land vehicles particularly refer to automotive vehicles, rail cars, motors, etc., aircrafts particularly refer to airplanes, airships, etc., and watercrafts particularly refer to ships, yachts, etc.
In the present invention, the VR output means may be a VR head-mounted display, such as VR glasses, VR helmets, or a stereoscopic display system including a stereoscopic image display and 3D glasses, or a cave type stereoscopic display system (CAVE system) using the surface of the vehicle as the projection surface, and alternatively, it may be a naked eye stereoscopic display system. However, the present invention is not limited to the VR output means as listed above, and it is self-evident that other VR output means may be also included.
In the present invention, the VR is preferably rendered very realistic. However, the VR may also be rendered to be relatively abstractive such that the viewer would pay more attention to the most important information.
According to one embodiment of the present invention, the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for the person in the vehicle. Hence, a three-dimensional dynamic visual scene with respect to the exterior environment of the vehicle can be provided for the person in the vehicle, wherein the three-dimensional dynamic visual scene can move along with movement of the vehicle in the real world. Hence, the VR according to the present invention firstly visualizes map data, and such VR is also fused with the actual movement of the vehicle in which a person is located. In this case, the person in the vehicle can, by means of the VR, in particular through the casing of the vehicle, without any visual blocking, the exterior environment of the vehicle. Therefore, the present invention can provide immersive experience for the person in the vehicle with the movement of the vehicle.
According to one embodiment of the present invention, the VR can be  output with a perspective at an altitude different from the actual altitude of the vehicle. By such an embodiment, for example, those in a land vehicle can view the exterior environment of the vehicle with an eagle perspective to experience like floating in the air.
Hence, those viewing the VR with an eagle perspective in the dynamic three-dimensional map view are located above the actual position of the vehicle. In other words, in a two-dimensional plane substantially parallel to the ground surface, the position of the VR viewed by the person is equivalent to a current car position (CCP) in a traditional navigation system. However, in a vertical direction perpendicular to the ground surface, the VR viewed by the person is based on an altitude different from the actual position of the vehicle.
In case the vehicle is a car, where the VR is output with a perspective at an altitude different from the actual altitude of the vehicle, it is possible:
-to look for the best route；
-to explore surrounding environment/sightseeing, such as to see which building is over there etc.；
-to look for POIs like restaurants or parking etc.；
-to find the possibility for parking；
-to assist the driver to pass through a complex environment or to park in a complex environment；
-to experience a new feeling of car riding, like flying in the air with birds.
According to one embodiment of the present invention, the altitude at which the perspective is can be adjusted. In this case, the person in the vehicle can adjust his/her perspective to obtain a better eyeshot.
For example, where the vehicle is a land vehicle such as a car, the person can, for example, adjust upwardly the altitude so as to view the map content at a more distant place from a higher altitude, or to view beyond high  buildings so as to avoid blocking of the eyeshot by the buildings. This is especially advantageous for looking for a geographic location by VR. In case the vehicle is an underground rail car, passengers can view the landscapes on the ground by the VR output means. In addition, in case the altitude is configured to be extremely high, such as hundreds of meters, a broad range of region where the vehicle is located can be viewed from an overhead viewpoint. Moreover, by means of the adjustable altitude, the entertainment of the viewing of the VR can be also enhanced.
Additionally, in case the vehicle is an aircraft such a an airplane, the person can, for example, downwardly adjust the altitude so as to view the VR scene on the ground surface where the aircraft sweeps over a place more close to the ground surface. Thus, a novel way of entertainment is provided for those taking the aircraft.
According to one embodiment of the present invention, in VR, virtual vehicles different from said vehicle can be presented, such as planes, rockets, flying carpets or airships. Hence, those in the vehicle can select the functions of the vehicles displayed in the VR, wherein the candidate virtual vehicles include, for example, planes, rockets, flying carpets or airships. For example, in the VR, the cabin or compartment of the plane, rocket or airship is additionally displayed such that the person viewing the VR feels like being in the plane, rocket or airship. In the present invention, the flying carpet is also called a magic carpet, which is an imaginary carpet that can carry people to fly in the air, and it is a vehicle in many mythologies, legends, fictions, cartoons or other works. According to the present invention, in the VR, the carpet located below is additionally displayed such that the person viewing the VR feels like being on the flying carpet. According to this embodiment of the present invention, the person in the vehicle can experience a feeling of driving or taking another vehicle, thereby obtaining a novel and exciting  driving or taking experience.
A particularly preferred embodiment of the present invention is that in the VR, the person in the vehicle can select the function of flying carpet (or magic carpet) . By realizing the function of a flying carpet, those in the vehicle can experience floating at a high altitude in the current exterior environment of the vehicle. In this embodiment, it is especially advantageous that when the vehicle passes through a channel, an overhead bridge, a flyover or buildings connected in the air, the altitude can be automatically adjusted so that the viewer can avoid “collision” with such building structures in the VR. Hence, both comfortable and novel immersive experience can be provided.
According to one embodiment of the present invention, information about current route, course or surrounding environment can be displayed in the VR. The information is, for example, a starting point, an end point, a predetermined route planning, travel data, names of the streets and buildings in the surrounding environment about the current route. In this way, those in the vehicle can obviously feel continuously approaching of the end point of the route.
In an extremely preferred embodiment, the so-called Point of Interest ( “POI” ) can be displayed in the VR. The POI may relate to scenic spots, such as rivers, places of historic interests like castles, museums or monuments, and/or to infrastructures such as bridges, traffic lights, telephone booths, shops, parking lots, etc. The names of the corresponding POIs can be stored in the navigation database. By this embodiment, the viewer of the VR may view the scenic spots in the initial vehicle or rapidly look for the desired infrastructure. Especially, in combination with the previous embodiment, i.e., where the VR is output with a perspective at an altitude different from the actual altitude of the vehicle, those in the vehicle can scan the surrounding scenic spots at different altitudes； for example, he/she can view the trend of  the whole river, the trend of the mountain or the arrangement of the landscape, and can also view the details of the scenic spot or infrastructure locally.
In an extremely preferred embodiment, the information stored in the address book of the navigation system can be displayed in the VR. For example, the addresses of family members and friends are stored in the address book, and in the VR according to the present invention, such places are marked additionally. For example, a balloon symbol of “Anna's Home” is marked above the home of Anna. Hence, the driver of the vehicle can conveniently learn important places in the surrounding environment.
According to one embodiment of the present invention, in VR, all actual movements of the vehicle can be reproduced. The embodiments of the present invention are based on the following consideration: in the VR technique, a technical problem that cannot be ignored is that the VR user often has motion sickness. The main reason for the motion sickness lies in the incoordination between the pictures visually observed by the user and the real state of the body. For example, if the vehicle has alternative acceleration and deceleration and at the same time jolts up and down, while the user only sees stable and constant pictures by the VR, since the user is in motion, organs in charge of balance such as the inner ear deliver signals to the brain indicating the human body is in motion, such that the visual signals contradicts with the somatosensory signals in the brain, and for this, the brain produces a dizzy feeling to warn the user to escape from the current state, thereby causing motion sickness. In the present invention, in order to avoid such motion sickness, it is provided that all actual movements of the vehicle can be reproduced in the VR. For example, the speed, acceleration and deceleration, jolts up and down, deflection left and right and the like of the vehicle are all realistically reflected in the VR scenes presented to those in the vehicle. In this way, on the one hand, those in the vehicle can have a feeling directly  connected with the motion, and on the other hand, the visual pictures received by those in the vehicle when viewing the VR coordinates with the actually felt motion, so that no motion sickness would occur.
In an extremely advantageous embodiment, it is additionally provided in the method according to the present invention that the motion state of the head of the person wearing the display device is detected, and all the motions of the head of the person can be reproduced in the VR. By this embodiment, the motion of the human body, especially the head, can be very accurately reflected in the VR, so that the visual signals received by the brain is consistent with the somatosensory signals, thereby avoiding as far as possible the occurrence of motion sickness.
According to one embodiment of the present invention, moving or zooming operations can be performed in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling. In particular, when move or zoom operation is performed, the direction is maintained to be the same as in the real world such that the VR viewer would feel comfortable without occurring motion sickness.
By this embodiment, firstly, the interactivity between the VR and the viewer can be enhanced. The viewer can view any place in the VR. In this way, for example, it is possible to conveniently look for a target and see the environment in the proximity of the target. Moreover, such an embodiment also gives a possibility of manipulating the travelling in the VR. For example, those in the vehicle can set the destination, add POIs, change the travelling route, and change the travelling mode (such as energy-saving type, sport type and comfortable type) . For such an embodiment, movement and zoom operations as well as manipulation of the information may be performed by a  data glove or a manipulation handle provided with the display device, or by tracking motion of the eyeball of the viewer.
According to one embodiment of the present invention, by fusing augmented reality with the virtual reality, hazard situations and navigation instructions can be displayed in the VR. In this case, the content of augmented reality (AR) is additionally displayed in the VR. For example, actual hazard situations, such as pedestrians, may be highlighted on the VR images, or navigation instructions may be superimposed on the VR pictures. By doing this, the contents on which attention shall be paid can be emphasized for those in the vehicle so as to increase the degree of their concern on such contents.
According to one embodiment of the present invention, the VR can be output on the VR output means with perspectives from different positions in the vehicle. Hence, those sit at the rear part of a vehicle can see the scenes that can be seen from the front part of the vehicle.
In one embodiment of the present invention, in the VR, other users can be displayed and communications with the other users can be performed. Hence, the other users switching on the function of displaying the VR are visible to one another simultaneously in the same VR world and can interact with one another, such as sending messages to one another. As used herein, the other users may relate to persons located within the same vehicle, persons located in other vehicles and persons located at fixed positions. Under the condition that the other users are persons located within the same vehicle, persons taking the same vehicle can be seen in the VR such that the persons viewing the VR would not feel isolated, thereby enhancing conformity to the real scene. Under the condition that the other users are persons located in other vehicles or persons located at fixed positions, one the one hand, their positions are visible to each other. When one goes to a location of another user by taking a vehicle, this facilitates setting of the destination and selection  of the route to approach the destination, and at the same time, visibility of other continuously approaching users is provided for the persons in the vehicle. If the other user is at the same time located in a respective vehicle, said embodiment avoids missing each other or going to different destinations. Meanwhile, those in the same VR world may send messages to each other to increase interactivity in travel. This increases the entertainment and interactivity for taking vehicles. Here, said embodiment of the present invention may be based on communication techniques like V2V and V2X, and a sequence of traffic information including real-time traffic, road information, pedestrian information and the like can be obtained, thereby enhancing the driving security, relieving the traffic jam, increasing the traffic efficiency and providing on-board entertainment information.
According to the present invention, the vehicle can be positioned by using the position service of the Global Positioning System (GPS) , the GLONASS, the Galileo, the COMPASS, a laser radar or the position service of a wireless communication network of a telecommunication mobile operator.
The orientation according to the present invention may be a space orientation of the vehicle, i.e., the orientation includes not only a two-dimensional position orientation directed by the head part of the vehicle but also a pitch of the vehicle with respect to the horizontal plane. The orientation can be determined by, for example, a direction sensor.
According to the present invention, the movement of the vehicle is detected by a speed sensor, a gyroscope, an acceleration sensor, a steering angle sensor, or the like.
According to one embodiment of the present invention, information about the movement of the vehicle includes acceleration, speed, and deflection. According to a preferred embodiment of the present invention, the  acceleration, speed and deflection can be obtained by the controller, the speed sensor, the acceleration sensor, the steering angle sensor, the gyroscope or the like of the vehicle per se.
According to another aspect of the present invention, described is a system for displaying VR in a vehicle, comprising:
-a receiving unit for acquiring information about the position, orientation and movement of the vehicle；
-a construction unit for constructing VR based on map data according to the position, orientation and movement of the vehicle；
-an output unit for outputting the VR to VR output means.
According to the present invention, the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle.
According to one embodiment of the present invention, the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
According to one embodiment of the present invention, the system for displaying VR in a vehicle further comprises a perspective adjusting unit for adjusting the altitude at which the perspective is such that the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle.
According to a further aspect of the present invention, described is a display device for displaying VR in a vehicle, comprising:
VR output means；
a memory configured to store computer executable instructions； and
a processor configured to execute the computer executable instructions, wherein when the computer executable instructions are executed by the processor, the display device is enable to:
-acquire information about the position, orientation and movement of the vehicle；
-construct VR based on map data according to the position, orientation and movement of the vehicle；
-output the VR on the VR output means.
According to one embodiment of the present invention, the map data can be provided by the vehicle via a interface of the display device, and/or the map data can be provided by the memory of the display device, and/or the map data can be downloaded from a back-end server. According to the embodiment of the present invention, the map data can be obtained from a plurality of sources. The map data according to the present invention may relate to real scene three-dimensional maps or stimulated three-dimensional maps. The map data can be stored or downloaded to the vehicle and/or the memory of the display device in the form of a map database. If the map data are only stored in the vehicle, the desired map data is transmitted in real time or in advance to the display device via the interface of the display device. As used herein, transmission of data, for example, can relate to wired connection (such as vehicle bus, like Local Interconnect Network (LIN) bus, Controller Area Network (CAN) bus or FlexRay bus) or to wireless connection. The wireless connection may be a WLAN for example, in accordance with IEEE802.11a, IEEE802.11b, IEEE802.11ac, IEEE802.11ad, IEEE802.11g, IEEE802.11h, or IEEE802.11n or Bluetooth, in particular Bluetooth V2.0, Bluetooth V3.0 or Bluetooth V4.0.
According to one embodiment of the present invention, the display device comprises an detector configured to detect the motion state of the head of the person wearing the display device, and to reproduce all the motions of the head of the person in the VR by the display device. By this embodiment, the motion of the human body, especially the head, can be very accurately  reflected in the VR, so that the visual signals received is consistent with the somatosensory signals by the brain, thereby avoiding as far as possible the occurrence of motion sickness.
According to one embodiment of the present invention, when the computer executable instructions are executed by the processor, the display device is enabled to execute the methods as described in the various embodiments of the present invention. In other words, the features in the method according to the present invention can be applied to the display device according to the present invention.
In one embodiment of the present invention, the display device is further equipped with or connected to a data glove, an operation handle or means for tracking the eyeball movement of the viewer, such that the user may perform moving or zooming operations in the VR in order to observe specific positions in the VR, and/or information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling. By such an embodiment, for example, it is possible to conveniently look for the target and see the environment in the proximity of the target, and there is also a possibility of manipulating the travelling in the VR.
According to a further aspect of the present invention, a computer program product is described, which can be loaded to an internal memory of a computer and comprises software code segments, when the product is executed on the computer, the methods according to various embodiments of the present application are implemented by using the software code segments.
An embodiment described according to any aspect of the present invention is also applicable to the other aspects of the present invention in a corresponding manner.
DESCRIPTION ON THE DRAWINGS
Examples of the present invention are schematically illustrated by means of the following figures. Among the drawings:
Fig. 1 is a schematic diagram illustrating an example according to the present invention.
Fig. 2 is a flow chart illustrating a method for displaying VR in a vehicle according to the present invention.
Fig. 3 is a block diagram illustrating one example of a hierarchy for displaying VR in a vehicle according to the present invention.
Fig. 4 is a block diagram illustrating another example of a hierarchy for displaying VR in a vehicle according to the present invention.
EMBODIMENTS
Fig. 1 illustrates an example of displaying VR in a vehicle according to the present invention. In this example, the vehicle relates to a motor vehicle 3. Persons 1 in the motor vehicle wear a display device 2 for display VR, respectively. Here, the display device 2 is a VR head-mounted display.
In Fig. 1, the persons wearing the display device in the motor vehicle usually include a person taking the motor vehicle, such as a person on the co-pilot seat, or a person on a rear seat. Under the condition that an automatic driving mode is set for the motor vehicle, the persons wearing the display device in the motor vehicle also include the driver on the pilot seat. In this case, the motor vehicle can be automatically driven according to parameters including the destination, the course, etc. preset by the driver, and the travel substantially requires no participation of the driver. In the automatic driving mode, if the motor vehicle identifies that it is necessary to handover the motor vehicle to the driver, the driver may be notified to take off the display device to manipulate the motor vehicle.
Fig. 2 is a flow chart illustrating a method for displaying VR in a vehicle according to the present invention. According to the present invention, firstly, information about the position, orientation and movement of the vehicle is acquired, and VR is constructed according to the position, orientation and movement of the vehicle and based on map data so as to output the VR on the VR output means. Hence, immersive experience can be provided for those in the vehicle with the movement of the vehicle.
Here, construction of the VR may be performed on a processor integrated in the display device, or in data processing means disposed in the vehicle. In the latter case, the constructed VR can be transmitted to the display device via an interface exchanging data with the vehicle so as to output the VR on the VR output means.
According to the present invention, the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle. Moreover, the VR can be output with a perspective at an altitude different from the actual altitude of the vehicle. Hence, the person wearing the display device can see VR scenes at an altitude different from the actual position of the vehicle. In particular, the person can view the exterior environment of the motor vehicle with an eagle perspective. The VR scenes seen by the person are just like scenes that can be observed at a position above the vehicle, particularly the motor vehicle, such that the person can experience like floating in the air.
In addition, the altitude at which the perspective is can be adjusted, in particular by the display device. Under the condition that the vehicle is a motor vehicle, the person in the motor vehicle, for example, can see VR scenes 10 m, 30 m, 50 m, 100 m, 200 m and the like above the motor vehicle. In addition, map contents at a more distant place can be viewed at a higher altitude, and at the same time, can be viewed beyond high buildings so as to  avoid blocking of the eyeshot by the buildings. Moreover, since the altitude is adjustable, the entertainment of viewing of the VR can be enhanced.
In particular, in VR, virtual vehicles different from the current vehicle can be presented, such as planes, rockets, flying carpets or airships. Hence, the person in the vehicle can experience a feeling of driving another vehicle, thereby obtaining a novel and exciting driving experience. Here, a particularly preferred virtual vehicle is a flying carpet or magic carpet, since a vehicle like a flying carpet has not yet been developed, and in practice, people are not yet able to have an experience of sitting on a flying carpet.
In addition, information about current route, course or surrounding environment can be displayed in the VR. For example, in the constructed VR, the route to be traveled can be highlighted. For example, the ground to be passed along the planned route can be colored with green, and a flag label can be added to the position of destination. Additionally, information about the current course can be displayed, including, for example, remaining mileages, expected remaining time, expected arrival time, etc. In addition, names of the buildings around the current position can be displayed. In particular, identifiers or descriptions may be added to the POIs； for example, balloon identifiers can be added above shops, parking lots, and scenery spots.
Fig. 3 is a block diagram illustrating one example of a hierarchy 4 for displaying VR in a vehicle according to the present invention, wherein the hierarchy 4 according to the present invention comprises a display device for displaying VR in the vehicle and a vehicle.
Here, motor vehicle 3 as shown in Fig. 1 is taken as an example of the vehicle. The motor vehicle 3 includes a sensor 7, configured to detect information about the position, orientation and movement of the vehicle. The motor vehicle 3 is further provided with an interface 6 for exchanging data with the display device 2.
In addition, Fig. 3 is exemplarily illustrates a display device 2 according to the present invention which is used for displaying VR in the vehicle. The display device 2 includes VR output means 10； a memory 9 configured to store computer executable instructions； and a processor 8 configured to execute the computer executable instructions, wherein when the computer executable instructions are executed by the processor, the display device is enable to:
-acquire information about the position, orientation and movement of the vehicle；
-construct VR based on map data according to the position, orientation and movement of the vehicle；
-output the VR on the VR output means.
The display device may further include an interface 5 exchanging data with the vehicle (which is the motor vehicle 3 here) , through which information about the position, orientation and movement of the vehicle can be acquired. Hence, the processor may construct VR based on map data according to the position, orientation and movement of the vehicle (which is the motor vehicle 3 here) . Hence, according to the current example of the present invention, the memory 9 included in the display device 2 stores map data, and construction of VR can be performed by the processor 8 of the display device 2. In addition, the display device further includes VR output means 10, on which the constructed VR is output.
According to the present invention, the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for those in the vehicle.
Preferably, VR can be output with a perspective at an altitude different from the actual altitude of the vehicle, such that the display device 2 can display VR scenes at an altitude different from the actual altitude of the  vehicle (which is the motor vehicle 3 here) . Here, the altitude can be adjusted by the display device 2. In particular, the display device 2 can present virtual vehicles different from said vehicle (which is the motor vehicle 3 here) , such as planes, rockets, flying carpets or airships.
As mentioned above, in the current example, the map data can be provided by the display device 2. According to other embodiments of the present invention, the map data can be provided by the vehicle via the interface 5 of the display device 2, and/or the map data can be downloaded from a back-end server, in particular to the memory 9 of the display device 2.
The display device 2 according to the present invention can also acquire and display information about current route, course or surrounding environment. The information about current route, course or surrounding environment can be acquired from the vehicle (which is the motor vehicle 3) via the interface 5 of the display device 2.
According to the present invention, in VR, all actual movements of the vehicle can be reproduced. All the actual movements of the vehicle can be detected by the sensor of the vehicle (which is the motor vehicle 3 here) and transmitted to the display device 2 via an interface 6 at the vehicle side and the interface 5 of the display device 2. Hence, all the actual movements of the vehicle can be reproduced faithfully by the processor 8 in the output VR. As used herein, the actual movements are, for example, speed, acceleration and deceleration, jolts up and down, deflections left and right, etc. In this way, on the one hand, those in the vehicle can have a feeling directly connected with the motion, and on the other hand, the visual pictures received by those in the vehicle when viewing the VR coordinate with the actually felt motion, so that no motion sickness would occur.
In addition, by means of the display device 2, hazard situations and navigation instructions can be displayed by fusing augmented reality with the  virtual reality. For example, under a dangerous condition that a pedestrian appears suddenly in front of the vehicle (which is the motor vehicle 3 here) , the sensor 7 can detect the dangerous condition, and in particular collect the position and picture of the pedestrian. The position and picture of the pedestrian are transmitted to the display device 2 via  interfaces  5 and 6, where the dangerous condition is displayed by fusing augmented reality with the virtual reality. Preferably, the picture of the pedestrian that has been detected is superimposed on the VR picture for highlighted display. Similarly, navigation instructions may be displayed in the VR. For example, the text “to turn right 100 m ahead” is displayed in the picture.
Fig. 4 is a block diagram illustrating another example of a hierarchy 4 for displaying VR in a vehicle according to the present invention. This example is partially identical with the example according to Fig. 3, so the above descriptions shall be consulted, and only differences therebetween are described here.
As illustrated in Fig. 4, display device 2 comprises, in addition to the processor 8, the memory 9 and the VR output means 10, a detector 11. The detector 11 can detect the motion status of the head of the person wearing the display device. Thus, all the movements of the head of the person can be reproduced in the VR by the display device. Hence, the motion of the human body, especially the head, can be very accurately reflected in the VR, so that the visual signals received by the brain is consistent with the somatosensory signals, thereby avoiding as far as possible the occurrence of motion sickness.
Fig. 4 also shows a manipulation device 12 connected to the display device 2. The manipulation device may be a data glove or an operation handle provided on the display device 2, or a device for tracking the eyeball movement of the viewer. In this way, moving or zooming operations can be performed in the VR in order to observe specific positions in the VR, and/or  information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle (which is the motor vehicle 3 here) for influencing the route, POI or other information associated with the travelling.
The present invention is not limited to the illustrated examples, but instead, it includes or extends to technical equivalents falling into the valid scope of the attached claims. For the description to the selected positions in the specification, such as up and down, please refer to the direct recitations and illustrated figures, and in case that the position changes, it can be transferred to a new position according to the meaning.

Claims (21)

  1. A method for displaying virtual reality in a vehicle, the method comprising:
    - acquiring information about the position, orientation and movement of the vehicle；
    - constructing VR based on map data according to the position, orientation and movement of the vehicle；
    - outputting the VR on VR output means.
  2. The method according to claim 1, characterized in,
    the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for the person in the vehicle.
  3. The method according to claim 1 or 2, characterized in,
    the VR is output with a perspective at an altitude different from the actual altitude of the vehicle.
  4. The method according to claim 3, characterized in,
    the altitude at which the perspective is can be adjusted.
  5. The method according to one of the preceding claims, characterized in,
    in VR, virtual vehicles different from said vehicle can be presented, such as planes, rockets, flying carpets or airships.
  6. The method according to one of the preceding claims, characterized in,
    information about current route, course or surrounding environment can be displayed in the VR.
  7. The method according to one of the preceding claims, characterized in,
    in VR, all actual movements of the vehicle can be reproduced.
  8. The method according to one of the preceding claims, characterized in,
    moving or zooming operations can be performed in the VR so as to facilitate observation of specific positions in the VR, and/or
    information can be obtained and manipulated in the VR, and the information can be transmitted to the vehicle for influencing the route, POI or other information associated with the travelling.
  9. The method according to one of the preceding claims, characterized in,
    by fusing augmented reality with the virtual reality, hazard situations and navigation instructions can be displayed in the VR.
  10. The method according to one of the preceding claims, characterized in,
    the VR is output on the VR output means with perspectives from different positions in the vehicle.
  11. The method according to one of the preceding claims, characterized in,
    in the VR, other users can be displayed and communications with the  other users can be performed.
  12. The method according to one of the preceding claims, characterized in,
    the information about the movement of the vehicle includes acceleration, speed, and deflection.
  13. A system for displaying virtual reality (VR) in a vehicle, comprising:
    - a receiving unit for acquiring information about the position, orientation and movement of the vehicle；
    - a construction unit for constructing VR based on map data according to the position, orientation and movement of the vehicle；
    - an output unit for outputting the VR to VR output means.
  14. The system according to claim 13, characterized in,
    the VR includes a three-dimensional dynamic visual scene of an exterior environment of the vehicle for the person in the vehicle.
  15. The system according to claim 13 or 14, characterized in,
    the VR is output with a perspective at an altitude different from the actual altitude of the vehicle.
  16. The system according to claim 15, characterized in, the system comprises
    a perspective adjusting unit for adjusting the altitude at which the perspective is such that the VR is output with a perspective at an altitude different from the actual altitude of the vehicle.
  17. A display device for displaying VR in a vehicle, the display device comprising:
    VR output means；
    a memory configured to store computer executable instructions； and
    a processor configured to execute the computer executable instructions, wherein when the computer executable instructions are executed by the processor, the display device is enable to:
    - acquire information about the position, orientation and movement of the vehicle；
    - construct VR based on map data according to the position, orientation and movement of the vehicle；
    - output the VR on the VR output means.
  18. The display device according to claim 17, characterized in,
    the map data can be provided by the vehicle via a interface of the display device, and/or the map data can be provided by the memory of the display device, and/or the map data can be downloaded from a back-end server.
  19. The display device according to claim 17 or 18, characterized in,
    the display device comprises a detector configured to detect the motion state of the head of the person wearing the display device, and to reproduce all the motions of the head of the person in the VR by the display device.
  20. The display device according to one of claims 17 to 19, characterized in, when the computer executable instructions are executed by the processor, the display device is enabled to execute the method according to one of claims 2 to 12.
  21. A computer program product, which can be loaded to an internal memory of a computer and comprises software code segments, wherein when the product is executed on the computer, the method according to one of claims 1 to 12 is implemented by using the software code segments.
PCT/CN2016/100542 2016-09-28 2016-09-28 Method, system, display device for displaying virtual reality in vehicle, and computer program product WO2018058361A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/100542 WO2018058361A1 (en) 2016-09-28 2016-09-28 Method, system, display device for displaying virtual reality in vehicle, and computer program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/100542 WO2018058361A1 (en) 2016-09-28 2016-09-28 Method, system, display device for displaying virtual reality in vehicle, and computer program product

Publications (1)

Publication Number Publication Date
WO2018058361A1 true WO2018058361A1 (en) 2018-04-05

Family

ID=61763548

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/100542 WO2018058361A1 (en) 2016-09-28 2016-09-28 Method, system, display device for displaying virtual reality in vehicle, and computer program product

Country Status (1)

Country Link
WO (1) WO2018058361A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112041214A (en) * 2018-04-30 2020-12-04 奥迪股份公司 Method and control device for operating a motor vehicle
CN112669671A (en) * 2020-12-28 2021-04-16 北京航空航天大学江西研究院 Mixed reality flight simulation system based on physical interaction
CN113448322A (en) * 2020-03-26 2021-09-28 宝马股份公司 Remote operation method and system for vehicle, storage medium, and electronic device
WO2021253995A1 (en) * 2020-06-17 2021-12-23 北京嘀嘀无限科技发展有限公司 Method and system for providing real-scene image for user
CN115988247A (en) * 2022-12-08 2023-04-18 小象智能(深圳)有限公司 XR vehicle-mounted film watching system and method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150097861A1 (en) * 2013-10-03 2015-04-09 Honda Motor Co., Ltd. System and method for dynamic in-vehicle virtual reality
CA2932241A1 (en) * 2014-08-11 2016-02-18 Mack Rides Gmbh & Co. Kg Method for operating a device, in particular an amusement ride, transport means, a fitness apparatus or the like
CN105913755A (en) * 2016-05-27 2016-08-31 大连楼兰科技股份有限公司 Internet-of-vehicles virtual reality building model display system and Internet-of-vehicles virtual reality building model display method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150097861A1 (en) * 2013-10-03 2015-04-09 Honda Motor Co., Ltd. System and method for dynamic in-vehicle virtual reality
CA2932241A1 (en) * 2014-08-11 2016-02-18 Mack Rides Gmbh & Co. Kg Method for operating a device, in particular an amusement ride, transport means, a fitness apparatus or the like
CN105913755A (en) * 2016-05-27 2016-08-31 大连楼兰科技股份有限公司 Internet-of-vehicles virtual reality building model display system and Internet-of-vehicles virtual reality building model display method

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112041214A (en) * 2018-04-30 2020-12-04 奥迪股份公司 Method and control device for operating a motor vehicle
CN113448322A (en) * 2020-03-26 2021-09-28 宝马股份公司 Remote operation method and system for vehicle, storage medium, and electronic device
WO2021253995A1 (en) * 2020-06-17 2021-12-23 北京嘀嘀无限科技发展有限公司 Method and system for providing real-scene image for user
CN112669671A (en) * 2020-12-28 2021-04-16 北京航空航天大学江西研究院 Mixed reality flight simulation system based on physical interaction
CN112669671B (en) * 2020-12-28 2022-10-25 北京航空航天大学江西研究院 Mixed reality flight simulation system based on physical interaction
CN115988247A (en) * 2022-12-08 2023-04-18 小象智能(深圳)有限公司 XR vehicle-mounted film watching system and method
CN115988247B (en) * 2022-12-08 2023-10-20 小象智能(深圳)有限公司 XR vehicle-mounted video watching system and method

Similar Documents

Publication Publication Date Title
WO2018058361A1 (en) Method, system, display device for displaying virtual reality in vehicle, and computer program product
US10546560B2 (en) Systems and methods for presenting virtual content in a vehicle
JP3700021B2 (en) Electro-optic vision system using position and orientation
CN104512336B (en) 3 dimension navigation
CN100403340C (en) Image generating apparatus, image generating method, and computer program
JP6026088B2 (en) Remote control system
US7135993B2 (en) On-vehicle information provision apparatus
CN109644256B (en) Vehicle-mounted video system
US9482540B2 (en) Navigation display method and system
US20130083061A1 (en) Front- and rear- seat augmented reality vehicle game system to entertain & educate passengers
CN113302621A (en) Using passenger attention data captured in a vehicle for positioning and location-based services
JP2015068831A (en) Function-extended three-dimensional (3d) navigation
US20210019942A1 (en) Gradual transitioning between two-dimensional and three-dimensional augmented reality images
US9163944B2 (en) System and method for displaying three dimensional views of points of interest
EP3147630A1 (en) 3d helicopter view at destination
CN111480194A (en) Information processing device, information processing method, program, display system, and moving object
US20220080826A1 (en) Information processing apparatus, information processing method, and mobile body apparatus
CN109990797A (en) A kind of control method of the augmented reality navigation display for HUD
CN102235869A (en) Method and information system for marking automobile destination
WO2016102304A1 (en) Method for presenting an image overlay element in an image with 3d information, driver assistance system and motor vehicle
WO2014171240A1 (en) System method and program for site display
US20210174573A1 (en) Image processing apparatus, display system, computer readable recoring medium, and image processing method
US6907324B2 (en) Instrument reference flight display system for horizon representation of direction to next waypoint
US20160362190A1 (en) Synthetic vision
Ikeda et al. Diminished reality system based on open-source software for self-driving mobility

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16917116

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16917116

Country of ref document: EP

Kind code of ref document: A1