CN114428577A - Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system - Google Patents
Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system Download PDFInfo
- Publication number
- CN114428577A CN114428577A CN202111665439.2A CN202111665439A CN114428577A CN 114428577 A CN114428577 A CN 114428577A CN 202111665439 A CN202111665439 A CN 202111665439A CN 114428577 A CN114428577 A CN 114428577A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- interaction
- interface
- interactive
- content server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 135
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000002452 interceptive effect Effects 0.000 claims abstract description 71
- 238000004891 communication Methods 0.000 claims abstract description 29
- 230000004044 response Effects 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 31
- 230000006870 function Effects 0.000 description 9
- 239000011521 glass Substances 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a vehicle-mounted interaction method, a vehicle-mounted interaction terminal and a vehicle-mounted system. The vehicle-mounted interaction method comprises the following steps: responding to the communication connection established with the VR head-mounted display, and sending an interaction request to an external VR content server to acquire a VR interface for interaction; in response to receiving the VR interface returned by the VR content server, outputting the VR interface to the VR head mounted display for display of the VR interface on the VR head mounted display. The invention also provides a vehicle-mounted interactive terminal and a vehicle-mounted system, and based on the invention, the possibility of using the VR head-mounted display in the vehicle-mounted environment is provided, the application scene of virtual reality interaction is expanded, and a user can conveniently access the meta space anytime and anywhere.
Description
Technical Field
The invention relates to the field of vehicles, in particular to a vehicle-mounted interaction method, a vehicle-mounted interaction terminal and a vehicle-mounted system for realizing virtual reality interaction in the field of vehicles.
Background
With the development of technologies such as 5G, big data, AI, etc., the combination of virtual and reality is an unsettlable trend, and the new form element universe considered as the next generation internet is also pushed over the wind gap. According to the definition given by robox, the metasystem contains eight major elements: identity, friends, immersion, low latency, diversification, anytime and anywhere, economic systems, and civilization. Virtual Reality (VR) is an indispensable part of the metas, and the current mainstream is to use the CPU (central processing unit) and the graphics card (image processor) of a PC for operation by using a VR head-mounted display + a PC (i.e., a computer terminal), thereby achieving an extremely good display effect. However, this method requires a high-end PC, and the PC is fixed and can only be used in some fixed scenes.
With the development of economy, especially the development of the automobile industry, the number of automobiles in the society is increasing nowadays, and with the improvement of living standard of people, the automobiles are very popular as transportation tools, and the configuration requirements of consumers for vehicles when purchasing the automobiles are higher and higher, especially under the condition that the vehicles are developed to be more intelligent and networked, the vehicles are generally provided with high-performance processors, especially high-performance Graphic Processors (GPUs), and the vehicles are also provided with high-speed communication networks.
In view of the above, it is desirable to provide a vehicle-mounted interaction method, a vehicle-mounted interaction terminal, and a vehicle-mounted system, which can extend VR by using hardware of a vehicle, thereby extending a use scenario of virtual reality interaction and facilitating a user to access the meta space anytime and anywhere.
Disclosure of Invention
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
As described above, in order to solve the problems that in the prior art, the implementation of the VR function needs to rely on a fixed high-end computer terminal, cannot provide diversified use scenes, and cannot meet the requirement of accessing the meta universe of users anytime and anywhere, the invention provides a vehicle-mounted interaction method, a vehicle-mounted interaction terminal, and a vehicle-mounted system.
One aspect of the invention provides a vehicle-mounted interaction method, which is applied to a vehicle-mounted system of a vehicle, and comprises the following steps:
responding to the communication connection established with the VR head-mounted display, and sending an interaction request to an external VR content server to acquire a VR interface for interaction;
in response to receiving the VR interface returned by the VR content server, outputting the VR interface to the VR head mounted display for display of the VR interface on the VR head mounted display.
In an embodiment of the vehicle-mounted interaction method, optionally, the vehicle-mounted interaction method further includes:
sending a real-time 3D on-board scene corresponding to the VR head-mounted display current location to the VR content server to cause the VR content server to generate the VR interface corresponding to a current on-board environment based on the real-time 3D on-board scene.
In an embodiment of the vehicle-mounted interaction method, optionally, the vehicle-mounted interaction method further includes:
acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point;
and adjusting a preset reference 3D vehicle-mounted scene based on the relative position coordinates to acquire the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to the preset reference point.
In an embodiment of the vehicle-mounted interaction method, optionally, the vehicle-mounted interaction method further includes:
and outputting the interaction data acquired from the in-vehicle interaction device to the VR content server so that the VR content server responds to the interaction data and updates the returned VR interface.
In an embodiment of the foregoing vehicle-mounted interaction method, optionally, the in-vehicle interaction device includes an in-vehicle camera device; wherein
And responding to the user image collected by the in-vehicle camera device, and identifying the user image to acquire the user limb data in the interactive data.
In an embodiment of the foregoing vehicle-mounted interaction method, optionally, the in-vehicle interaction apparatus includes an in-vehicle interaction input device; wherein
The interactive data comprises information data input by a user based on the in-vehicle interactive input device.
Another aspect of the present invention further provides a vehicle-mounted interactive terminal, where the vehicle-mounted interactive terminal includes:
a memory and a communication module; and
the processor is connected with the memory and the communication module; wherein
The processor is configured to:
responding to the communication connection established with the VR head-mounted display through the communication module, and sending an interaction request to an external VR content server through the communication module to acquire a VR interface for interaction;
in response to receiving the VR interface returned by the VR content server, outputting the VR interface to the VR head mounted display for display of the VR interface on the VR head mounted display.
In an embodiment of the foregoing vehicle-mounted interactive terminal, optionally, the processor is further configured to:
sending a real-time 3D in-vehicle scene corresponding to the VR head-mounted display current location to the VR content server to cause the VR content server to generate the VR interface corresponding to a current in-vehicle environment based on the real-time 3D in-vehicle scene.
In an embodiment of the foregoing vehicle-mounted interactive terminal, optionally, the processor is further configured to:
acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point;
and adjusting a preset reference 3D vehicle-mounted scene based on the relative position coordinates to acquire the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to the preset reference point.
In an embodiment of the foregoing vehicle-mounted interactive terminal, optionally, the processor is further configured to:
and outputting the interaction data acquired from the in-vehicle interaction device to the VR content server through the communication module so that the VR content server responds to the interaction data and updates the returned VR interface.
In an embodiment of the foregoing vehicle-mounted interactive terminal, optionally, the in-vehicle interactive device includes an in-vehicle camera device; wherein
And responding to the user image acquired by the in-vehicle camera device, and identifying the user image by the processor to acquire user limb data in the interactive data.
In an embodiment of the foregoing vehicle-mounted interactive terminal, optionally, the in-vehicle interactive apparatus includes an in-vehicle interactive input device; wherein
The interactive data comprises information data input by a user based on the in-vehicle interactive input device.
The invention also provides a vehicle-mounted system, which comprises the vehicle-mounted interactive terminal and an in-vehicle interactive device, wherein the vehicle-mounted interactive terminal is provided by any one embodiment of the invention; wherein
The in-vehicle interaction device comprises an in-vehicle camera device and/or an in-vehicle interaction input device.
Another aspect of the present invention also provides a computer readable medium having stored thereon computer readable instructions, which, when executed by a processor, implement the steps of the vehicle-mounted interaction method as provided in any one of the embodiments of the present invention.
According to the vehicle-mounted interaction method, the vehicle-mounted interaction terminal and the vehicle-mounted system, the VR can be expanded by utilizing the hardware of the vehicle, so that the use scene of virtual reality interaction is expanded, the equipment purchase cost is saved, and a user can conveniently access the meta universe at any time and any place.
Drawings
The above features and advantages of the present disclosure will be better understood upon reading the detailed description of embodiments of the disclosure in conjunction with the following drawings. In the drawings, components are not necessarily drawn to scale, and components having similar relative characteristics or features may have the same or similar reference numerals.
Fig. 1 is a flow chart of a vehicle-mounted interaction method according to an aspect of the present invention.
Fig. 2 is a flow chart illustrating steps related to a 3D vehicle scene in a vehicle interaction method provided by an aspect of the present invention.
Fig. 3 is a flow chart illustrating steps related to interactive data in a vehicle-mounted interaction method provided by an aspect of the present invention.
Fig. 4 shows a schematic structural diagram of the vehicle-mounted interactive terminal provided by another aspect of the present invention.
Reference numerals
100 vehicle-mounted interactive terminals;
101 a memory;
102 a processor;
103 a bus;
104 a random access memory;
105 a cache memory;
106 a storage system;
107 program modules;
108 an external device;
109 a display;
110 input/output (I/O) interfaces;
111 network adapter.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. It is noted that the aspects described below in connection with the figures and the specific embodiments are only exemplary and should not be construed as imposing any limitation on the scope of the present invention.
The following description is presented to enable any person skilled in the art to make and use the invention and is incorporated in the context of a particular application. Various modifications, as well as various uses in different applications will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to a wide range of embodiments. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
In the following detailed description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the practice of the invention may not necessarily be limited to these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.
The reader's attention is directed to all papers and documents which are filed concurrently with this specification and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference. All the features disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.
Note that where used, the designations left, right, front, back, top, bottom, positive, negative, clockwise, and counterclockwise are used for convenience only and do not imply any particular fixed orientation. In fact, they are used to reflect the relative position and/or orientation between the various parts of the object. Furthermore, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
It is noted that, where used, further, preferably, still further and more preferably is a brief introduction to the exposition of the alternative embodiment on the basis of the preceding embodiment, the contents of the further, preferably, still further or more preferably back band being combined with the preceding embodiment as a complete constituent of the alternative embodiment. Several further, preferred, still further or more preferred arrangements of the belt after the same embodiment may be combined in any combination to form a further embodiment.
The invention is described in detail below with reference to the figures and specific embodiments. It is noted that the aspects described below in connection with the figures and the specific embodiments are only exemplary and should not be construed as imposing any limitation on the scope of the present invention.
As described above, in order to solve the problems that in the prior art, the implementation of the VR function needs to depend on a fixed high-end computer terminal, cannot provide diversified use scenes, and cannot meet the requirement of accessing the meta universe of a user anytime and anywhere, the invention provides a vehicle-mounted interaction method, a vehicle-mounted interaction terminal, and a vehicle-mounted system. Please refer to fig. 1 to fig. 4 to understand the vehicle-mounted interaction method, the vehicle-mounted interaction terminal and the vehicle-mounted system provided by the present invention.
Referring to fig. 1, as shown in fig. 1, a vehicle-mounted interaction method provided by an aspect of the present invention includes:
in response to establishing a communication connection with the VR headset display,
sending an interaction request to an external VR content server to acquire a VR interface for interaction;
in response to receiving the VR interface returned by the VR content server,
and outputting the VR interface to a VR head-mounted display so as to display the VR interface on the VR head-mounted display.
VR head mounted displays in the present invention include various Virtual Reality (VR) glasses, eye shields, helmets, etc. that are or will be available, and include, but are not limited to, external head display devices, integral head display devices, mobile head display devices. The head-mounted display device can be used for sealing the vision and the hearing of people to the outside, and guiding a user to generate a feeling of being in a virtual environment. The display principle includes, but is not limited to, displaying images of left and right eyes on left and right eye screens respectively, and generating stereoscopic impression in the brain after the human eyes acquire the information with the difference.
The communication connection can also be realized by various existing or future wired or wireless communication technologies. For example, the wireless connection may be bluetooth or WiFi, or the wired connection may be USB transmission line.
The VR content server can provide a VR interface for interaction for VR devices accessed therein, can provide various VR interactive contents, provide various meta-space elements, and can also be regarded as a meta-space server. The vehicle-mounted system can output the interaction request to the VR content server through the existing or future data transmission mode, which is not limited herein.
Therefore, in the invention, in response to the vehicle-mounted interaction method applied to the vehicle-mounted system, in response to the communication connection with the VR head-mounted display, the interaction request is sent to the external VR content server to acquire the VR interface for interaction, so that the acquired VR interface can be output to the VR head-mounted display of the terminal, and the VR head-mounted display can display the VR interface.
Through opening the connection channel with VR head mounted display, to VR content server delivery relevant data, can realize VR head mounted display based on vehicle's application.
Further, in order to enhance the sense of actual experience of the user, the user can be connected to the real world environment in the vehicle even if the VR head mounted display is used, and a VR interface corresponding to the real-time scene in the vehicle can be displayed on the VR head mounted display. Referring to fig. 2, another aspect of the vehicle-mounted interaction method provided by an aspect of the present invention further includes the following steps:
acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point;
adjusting a preset reference 3D vehicle-mounted scene based on the relative position coordinates to obtain a real-time 3D vehicle-mounted scene;
and sending a real-time 3D vehicle-mounted scene corresponding to the current position of the VR head-mounted display to a VR content server so that the VR content server generates a VR interface corresponding to the current vehicle-mounted environment based on the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to a preset reference point.
Specifically, the method can be implemented by presetting a preset reference point corresponding to the initial position of the VR head-mounted display in advance, and presetting a 3D scene of the interior of the vehicle corresponding to the preset reference point. The 3D scene in the vehicle interior may include some hardware devices that are fixed and unchangeable in the vehicle interior, such as a vehicle door, a steering wheel, an instrument panel, and the like, and by providing the 3D scene including the hardware devices, it is helpful for a user to see the actual situation in the vehicle interior in a virtual display, so that when a subsequent user needs to perform a large-scale limb interaction, the user can be prevented from colliding with the hardware devices, and unnecessary loss is avoided.
Subsequently, in the process that the user uses the VR headset, the relative positional relationship (x, y, z) of the VR headset relative to the preset reference point can be obtained in real time according to the gyroscope of the VR headset, so that the preset reference 3D vehicle-mounted scene is adapted according to the relative positional relationship (x, y, z) obtained in real time to generate a real-time 3D vehicle-mounted scene corresponding to the real-time relative positional relationship.
The VR content server can fuse corresponding VR content to update a VR interactive interface in real time according to real-time 3D vehicle-mounted scenes fed back by the vehicle-mounted system, and then returns the VR interactive interface to the vehicle-mounted system and returns the VR head-mounted and display, so that a user can see the actual situation inside a vehicle in virtual display, and the virtual display system is beneficial to avoiding collision with hardware equipment and avoiding unnecessary loss when the subsequent user needs to perform large-amplitude limb interaction.
In another aspect, in order to advance the interaction between the user and the meta universe, the present invention further relates to a related step of acquiring interaction data, please refer to fig. 3, an aspect of the present invention provides a vehicle-mounted interaction method further including the following steps:
outputting the interactive data acquired from the in-vehicle interactive device to a VR content server so that the VR content server responds to the interactive data and updates a returned VR interface; and
and receiving the updated VR interface returned by the VR content server.
Further, for the vehicle-mounted system, the interactive data can be obtained from the in-vehicle interactive device in different manners.
In one embodiment, the in-vehicle interaction device comprises an in-vehicle camera device. Subsequently, the acquiring of the interaction data by the vehicle-mounted system further comprises:
and responding to the user image collected by the in-vehicle camera device, and identifying the user image to acquire the user limb data in the interactive data.
In the above embodiments, the above-mentioned in-vehicle camera device may be directed to one or more camera devices fixedly installed or added later on the vehicle. One or more camera devices can also be used for realizing other human-computer interaction and other scenes needing to collect images in the carriage. The one or more cameras may be a general camera, or a wide-angle camera, an infrared camera, and those skilled in the art may use existing or future cameras to implement the above-mentioned cameras, which is not limited to the above examples. In a scene with a plurality of image pickup devices, the plurality of image pickup devices can be arranged according to the distribution of seats, for example, the image pickup device of a passenger seat, the plurality of image pickup devices of a rear seat and the like, so that the relevant images of each passenger can be picked up, the image pickup effect can be ensured, and the situations that the body motions of the passenger are not picked up in an omission way or the body motions of the passenger cannot be recognized due to the poor image pickup angle can be reduced.
Further, the image recognition may be implemented in existing or future ways to recognize the user's gesture, including, but not limited to, recognizing the user's motion using 3D gesture estimation methods, for example.
By obtaining the limb data of the user, the interaction information represented by the user based on the change of the limb action can be obtained, and the interaction data are fed back to the VR content server, so that the VR content server responds to the interaction data to update the interaction interface.
In another embodiment, the in-vehicle interaction device includes in-vehicle interaction input devices, which include, but are not limited to, various vehicle-mounted displays with touch control functions, which are fixedly mounted in advance or later installed on the vehicle, and also include, but are not limited to, various physical keys, buttons, and other input devices, which are fixedly mounted in advance or later installed on the vehicle. But also includes various voice input devices such as a microphone fixedly installed in the vehicle in advance or additionally installed afterwards.
Since the in-vehicle system is able to obtain input interaction data for these in-vehicle interaction input devices, interaction inputs for VR interactions can be obtained by means of these devices.
So far, the specific implementation of the vehicle-mounted interaction method provided by the aspect of the invention has been described. The invention realizes the display of virtual reality effect by using a vehicle-mounted strong GPU, realizes the access of a low-delay network by using a vehicle-mounted communication module (5G), and can access the metaspace at any time and any place by only using a head-mounted display, namely, an interactive interface is displayed on VR glasses through connecting channels such as Bluetooth, USB and the like.
The user images are collected through the camera device in the vehicle, the coordinates of the human body in the 3D space are obtained through 3D human body posture estimation, and then the actions of the human body can be recognized, so that the interaction of the user in the meta space is realized. The 3D scene of the preset vehicle is transmitted to the VR glasses, so that a user can see information in the vehicle in virtual reality, mistaken touch is avoided, the preset 3D scene takes the VR glasses at a certain position as an original point, when the virtual reality vehicle is used by an actual user, the 3D scene is adapted after current coordinates (x, y and z) are obtained according to a gyroscope of the VR glasses, and finally the information is transmitted to the user.
The interaction of the user is not limited to the recognition of the captured motion of the camera, other devices in the vehicle can be used as interaction tools, for example, the central control screen can be used as a virtual reality desktop, and the user can complete information input through the central control screen in the meta universe.
The vehicle-mounted interaction method provided by the invention can expand VR by utilizing hardware of the vehicle, thereby expanding the use scene of virtual reality interaction, saving equipment purchase cost and facilitating the user to access the meta universe anytime and anywhere.
Another aspect of the present invention further provides a vehicle-mounted interactive terminal, where the vehicle-mounted interactive terminal includes: a memory and a communication module; and a processor connected with the memory and the communication module; wherein the processor is configured to: responding to the communication connection established with the VR head-mounted display through the communication module, and sending an interaction request to an external VR content server through the communication module to acquire a VR interface for interaction; and responding to the received VR interface returned by the VR content server, and outputting the VR interface to the VR head-mounted display so as to display the VR interface on the VR head-mounted display.
In an embodiment, the processor is further configured to: sending a real-time 3D on-board scene corresponding to the current position of the VR head-mounted display to the VR content server to cause the VR content server to generate a VR interface corresponding to the current on-board environment based on the real-time 3D on-board scene.
In an embodiment, the processor is further configured to: acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point; and adjusting the preset reference 3D vehicle-mounted scene based on the relative position coordinates to acquire the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to the preset reference point.
In an embodiment, the processor is further configured to: and outputting the interactive data acquired from the in-vehicle interactive device to the VR content server through the communication module so that the VR content server responds to the interactive data and updates the returned VR interface.
In one embodiment, the in-vehicle interaction device comprises an in-vehicle camera device; wherein in response to the in-vehicle camera device acquiring the user image, the processor identifies the user image to obtain the user limb data in the interactive data.
In one embodiment, the in-vehicle interaction device comprises an in-vehicle interaction input device; wherein the interactive data comprises information data input by a user based on the in-vehicle interactive input device.
That is to say, the vehicle-mounted interaction terminal provided by another aspect of the present invention is the vehicle-mounted interaction method described above, and specific reference is made to the detailed description of the vehicle-mounted interaction method above, which is not repeated herein.
Please refer to fig. 4 for further understanding of the vehicle-mounted interactive terminal provided by another aspect of the present invention. In the embodiment shown in fig. 4, the vehicle-mounted interactive terminal 100 is represented in the form of a general-purpose computer device, and is used for implementing the steps of the vehicle-mounted interactive method described in any one of the above embodiments. For details, please refer to the above description of the vehicle-mounted interaction method, which is not repeated herein.
The components of the in-vehicle interactive terminal 100 may include one or more memories 101, one or more processors 102, and a bus 103 that connects the various system components (including the memories 101 and processors 102).
The bus 103 includes a data bus, an address bus, and a control bus. The product of the number of bits of the data bus and the operating frequency is proportional to the data transfer rate, the number of bits of the address bus determines the maximum addressable memory space, and the control bus (read/write) indicates the type of bus cycle and the time at which the present I/O operation is completed. The processor 102 is connected to the memory 101 via the bus 103 and is configured to implement the vehicle interaction method provided by any of the above embodiments.
The processor 102 is a final execution unit for information processing and program operation, and serves as an operation and control core of the vehicle-mounted interactive terminal 100. The operation of all software layers in the computer system will eventually be mapped to the operation of the processor 102 by the instruction set. The processor 102 has the main functions of processing instructions, executing operations, controlling time and processing data.
The memory 101 is a variety of storage devices for storing programs and data in the computer. Memory 101 may include computer system readable media in the form of storage volatile memory. Such as Random Access Memory (RAM)104 and/or cache memory 105.
Random Access Memory (RAM)104 is an internal memory that exchanges data directly with processor 102. It can be read and written at any time (except for refreshing), and is fast, usually used as a temporary data storage medium for an operating system or other programs in operation, and the stored data will be lost when power is off. Cache memory (Cache)105 is a level one memory that exists between main memory and processor 102, and has a relatively small capacity but much higher speed than main memory, close to the speed of processor 102.
It should be noted that, in the case that the vehicle-mounted interaction terminal 100 includes a plurality of memories 101 and a plurality of processors 102, a distributed structure may be provided between the plurality of memories 101 and between the plurality of processors 102, for example, the vehicle-mounted interaction terminal may include memories and processors respectively located at a local end and a background cloud end, and the vehicle-mounted interaction method described above is implemented by the local end and the background cloud end together. Furthermore, in the embodiment adopting the distributed structure, the specific implementation terminal may be adjusted according to the actual situation in each step, and the specific implementation scheme of each step in a specific terminal should not unduly limit the protection scope of the present invention.
The in-vehicle interactive terminal 100 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. In this embodiment, the storage system 106 may be used to read from and write to non-removable, nonvolatile magnetic media.
The in-vehicle interactive terminal 100 may also communicate with one or more external devices 108. The external device 108 in this embodiment includes a vehicle-mounted camera for capturing the limb movements of the user to obtain the interaction data, may include the VR content server described above, may include the VR head-mounted display described above, and may further include the in-vehicle interaction input device described above. The external device 108 further includes a display 109, i.e. various vehicle-mounted displays with touch functions, which are fixedly installed in advance or installed afterwards as described above, and can be used for receiving the interactive data of the user, and the vehicle-mounted interactive terminal 100 can receive the interactive data of the user through the display 109.
The in-vehicle interaction terminal 100 may also communicate with one or more devices that enable a user to interact with the in-vehicle interaction terminal 100, and/or with any device (e.g., network card, modem, etc.) that enables the in-vehicle interaction terminal 100 to communicate with one or more other computing devices. Such communication may be through input/output (I/O) interfaces 110.
The in-vehicle interaction terminal 100 may also communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 111. As shown in FIG. 2, the network adapter 111 communicates with the other modules of the in-vehicle interaction terminal 100 through the bus 103. It should be appreciated that, although not shown in the figures, other hardware and/or software modules may be used in conjunction with the in-vehicle interaction terminal 100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
So far, the specific implementation manner of the vehicle-mounted interactive terminal provided by another aspect of the invention has been described. Further, another aspect of the present invention further provides a vehicle-mounted system, where the vehicle-mounted system includes the vehicle-mounted interactive terminal described in any one of the embodiments of the present invention, and an in-vehicle interactive device; the in-vehicle interaction device includes the in-vehicle camera device and/or the in-vehicle interaction input device, which have been described in detail above, and are not described herein again.
Another aspect of the present invention also provides a computer readable medium having stored thereon computer readable instructions, which, when executed by a processor, implement the steps of the vehicle-mounted interaction method as provided in any one of the embodiments of the present invention.
According to the vehicle-mounted interaction method, the vehicle-mounted interaction terminal and the vehicle-mounted system, the VR can be expanded by utilizing the hardware of the vehicle, so that the use scene of virtual reality interaction is expanded, the equipment purchase cost is saved, and a user can conveniently access the meta universe at any time and any place.
The various illustrative logical modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks (disks) usually reproduce data magnetically, while discs (discs) reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. It is to be understood that the scope of the invention is to be defined by the appended claims and not by the specific constructions and components of the embodiments illustrated above. Those skilled in the art can make various changes and modifications to the embodiments within the spirit and scope of the present invention, and these changes and modifications also fall within the scope of the present invention.
Claims (14)
1. A vehicle-mounted interaction method is applied to a vehicle-mounted system of a vehicle, and is characterized by comprising the following steps:
responding to the communication connection established with the VR head-mounted display, and sending an interaction request to an external VR content server to acquire a VR interface for interaction;
in response to receiving the VR interface returned by the VR content server, outputting the VR interface to the VR head mounted display for display of the VR interface on the VR head mounted display.
2. The vehicle-mounted interaction method of claim 1, further comprising:
sending a real-time 3D on-board scene corresponding to the VR head-mounted display current location to the VR content server to cause the VR content server to generate the VR interface corresponding to a current on-board environment based on the real-time 3D on-board scene.
3. The vehicle-mounted interaction method of claim 2, further comprising:
acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point;
and adjusting a preset reference 3D vehicle-mounted scene based on the relative position coordinates to acquire the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to the preset reference point.
4. The vehicle-mounted interaction method of claim 1, further comprising:
and outputting the interaction data acquired from the in-vehicle interaction device to the VR content server so that the VR content server responds to the interaction data and updates the returned VR interface.
5. The vehicle-mounted interaction method according to claim 4, wherein the in-vehicle interaction device comprises an in-vehicle camera device; wherein
And responding to the user image collected by the in-vehicle camera device, and identifying the user image to acquire the user limb data in the interactive data.
6. The vehicle-mounted interaction method of claim 4, wherein the in-vehicle interaction device comprises an in-vehicle interaction input device; wherein
The interactive data comprises information data input by a user based on the in-vehicle interactive input device.
7. The vehicle-mounted interactive terminal is characterized by comprising:
a memory and a communication module; and
the processor is connected with the memory and the communication module; wherein
The processor is configured to:
responding to the communication connection established with the VR head-mounted display through the communication module, and sending an interaction request to an external VR content server through the communication module to acquire a VR interface for interaction;
in response to receiving the VR interface returned by the VR content server, outputting the VR interface to the VR head mounted display for display of the VR interface on the VR head mounted display.
8. The vehicle interactive terminal of claim 7, wherein the processor is further configured to:
sending a real-time 3D in-vehicle scene corresponding to the VR head-mounted display current location to the VR content server to cause the VR content server to generate the VR interface corresponding to a current in-vehicle environment based on the real-time 3D in-vehicle scene.
9. The vehicle interactive terminal of claim 8, wherein the processor is further configured to:
acquiring the current relative position coordinate of the VR head-mounted display relative to a preset datum point;
and adjusting a preset reference 3D vehicle-mounted scene based on the relative position coordinates to acquire the real-time 3D vehicle-mounted scene, wherein the preset reference 3D vehicle-mounted scene is a 3D vehicle-mounted scene corresponding to the preset reference point.
10. The vehicle interactive terminal of claim 7, wherein the processor is further configured to:
and outputting the interaction data acquired from the in-vehicle interaction device to the VR content server through the communication module so that the VR content server responds to the interaction data and updates the returned VR interface.
11. The vehicle-mounted interactive terminal according to claim 10, wherein the in-vehicle interactive device comprises an in-vehicle camera device; wherein
And responding to the user image acquired by the in-vehicle camera device, and identifying the user image by the processor to acquire the user limb data in the interactive data.
12. The in-vehicle interactive terminal of claim 10, wherein the in-vehicle interactive apparatus comprises an in-vehicle interactive input device; wherein
The interactive data comprises information data input by a user based on the in-vehicle interactive input device.
13. An in-vehicle system, characterized in that the in-vehicle system comprises the in-vehicle interactive terminal according to any one of claims 7-12, and an in-vehicle interactive device; wherein
The in-vehicle interaction device comprises an in-vehicle camera device and/or an in-vehicle interaction input device.
14. A computer readable medium having stored thereon computer readable instructions which, when executed by a processor, carry out the steps of the in-vehicle interaction method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111665439.2A CN114428577A (en) | 2021-12-31 | 2021-12-31 | Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111665439.2A CN114428577A (en) | 2021-12-31 | 2021-12-31 | Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114428577A true CN114428577A (en) | 2022-05-03 |
Family
ID=81311140
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111665439.2A Pending CN114428577A (en) | 2021-12-31 | 2021-12-31 | Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114428577A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114996183A (en) * | 2022-06-07 | 2022-09-02 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted augmented reality system, vehicle and communication implementation method |
| CN115017352A (en) * | 2022-06-22 | 2022-09-06 | 润芯微科技(江苏)有限公司 | Mobile phone car machine interaction system based on image recognition |
| CN115550886A (en) * | 2022-11-29 | 2022-12-30 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted augmented reality equipment control method and system and vehicle-mounted interaction system |
| CN115567632A (en) * | 2022-09-08 | 2023-01-03 | 闪耀现实(无锡)科技有限公司 | Method and device for establishing data path, system, vehicle, equipment and medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106095102A (en) * | 2016-06-16 | 2016-11-09 | 深圳市金立通信设备有限公司 | The method of a kind of virtual reality display interface process and terminal |
| CN107274469A (en) * | 2017-06-06 | 2017-10-20 | 清华大学 | The coordinative render method of Virtual reality |
| CN107369207A (en) * | 2016-05-11 | 2017-11-21 | 百度（美国）有限责任公司 | The system and method that enhancing virtual reality content is provided in automatic driving vehicle |
| CN108762492A (en) * | 2018-05-14 | 2018-11-06 | 歌尔科技有限公司 | Method, apparatus, equipment and the storage medium of information processing are realized based on virtual scene |
| CN109032353A (en) * | 2018-07-20 | 2018-12-18 | 爱驰汽车有限公司 | Interior virtual reality experience method and system based on car-mounted terminal, car-mounted terminal |
| CN109936834A (en) * | 2017-12-19 | 2019-06-25 | 通用汽车环球科技运作有限责任公司 | Augmented reality vehicle user interface |
| CN110582334A (en) * | 2017-05-10 | 2019-12-17 | 环球城市电影有限责任公司 | Virtual reality mobile pod |
| CN111201474A (en) * | 2017-10-12 | 2020-05-26 | 奥迪股份公司 | Method for operating a head-wearable electronic display device and display system for displaying virtual content |
| CN112241204A (en) * | 2020-12-17 | 2021-01-19 | 宁波均联智行科技有限公司 | Gesture interaction method and system of vehicle-mounted AR-HUD |
| CN112506351A (en) * | 2020-12-23 | 2021-03-16 | 武汉小绿人动力技术股份有限公司 | Immersive racing car system in outdoor racing car field and control method |
| CN112799499A (en) * | 2019-10-24 | 2021-05-14 | 福特全球技术公司 | Motor vehicle man-machine interaction system and method |
| US20210199980A1 (en) * | 2017-09-26 | 2021-07-01 | Audi Ag | Method for operating a head-mounted electronic display device, and display system for displaying a virtual content |
| CN113147596A (en) * | 2021-04-13 | 2021-07-23 | 一汽奔腾轿车有限公司 | AR technology-based head-mounted vehicle-mounted information display system and method |
-
2021
- 2021-12-31 CN CN202111665439.2A patent/CN114428577A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107369207A (en) * | 2016-05-11 | 2017-11-21 | 百度（美国）有限责任公司 | The system and method that enhancing virtual reality content is provided in automatic driving vehicle |
| CN106095102A (en) * | 2016-06-16 | 2016-11-09 | 深圳市金立通信设备有限公司 | The method of a kind of virtual reality display interface process and terminal |
| CN110582334A (en) * | 2017-05-10 | 2019-12-17 | 环球城市电影有限责任公司 | Virtual reality mobile pod |
| CN107274469A (en) * | 2017-06-06 | 2017-10-20 | 清华大学 | The coordinative render method of Virtual reality |
| US20210199980A1 (en) * | 2017-09-26 | 2021-07-01 | Audi Ag | Method for operating a head-mounted electronic display device, and display system for displaying a virtual content |
| CN111201474A (en) * | 2017-10-12 | 2020-05-26 | 奥迪股份公司 | Method for operating a head-wearable electronic display device and display system for displaying virtual content |
| CN109936834A (en) * | 2017-12-19 | 2019-06-25 | 通用汽车环球科技运作有限责任公司 | Augmented reality vehicle user interface |
| CN108762492A (en) * | 2018-05-14 | 2018-11-06 | 歌尔科技有限公司 | Method, apparatus, equipment and the storage medium of information processing are realized based on virtual scene |
| CN109032353A (en) * | 2018-07-20 | 2018-12-18 | 爱驰汽车有限公司 | Interior virtual reality experience method and system based on car-mounted terminal, car-mounted terminal |
| CN112799499A (en) * | 2019-10-24 | 2021-05-14 | 福特全球技术公司 | Motor vehicle man-machine interaction system and method |
| CN112241204A (en) * | 2020-12-17 | 2021-01-19 | 宁波均联智行科技有限公司 | Gesture interaction method and system of vehicle-mounted AR-HUD |
| CN112506351A (en) * | 2020-12-23 | 2021-03-16 | 武汉小绿人动力技术股份有限公司 | Immersive racing car system in outdoor racing car field and control method |
| CN113147596A (en) * | 2021-04-13 | 2021-07-23 | 一汽奔腾轿车有限公司 | AR technology-based head-mounted vehicle-mounted information display system and method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114996183A (en) * | 2022-06-07 | 2022-09-02 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted augmented reality system, vehicle and communication implementation method |
| CN115017352A (en) * | 2022-06-22 | 2022-09-06 | 润芯微科技(江苏)有限公司 | Mobile phone car machine interaction system based on image recognition |
| CN115017352B (en) * | 2022-06-22 | 2023-04-07 | 润芯微科技(江苏)有限公司 | Mobile phone car machine interaction system based on image recognition |
| CN115567632A (en) * | 2022-09-08 | 2023-01-03 | 闪耀现实(无锡)科技有限公司 | Method and device for establishing data path, system, vehicle, equipment and medium |
| CN115567632B (en) * | 2022-09-08 | 2024-02-23 | 闪耀现实(无锡)科技有限公司 | Method and device for establishing data path, system, vehicle, equipment and medium |
| CN115550886A (en) * | 2022-11-29 | 2022-12-30 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted augmented reality equipment control method and system and vehicle-mounted interaction system |
| CN115550886B (en) * | 2022-11-29 | 2023-03-28 | 蔚来汽车科技(安徽)有限公司 | Vehicle-mounted augmented reality equipment control method and system and vehicle-mounted interaction system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114428577A (en) | Vehicle-mounted interaction method, vehicle-mounted interaction terminal and vehicle-mounted system | |
| CN109564466B (en) | System and method for reducing motion photon latency and memory bandwidth in virtual reality systems | |
| US10255481B2 (en) | Display device and operating method thereof with adjustments display | |
| CN110766777A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| US20180068489A1 (en) | Server, user terminal device, and control method therefor | |
| CN110070496B (en) | Method and device for generating image special effect and hardware device | |
| US20220269338A1 (en) | Augmented devices | |
| CN112083795A (en) | Object control method and device, storage medium and electronic equipment | |
| WO2022252924A1 (en) | Image transmission and display method and related device and system | |
| CN114461070A (en) | System, method and vehicle for realizing vehicle-mounted virtual reality | |
| CN110879683A (en) | Screen information sharing method, terminal and vehicle | |
| CN115185408A (en) | Vehicle-mounted entertainment information display method, device, equipment and medium | |
| CN110641478B (en) | Automobile domain controller display method and device, automobile and readable storage medium | |
| CN111435269A (en) | Display adjusting method, system, medium and terminal of vehicle head-up display device | |
| CN116088784B (en) | Image screen projection method, device, electronic equipment, chip, storage medium and vehicle | |
| US11910068B2 (en) | Panoramic render of 3D video | |
| CN113946259B (en) | Vehicle information processing method and device, electronic equipment and readable medium | |
| CN109361888A (en) | Method of adjustment and device, the video call device and storage medium of call background | |
| US20230052104A1 (en) | Virtual content experience system and control method for same | |
| US20210097731A1 (en) | Presenting environment based on physical dimension | |
| CN115129207A (en) | Working method and device of vehicle-mounted display module interactive system, storage medium and terminal | |
| CN112954291A (en) | Method, apparatus and storage medium for processing 3D panorama image or video of vehicle | |
| US12034908B2 (en) | Stereoscopic-image playback device and method for generating stereoscopic images | |
| US20230239458A1 (en) | Stereoscopic-image playback device and method for generating stereoscopic images | |
| CN114581627B (en) | ARHUD-based imaging method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 314500 988 Tong Tong Road, Wu Tong Street, Tongxiang, Jiaxing, Zhejiang Applicant after: United New Energy Automobile Co.,Ltd. Address before: 314500 988 Tong Tong Road, Wu Tong Street, Tongxiang, Jiaxing, Zhejiang Applicant before: Hozon New Energy Automobile Co., Ltd. |
|
| CB02 | Change of applicant information |