WO2023246401A1

WO2023246401A1 - In-vehicle augmented reality system, vehicle, and communication implementation method

Info

Publication number: WO2023246401A1
Application number: PCT/CN2023/095491
Authority: WO
Inventors: 张慧敏; 沈波
Original assignee: 蔚来汽车科技(安徽)有限公司
Priority date: 2022-06-24
Filing date: 2023-05-22
Publication date: 2023-12-28
Also published as: CN115042730A

Abstract

An in-vehicle augmented reality system, comprising at least one head-mounted display device (1), a cockpit control end (2), a vehicle body control end (3) and a cockpit (4), wherein the head-mounted display device (1) is connected to the cockpit control end (2), and the head-mounted display device (1) is used for acquiring control information and sending the control information to the cockpit control end (2); the cockpit control end (2) is connected to the vehicle body control end (3), and the cockpit control end (2) is used for performing decision making and determination according to the control information, so as to generate a control signal, and sending the control signal to the vehicle body control end (3); the vehicle body control end (3) is connected to the cockpit (4), and the vehicle body control end (3) is used for sending a call request to the cockpit (4) according to the control signal; and the cockpit (4) is used for executing a control operation according to the call request. A vehicle, comprising an in-vehicle augmented reality system; and a communication implementation method applied to an in-vehicle augmented reality system. By means of combining an intelligent cockpit with VR technology, a 3D virtual reality scenario and motion space experience of three degrees of freedom are provided for a user.

Description

A vehicle-mounted augmented reality system, vehicle and communication implementation method

Technical field

This application relates to the field of automotive technology, and in particular to a vehicle-mounted augmented reality system, vehicle and communication implementation method.

Background technique

As new energy vehicles become more and more popular, users increasingly hope that new energy vehicles can meet more entertainment needs. In daily life, VR/AR glasses can meet most of the entertainment needs of users. Therefore, it will The combination of VR/AR glasses and new energy vehicles has great market potential. Existing technology has tried to connect VR/AR glasses in new energy vehicles, but this can only provide users with 3D virtual reality and cannot provide users with a three-degree-of-freedom motion space experience. , unable to get an immersive experience.

Contents of the invention

The purpose of this application is to provide a vehicle-mounted augmented reality system, a vehicle and a communication implementation method, which provide users with a 3D virtual reality scene and a three-degree-of-freedom motion space experience by combining a smart cockpit with VR technology.

The purpose of this application is achieved by adopting the following technical solutions. A vehicle-mounted augmented reality system proposed according to this application includes at least one head-mounted display device, a cockpit control terminal, a vehicle body control terminal and a cockpit. The head-mounted display device is connected to the cockpit control terminal. The head-mounted display device is connected to the cockpit control terminal. The display device is used to obtain control information and send the control information to the cockpit control end. The cockpit control end is connected to the vehicle body control end. The cockpit control end is used to make decisions and decisions based on the control information. Determine to generate a control signal, and send the control signal to the vehicle body control end, the vehicle body control end is connected to the cockpit, and the body control end is used to send a call request to the cockpit according to the control signal , the cockpit is used to perform control operations according to the calling request.

In some embodiments, the cockpit control terminal includes a head-mounted display device management module and a head-mounted display device control application module. The head-mounted display device management module Connected to the head-mounted display device control application module.

In some embodiments, the head-mounted display device management module is configured to receive the control information, make decisions and judgments based on the control information to generate the control signal, and send the control signal to the A head-mounted display device control application module, the head-mounted display device control application module is used to send the control signal to the vehicle body control terminal.

In some embodiments, the body control terminal includes a body control unit and an SOA application module, and the body control unit is connected to the SOA application module.

In some embodiments, the vehicle body control unit is configured to receive the control signal and send a calling request to the SOA application module according to the control signal, and the SOA application module is configured to send the calling request to the cockpit.

In some implementations, the vehicle body control unit is configured to receive the control signal and query whether a registration service exists in the service registration list according to the control signal, and send a calling request to the SOA application module according to the existing registration service.

In some embodiments, the cabin includes a seat assembly and a suspension assembly, and the seat assembly and the suspension assembly are configured to perform control operations according to the invocation request.

In some embodiments, the seat assembly includes a seat, a seat control unit and a seat drive mechanism. The seat control unit is connected to the seat drive mechanism. The seat drive mechanism is connected to the seat drive mechanism. The suspension assembly includes a suspension, a suspension control unit and a suspension driving mechanism. The suspension control unit is connected to the suspension driving mechanism, and the suspension driving mechanism is connected to the suspension.

This application also provides a communication implementation method, which is applied to a vehicle-mounted augmented reality system, including: S101. Obtain control information through a head-mounted display device and send the control information to the cockpit control terminal; S102. Perform operations based on the control information. Decision and judgment are made to generate a control signal, and the control signal is sent to the vehicle body control terminal; S103. Send a call request to the cockpit according to the control signal; S104. Perform a control operation according to the call request.

This application also provides a vehicle, including the aforementioned vehicle-mounted augmented reality system.

The beneficial effects of this application include at least:

This application uses a head-mounted display device to connect to the cockpit control end, and the head-mounted display device is used to obtain control information and send the control information to the cockpit control end. The cockpit control end is connected to the body control end, and the cockpit control end is used to control according to information for decision making and judgment in order to Generate a control signal and send the control signal to the body control terminal. The body control terminal is connected to the cockpit. The body control terminal is used to send a call request to the cockpit according to the control signal. The cockpit is used to perform control operations according to the call request, realizing the intelligent cockpit. Combined with VR technology, it provides users with 3D virtual reality scenes and three degrees of freedom motion space experience.

The above description is only an overview of the technical solutions of the present application. In order to have a clearer understanding of the technical means of the present application, they can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand. , the following is a detailed description of the preferred embodiments, together with the accompanying drawings.

Description of the drawings

Figure 1 shows a schematic structural diagram of a vehicle-mounted augmented reality system according to an embodiment of the present application.

Detailed ways

In order to further elaborate on the technical means of the present application, the specific implementation manner of a vehicle-mounted augmented reality system, vehicle and communication implementation method proposed according to the present application will be described in detail below with reference to the drawings and preferred embodiments. The following examples are only used to illustrate the technical solution of the present application more clearly, and are therefore only used as examples and cannot be used to limit the protection scope of the present application.

Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the technical field belonging to this application; the terms used herein are for the purpose of describing specific embodiments only and are not intended to be used in Limitation of this application; the terms "including" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion.

In the description of the embodiments of this application, unless otherwise clearly stated and limited, the technical term "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integrated connection; it can also be a mechanical connection. A connection may be an electrical connection, a bridge connection, or a network connection; it may be a direct connection or an indirect connection through an intermediary; it may be an internal connection between two components or an interaction between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of this application can be understood according to specific circumstances.

As shown in Figure 1, the vehicle-mounted augmented reality system described in this application includes at least one head-mounted display device 1, a cockpit control terminal 2, a vehicle body control terminal 3 and a cockpit 4. The head-mounted display device 1 can be AR/VR/ For augmented reality devices such as MR, the head-mounted display device 1 may be an integrated head-mounted display device or a split head-mounted display device. The head-mounted display device 1 is connected to the cockpit control terminal 2 . For example, the head-mounted display device 1 and the cockpit control terminal 2 may be connected through wired or wireless means. The cockpit 4 includes a seat assembly 41 and a suspension assembly 42. The user wears the head-mounted display device 1 and sits on the seat of the seat assembly 41. The number of seat assemblies 41 corresponds to the number of the head-mounted display device 1. . The seat 41 assembly can realize vibration, forward and backward movement, left and right movement, and up and down lifting movements. The suspension assembly 42 is used to enable the cabin 4 to move in the x-y-z three-axis direction, including forward and backward pitching motion around the x-axis, left and right yaw motion around the y-axis, and up and down lifting motion around the z-axis.

The head-mounted display device 1 is provided with a sensor for sensing the degree-of-freedom space position information. Based on the degree-of-freedom space position information, the head-mounted display device 1 can generate a virtual space, so that the user can feel that he or she is in the virtual space. the orientation. The head-mounted display device 1 can provide users with virtual reality scenes including games, meetings, and movie watching. In these virtual reality scenes, in addition to generating 3D stereoscopic images, the head-mounted display device 1 also generates control information and passes it through The control information is transmitted to the cockpit control terminal 2 through wired or wireless transmission. The control information at least includes one or more of motion posture changes, pitch, lift, vibration, and swing.

The cockpit control terminal 2 has chips related to cockpit control and calculation. The cockpit control terminal 2 is connected to the body control terminal 3. For example, the cockpit control terminal 2 and the body control terminal 3 may be connected through Ethernet. The cockpit control terminal 2 is used to receive the control information transmitted by the head-mounted display device 1 and make decisions and judgments based on the control information to generate a control signal and send the control signal to the vehicle body control terminal 3 . Specifically, the cockpit control terminal 2 includes a head-mounted display device management module 21 and a head-mounted display device control application module 22. The head-mounted display device management module 21 is connected to the head-mounted display device control application module 22. The head-mounted display device management module 21 is connected to the head-mounted display device control application module 22. The display device management module 21 is used to receive feedback control information from each head-mounted display device 1, and integrate the control information sent by each head-mounted display device 1 to make decisions and judgments to generate control signals. The head-mounted display device The management module 21 is also used to send the control signal to the head-mounted display device control application module 22 , and the head-mounted display device control application module 22 sends the control signal to the vehicle body control terminal 3 .

For example, the head-mounted display device a generates the control information of "left and right swing" + "vibration", and the head-mounted display device b generates the control information of "left and right swing" + "backward". These control information are transmitted through wired or wireless means. to the head-mounted display device management module 21. The head-mounted display device management module 21 receives these control information and conducts comprehensive analysis and decision-making to generate "cabin 4 swings left and right", "seat assembly a vibrates" and "seat Component b retreats" three control signals, and sends these three control signals to the head-mounted display device control application module 22. The head-mounted display device control application module 22 sends these three control signals to the body control terminal 3.

The body control terminal 3 is connected to the cockpit 4 . For example, the body control terminal 3 and the cockpit 4 are connected through a vehicle bus. The body control terminal 3 is used to receive various control signals from the cockpit control terminal 2 and send control operations to various actuators connected to the body control terminal 3 to implement various control signal executions. In this application, the vehicle body control terminal 3 is used to receive the control signal and send a calling request to the cockpit 4 according to the control signal. Specifically, the body control terminal 3 includes a body control unit 31 and an SOA application module 32. The body control unit 31 is connected to the SOA application module 32. The body control unit 31 is used to receive the control signal sent by the head-mounted display device control application module 22. The call request is sent to the SOA application module 32 according to the control signal, and the SOA application module 32 is used to send the call request to the cockpit 4 .

The seat assembly 41 and suspension assembly 42 of the cockpit 4 are used to receive the call request and perform control operations according to the call request, and feed back the results to the vehicle body control terminal 3 . The seat assembly 41 includes a seat, a seat control unit and a seat driving mechanism. The seat control unit is connected to the seat driving mechanism, and the seat driving mechanism is connected to the seat. The suspension assembly 42 includes a suspension, a suspension control unit, and a suspension driving mechanism. The suspension control unit is connected to the suspension driving mechanism, and the suspension driving mechanism is connected to the suspension. The seat control unit is used to receive the call request from the body control terminal 3 and control the seat drive mechanism to drive the seat to perform control operations. The suspension control unit is used to receive the call request from the body control terminal 3 and control the suspension driving mechanism to drive the suspension to perform control operations.

After the seat assembly 41 and the suspension assembly 42 are connected to the body control terminal 3 , the seat assembly 41 and the suspension assembly 42 register services with the body control unit 31 of the body control terminal 3 through SOA (service-oriented architecture design). SOA is used to define interfaces and calling standards between different functional units of an application, thereby enabling remote calling. After successful registration, the SOA application module 32 of the body control terminal 3 can call the seat assembly 41 and suspension through SOA. Component 42 Services.

Specifically, the body control unit 31 is used to receive the control signal sent by the head-mounted display device control application module 22, and query whether there is a registration service in the service registration list according to the control signal, and send a request to the SOA application module 32 according to the existing registration service. Send a call request.

Following the above, the head-mounted display device control application module 22 sends three control signals of "cabin 4 swings left and right", "seat assembly a vibrates" and "seat assembly b retreats" to the body control unit 31. The body control end 3 The body control unit 31 receives these three control signals and queries whether there are registration services for suspension component 42, seat component a, and seat component b in the service registration list. The SOA application module 32 sends a call request, and the SOA application module 32 receives the call request and sends corresponding call requests to the suspension component 42, the seat component a, and the seat component b respectively.

After the seat control unit of seat component a receives the "vibration" call request, the seat control unit of seat component a controls the seat drive mechanism of seat component a to drive the seat to vibrate; the seat of seat component b After the control unit receives the "retreat" call request, the seat control unit of the seat assembly b controls the seat drive mechanism of the seat assembly b to drive the seat to retreat; the suspension control unit of the suspension assembly 42 receives the "left and right deflection" request. After the call request of "pendulum", the suspension driving mechanism is controlled to drive the suspension to make the cabin 4 swing left and right. After the above operation is completed, the seat component a, the seat component b and the suspension component 42 feed back the results to the body control terminal 3 .

In this application, the head-mounted display device 1 is connected to the cockpit control terminal 2, and the head-mounted display device 1 is used to obtain control information and send the control information to the cockpit control terminal 2. The cockpit control terminal 2 is connected to the body control terminal 3, and the cockpit The control terminal 2 is used to make decisions and judgments based on the control information to generate control signals, and send the control signals to the body control terminal 3. The body control terminal 3 is connected to the cockpit 4, and the body control terminal 3 is used to send control signals to the cockpit 4. The call request, cockpit 4 is used to perform control operations according to the call request, realizing the combination of smart cockpit and VR technology, providing users with a 3D virtual reality scene and a motion space experience with three degrees of freedom.

Based on the above vehicle-mounted augmented reality system, this application also provides a communication implementation method, including:

S101. Obtain control information through the head-mounted display device and send the control information to the cockpit control terminal;

S102. Make decisions and judgments based on the control information to generate control signals, and send the control signals to the vehicle body control terminal;

S103. Send a calling request to the cockpit according to the control signal;

S104. Perform a control operation according to the calling request.

It can be understood that the communication implementation method described in this application is not limited to the vehicle-mounted augmented reality system described in this application. This method is also applicable in other similar embodiments of vehicle-mounted augmented reality systems.

The above description of the disclosed aspects is provided to enable any person skilled in the art to practice the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, this application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

A vehicle-mounted augmented reality system, characterized in that it includes at least one head-mounted display device, a cockpit control terminal, a vehicle body control terminal and a cockpit, the head-mounted display device is connected to the cockpit control terminal, and the head-mounted display device is connected to the cockpit control terminal. The display device is used to obtain control information and send the control information to the cockpit control end. The cockpit control end is connected to the body control end. The cockpit control end is used to make decisions and judgments based on the control information. to generate a control signal and send the control signal to the vehicle body control terminal, the vehicle body control terminal is connected to the cockpit, and the vehicle body control terminal is used to send a calling request to the cockpit according to the control signal, The cockpit is used to perform control operations according to the calling request.
The vehicle-mounted augmented reality system according to claim 1, wherein the cockpit control terminal includes a head-mounted display device management module and a head-mounted display device control application module, and the head-mounted display device management module is connected to the head-mounted display device management module. The head-mounted display device controls the application module connection.
The vehicle-mounted augmented reality system according to claim 2, wherein the head-mounted display device management module is configured to receive the control information and make decisions and judgments based on the control information to generate the control signal, And the control signal is sent to the head-mounted display device control application module, and the head-mounted display device control application module is used to send the control signal to the vehicle body control terminal.
The vehicle-mounted augmented reality system according to claim 1, wherein the vehicle body control terminal includes a vehicle body control unit and an SOA application module, and the vehicle body control unit is connected to the SOA application module.
The vehicle-mounted augmented reality system according to claim 4, wherein the vehicle body control unit is configured to receive the control signal and send a calling request to the SOA application module according to the control signal, and the SOA application module uses to send the calling request to the cockpit.
The vehicle-mounted augmented reality system according to claim 5, characterized in that the body control unit is configured to receive the control signal and query whether a registration service exists in the service registration list according to the control signal, and according to the existing registration service Send a calling request to the SOA application module.
The vehicle-mounted augmented reality system according to claim 1, characterized in that: The cabin includes a seat assembly and a suspension assembly, and the seat assembly and the suspension assembly are used to perform control operations according to the calling request.
The vehicle-mounted augmented reality system according to claim 2, wherein the seat assembly includes a seat, a seat control unit and a seat drive mechanism, and the seat control unit is connected to the seat drive mechanism, The seat driving mechanism is connected to the seat. The suspension assembly includes a suspension, a suspension control unit and a suspension driving mechanism. The suspension control unit is connected to the suspension driving mechanism. The suspension The rack driving mechanism is connected with the suspension.
A communication implementation method applied to a vehicle-mounted augmented reality system, which is characterized by including:

S101. Obtain control information through the head-mounted display device and send the control information to the cockpit control terminal;

S102. Make decisions and judgments based on the control information to generate control signals, and send the control signals to the vehicle body control terminal;

S103. Send a calling request to the cockpit according to the control signal;

S104. Perform a control operation according to the calling request.
A vehicle, characterized by comprising the vehicle-mounted augmented reality system according to any one of claims 1 to 8.