WO2023284382A1

WO2023284382A1 - Display method and apparatus, head-mounted augmented reality device, and storage medium

Info

Publication number: WO2023284382A1
Application number: PCT/CN2022/091719
Authority: WO
Inventors: 李炳耀
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-07-16
Filing date: 2022-05-09
Publication date: 2023-01-19
Also published as: CN113534466B; CN113534466A

Abstract

A display method and apparatus (300), a head-mounted augmented reality device (100), and a storage medium (400), relating to the technical field of augmented reality. The display method is applied to the head-mounted augmented reality device (100) comprising a display screen (130), and comprises: when the head-mounted augmented reality device (100) located in a vehicle (200) detects that the vehicle (200) starts running, obtaining a running state of the vehicle (200) (S110); obtaining a virtual road condition image corresponding to the running state (S120); displaying a target content image in a first display region (131) of the display screen (130), and displaying the virtual road condition image in a second display region (132) of the display screen (130) (S130). In the vehicle (200) that is running, the display screen (130) of the head-mounted augmented reality device (100) is divided into two regions for respectively displaying the target content image and the virtual road condition image, such that a user can view the virtual road condition image corresponding to the running state of the vehicle (200) while viewing the target content image, so as to synchronize visual information with inner ear information, and thus an effect of preventing motion sickness is achieved.

Description

Display method, device, head-mounted augmented reality device, and storage medium

Cross References to Related Applications

This application claims the priority of the Chinese application with application number CN202110806323.X filed on July 16, 2021, which is hereby incorporated by reference in its entirety for all purposes.

technical field

The present application relates to the field of augmented reality technology, and more specifically, to a display method, device, head-mounted augmented reality device, and storage medium.

Background technique

In recent years, with the development of vehicle technology, more and more users rely on vehicles as a means of travel. However, the user is prone to motion sickness when riding in a vehicle. In particular, when the user watches the content of an electronic device while riding in a vehicle, the probability of motion sickness will be increased due to the asynchrony between the information seen by the eyes and the information of the inner ear.

Contents of the invention

In view of the above problems, the present application proposes a display method, device, head-mounted augmented reality device and storage medium to solve the above problems.

In the first aspect, the embodiment of the present application provides a display method, which is applied to a head-mounted augmented reality device, and the head-mounted augmented reality device includes a display screen. The method includes: when the head-mounted augmented reality device in the vehicle When the augmented reality device detects that the vehicle starts to drive, it acquires the driving state of the vehicle; acquires a virtual road condition image corresponding to the driving state; displays the target content image in the first display area of the display screen, and The second display area of the display screen displays the virtual road condition image.

In the second aspect, the embodiment of the present application provides a display device, which is applied to a head-mounted augmented reality device, the head-mounted augmented reality device includes a display screen, and the device includes: a driving state acquisition module, configured to When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, it acquires the driving state of the vehicle; the virtual road condition image acquisition module is used to acquire a virtual road condition image corresponding to the driving state; a display module , for displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

In a third aspect, an embodiment of the present application provides a head-mounted augmented reality device, including a display screen, a memory, and a processor, the display screen and the memory are coupled to the processor, and the memory stores instructions, The processor performs the method described above when the instructions are executed by the processor.

In a fourth aspect, the embodiment of the present application provides a computer-readable storage medium, where program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the above method.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 shows a schematic diagram of an application scenario of a display method provided by an embodiment of the present application;

Fig. 2 shows a schematic diagram of another application scenario of the display method provided by the embodiment of the present application;

FIG. 3 shows a schematic flowchart of a display method provided by an embodiment of the present application;

FIG. 4 shows a schematic diagram of a display screen of a head-mounted augmented reality device provided by an embodiment of the present application;

Fig. 5 shows a schematic flowchart of a display method provided by another embodiment of the present application;

FIG. 6 shows a schematic flowchart of step S220 of the display method shown in FIG. 5 of the present application;

FIG. 7 shows a schematic flowchart of a display method provided by another embodiment of the present application;

FIG. 8 shows a schematic flowchart of a display method provided by another embodiment of the present application;

FIG. 9 shows a schematic flow chart of a display method provided by still another embodiment of the present application;

Fig. 10 shows a schematic flow chart of a display method provided by yet another embodiment of the present application;

FIG. 11 shows a block diagram of a display device provided by an embodiment of the present application;

FIG. 12 shows a block diagram of an electronic device for performing a display method according to an embodiment of the present application in an embodiment of the present application;

FIG. 13 shows a storage unit used to store or carry program codes for implementing the display method according to the embodiment of the present application according to the embodiment of the present application.

detailed description

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

The inventor found through research that in order to prevent the user from motion sickness when riding a vehicle, the user can be allowed to wear a head-mounted virtual reality (VR, Virtual Reality) device, wherein the anti-motion sickness principle is in the head-mounted virtual reality device Play the picture that matches the driving state of the vehicle, so that the face can perceive the information that is consistent with the motion state of the human body, so as to achieve the effect of preventing motion sickness. Among them, in the application of the head-mounted virtual reality device, the head-mounted virtual reality device needs to access all vehicle information, such as brake information, accelerator information, suspension information, etc., and the head-mounted virtual reality device , calculate the driving state of the vehicle, and display a matching screen according to the driving state of the vehicle, so that the user's visual information and the information received by the body match each other, so as to achieve the effect of preventing motion sickness.

However, the head-mounted virtual reality device needs to obtain the vehicle to provide an interface for obtaining vehicle information. This requires the establishment of a cooperative relationship with the vehicle manufacturer. The cost of cooperation is high, and it needs to take into account different vehicle manufacturers and different models. The universality of the technology and less promotional. In addition, users cannot see the real world when wearing a head-mounted virtual reality device, and users cannot quickly perceive and respond to emergencies in a driving vehicle, and the safety is not high.

In view of the above problems, the inventor found after long-term research, and proposed the display method, device, head-mounted augmented reality (AR, Augmented Reality) device and storage medium provided by the embodiment of the present application. The display screen of the head-mounted augmented reality device is divided into two areas to display the target content image and the virtual road condition image corresponding to the vehicle's driving state, so that the user can watch the target content image while viewing the vehicle's driving state. Virtual road condition images to synchronize eye-sight information and inner ear information to achieve the effect of preventing motion sickness. Wherein, the specific display method will be described in detail in the following embodiments.

The following will introduce the application scenarios that can be used for the display method provided by the embodiment of the present application.

Please refer to FIG. 1. FIG. 1 shows a schematic diagram of an application scenario of the display method provided by the embodiment of the present application. The application scenario includes a head-mounted augmented reality device 100 and a vehicle 200. As shown in FIG. 1, wearing the headset A user of the augmented reality device 100 may sit in the passenger seat of the vehicle 200.

Please refer to FIG. 2. FIG. 2 shows a schematic diagram of another application scenario of the display method provided by the embodiment of the present application. The application scenario includes a head-mounted augmented reality device 100 and a vehicle 200. As shown in FIG. The user of the wearable augmented reality device 100 can sit in the rear seat of the vehicle 200 .

Of course, the user wearing the head-mounted augmented reality device 100 can also sit in other positions in the vehicle 200 , which will not be repeated here.

In this embodiment of the application, the head-mounted augmented reality device 100 may be an integrated head-mounted display device, and the head-mounted augmented reality device 100 may also be an intelligent terminal such as a mobile phone connected to an external head-mounted display device, that is, a head-mounted augmented reality device. The augmented reality device 100 can be used as a processing and storage device of a head-mounted display device, plugged into or connected to an external head-mounted display device, and can display virtual objects in the head-mounted display device.

Referring to FIG. 3 , FIG. 3 shows a schematic flowchart of a display method provided by an embodiment of the present application. The display method is used to divide the display screen of the head-mounted augmented reality device into two areas to display the target content image and the virtual road condition image corresponding to the driving state of the vehicle in a driving vehicle, so that the user can watch the target content. At the same time as the image, you can watch the virtual road condition image corresponding to the driving state of the vehicle, so that the visual information and the inner ear information can be synchronized to achieve the effect of preventing motion sickness. In a specific embodiment, the display method is applied to a display device 300 as shown in FIG. 10 and a head-mounted augmented reality device 100 ( FIG. 11 ) configured with the display device 300 . The process shown in Figure 3 will be described in detail below. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include the following steps:

Step S110: When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire the driving state of the vehicle.

In this embodiment, the user can wear a head-mounted augmented reality device when riding in a vehicle. The user can sit in the front row of the vehicle or in the rear row of the vehicle, which is not limited here. Correspondingly, the head-mounted augmented reality device is located in the vehicle.

In this embodiment, the head-mounted augmented reality device can detect the state of the vehicle, and when it is detected that the state of the vehicle indicates that the vehicle starts to drive, the driving state of the vehicle can be obtained.

In some implementations, the head-mounted augmented reality device can detect whether it is in the vehicle, and when it is detected that it is in the vehicle, it can detect the state of the vehicle it is in. For example, the head-mounted augmented reality device may include a camera, and the head-mounted augmented reality device may detect whether it is in the vehicle through the camera; The augmented reality device can obtain information input from the outside through the information input device, and determine whether it is in the vehicle based on the information input from the outside.

In some implementations, the head-mounted augmented reality device can detect whether it is in the wearing state. When it is detected that it is in the wearing state, it can detect whether the user wearing the head-mounted augmented reality device is in the vehicle. When it is in the vehicle, it can detect the state of the vehicle it is in. For example, the head-mounted augmented reality device may include a pressure sensor, and the head-mounted augmented reality device may detect whether it is in a wearing state through the pressure sensor; A type of augmented reality device can detect whether it is in a wearing state through a contact sensor.

As a method, the head-mounted augmented reality device detecting the state of the vehicle may include: detecting the advancing distance of the vehicle within a preset time period, wherein, when it is detected that the vehicle is When the forward distance within the preset time period is less than the preset forward distance, it can be determined that the vehicle is in a parked state; When the distance is less than the preset distance, it can be determined that the vehicle starts to drive.

As yet another way, the head-mounted augmented reality device detecting the state of the vehicle may include: detecting whether the vehicle is turned off, and when it is detected that the vehicle is turned off, it may be determined that the vehicle is in a parked state , when it is detected that the vehicle changes from being turned off to not being turned off, it can be determined that the vehicle starts to drive.

As yet another way, the head-mounted augmented reality device detecting the state of the vehicle may include: the head-mounted augmented reality device may include a motion sensor, and the head-mounted augmented reality device may monitor the state of the vehicle through its motion sensor. The state of the vehicle is detected, and when the target motion parameter is detected by the motion sensor, it can be determined that the vehicle is detected to start driving.

In some implementations, the head-mounted augmented reality device and the vehicle can be connected, and then the head-mounted augmented reality device can obtain the driving status sent by the vehicle based on the connection with the vehicle. For example, the head-mounted augmented reality device may be connected to the vehicle-mounted terminal of the vehicle, and the head-mounted augmented reality device may obtain the driving status sent by the vehicle-mounted terminal based on the connection to the vehicle-mounted terminal.

In some implementations, the head-mounted augmented reality device may include a motion sensor, and the head-mounted augmented reality device may detect and obtain the driving state of the vehicle through its motion sensor.

As an implementable manner, the driving state of the vehicle may include, but is not limited to: a speed change of the vehicle and a driving direction of the vehicle.

Step S120: Obtain a virtual road condition image corresponding to the driving state.

In some implementations, after the driving state of the vehicle is acquired, a virtual road condition image corresponding to the driving state of the vehicle may be acquired. For example, when it is acquired that the driving state of the vehicle is accelerating forward, its corresponding virtual road condition image can be acquired as the road accelerating backward; Forward; when the driving state of the vehicle is obtained as turning left, the corresponding virtual road condition image can be obtained as the road winds to the left, and the virtual road condition image will switch to the content seen from the right perspective at the same time; when the vehicle is obtained When the driving state of the user is turning right, the corresponding virtual road condition image can be obtained, which shows that the road is winding to the right, and the virtual road condition image will be switched to the content seen from the left perspective at the same time.

As an implementable manner, a mapping relationship table can be established in advance as a preset mapping relationship table, and the preset mapping relationship table includes multiple driving states, multiple virtual road condition images, and multiple driving states and multiple virtual road conditions Correspondence between images, wherein multiple driving states and multiple virtual road condition images can be in one-to-one correspondence. As shown in Table 1, the preset mapping relationship table may include driving state A1, driving state A2, driving state A3, virtual road condition image B1, virtual road condition image B2, virtual road condition image B3, wherein, driving state A1 and virtual road condition There is a corresponding relationship between the image B1, the corresponding relationship between the driving state A2 and the virtual road condition image B2, and the corresponding relationship between the driving state A3 and the virtual road condition image B3.

Table 1

行驶状态ADriving state A	虚拟路况图像BVirtual traffic image B
A1A1	B1B1
A2A2	B2B2
A3A3	B3B3
……...	……...

Therefore, in this embodiment, after the driving state of the vehicle is acquired, a virtual road condition image corresponding to the driving state of the vehicle may be acquired based on a preset mapping relationship table. For example, when it is acquired that the driving state of the vehicle matches the driving state A1, the virtual road condition image corresponding to the vehicle's driving state may be determined as the virtual road condition image B1 based on the preset mapping relationship table.

Step S130: Displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

In this embodiment, the target content image represents the actual content image desired by the user, for example, it may be a video image, a game image, etc.; the virtual road condition image is a virtual road image matching the vehicle's motion state. In some embodiments, the target content image may be stored locally on the head-mounted augmented reality device, and the target content image may also be projected by an electronic device (such as a smart phone) connected to the head-mounted augmented reality device, so as to realize the content is displayed in an augmented reality headset, preventing motion sickness from using a smartphone in a moving vehicle.

In this embodiment, the display area of the display screen of the head-mounted augmented reality device includes a first display area and a second display area. Wherein, the size relationship between the first display area and the second display area can be manually set by the user, or can be automatically set by the head-mounted augmented reality device. For example, the first display area can be larger than the second display area, or smaller than the second display area. The second display area, etc.; the positional relationship between the first display area and the second display area can be manually set by the user, or can be automatically set by the head-mounted augmented reality device, for example, the first display area can be located in the second display area , may also be surrounded by the second display area, etc., which is not limited here.

As an implementable manner, the second display area is located around the first display area. For example, the second display area can be located on the left and right sides of the first display area; the second display area can be located on the left and right sides and the lower side of the first display area; side; the second display area may be located on the right side, lower side, etc. of the first display area. Based on this, the target content image is displayed in the first display area, and the virtual road image is displayed around the target content image, while the user is watching the target content image, the remaining light can see the surrounding virtual road image, which can make the brain perceive Real motion status to prevent motion sickness.

In some implementations, when the head-mounted augmented reality device is in the wearing state, the second display area is located at the two sides and the bottom side of the first display area. Based on this, the target content image is displayed in the first display area, and the virtual road image is displayed on the left, right and bottom sides of the target content image, so the user can see the left, right and left sides while viewing the target content image. The virtual road images on the side and bottom can make the brain perceive the real state of motion, thereby preventing motion sickness. As a way, the first display area can correspond to the large upper middle area, and the second display area can correspond to the left narrow area, right narrow area, and bottom narrow area. The force is mainly concentrated on the target content image displayed in the large middle and upper area, and the virtual road condition image displayed in the left narrow area, right narrow area and bottom narrow area can be seen in the peripheral vision of the eyes, so that the brain perceives the real state of motion , thereby preventing motion sickness.

Please refer to FIG. 4 , which shows a schematic diagram of a display screen of a head-mounted augmented reality device provided by an embodiment of the present application. As shown in FIG. 4, the display area of the display screen 130 is divided into a first display area 131 and a second display area 132, wherein the first display area 131 can be used to display target content images, and the second display area 132 can be Used to display virtual road images.

In some implementations, the display screen of the head-mounted augmented reality device can be divided into the first display area and the second display area by default, then after obtaining the virtual road condition image corresponding to the driving state, the A display area displays the target content image, and a virtual road condition image is displayed in the second display area of the display screen.

In some implementations, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, the display screen of the head-mounted augmented reality device may be divided into a first display area while acquiring the driving state of the vehicle and the second display area, after acquiring the virtual road condition image corresponding to the driving state, the target content image can be displayed in the first display area of the display screen, and the virtual road condition image can be displayed in the second display area of the display screen.

According to the display method provided by an embodiment of the present application, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, it obtains the driving state of the vehicle, obtains a virtual road condition image corresponding to the driving state, and displays it on the first screen of the display screen. The target content image is displayed in the display area, and the virtual road condition image is displayed in the second display area of the display screen, so that in the moving vehicle, the display screen of the head-mounted augmented reality device is divided into two areas to display the target content image and The virtual road condition image enables the user to watch the virtual road condition image corresponding to the driving state of the vehicle while watching the target content image, so that the eye-sight information and the inner ear information are synchronized to achieve the effect of preventing motion sickness.

Referring to FIG. 5 , FIG. 5 shows a schematic flowchart of a display method provided by another embodiment of the present application. This method is applied to the above-mentioned head-mounted augmented reality device, and the following will elaborate on the process shown in Figure 5 in detail. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include The following steps:

Step S210: When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire motion parameters of the head-mounted augmented reality device.

In this embodiment, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, the motion parameters of the head-mounted augmented reality device may be detected and acquired.

As one way, the head-mounted augmented reality device may include a motion sensor, and then the head-mounted augmented reality device may detect the motion state of the head-mounted augmented reality device in real time through its motion sensor. When the sensor determines that the head-mounted augmented reality device is in a wearing state in a driving vehicle, the motion sensor may detect and acquire motion parameters of the head-mounted augmented reality device.

Step S220: Obtain the driving state of the vehicle based on the motion parameters of the head-mounted augmented reality device.

Wherein, the motion parameters of the head-mounted augmented reality device may include two components, one is the motion parameters of the vehicle, and the other is the motion parameters of the user's head. Therefore, in this embodiment, after the motion parameters of the head-mounted augmented reality device are obtained, the motion parameters of the vehicle can be separated from the motion parameters of the head-mounted augmented reality device, and the vehicle's driving motion can be obtained based on the motion parameters of the vehicle. state.

Please refer to FIG. 6 , which shows a schematic flowchart of step S220 of the display method shown in FIG. 5 of the present application. The process shown in Figure 6 will be described in detail below, and the method may specifically include the following steps:

Step S221: Perform filtering processing on the motion parameters of the head-mounted augmented reality device, and separate the motion parameters of the vehicle from the motion parameters of the head-mounted augmented reality device.

Wherein, the motion parameters of the head-mounted augmented reality device may include two components, one is the motion parameters of the vehicle, and the other is the motion parameters of the user's head. Therefore, in some implementation manners, after acquiring the motion parameters of the head-mounted augmented reality device, the motion parameters of the head-mounted augmented reality device may be filtered to separate the The motion parameters of the vehicle. That is to say, since the motion parameters of the head-mounted augmented reality device include the motion parameters of the vehicle and the motion parameters of the user's head, the motion parameters of the head-mounted augmented reality device can be filtered to filter out the head-mounted augmented reality device. Among the motion parameters of the augmented reality device, the motion parameters of the user's head are retained and the motion parameters of the vehicle are retained.

Among them, there are two main differences between the motion parameters of the vehicle and the motion parameters of the user's head: First, the natural frequency of the vehicle is different from the natural frequency of the user's head. Each object has its own natural frequency, such as the vehicle The vibration of the whole vehicle caused by the vibration of the engine when starting is the natural frequency of the vehicle, and this vibration frequency will always exist as long as the vehicle is started. The natural frequency of mechanical equipment is generally high, while the natural frequency of human head movement is low, and there is no intersection between the two; second, the motion parameters of the vehicle can be described by corresponding mathematical models, such as the acceleration of the vehicle , deceleration, left turn, right turn, etc. all correspond to different motion parameters, while the motion parameters of the user's head have no general rules to describe. Therefore, the motion parameters of the head-mounted augmented reality device can be filtered according to the difference between the motion parameters of the vehicle and the motion parameters of the user's head, and the motion parameters of the vehicle can be separated from the motion parameters of the head-mounted augmented reality device.

In some implementations, since there is no intersection between the natural frequency of the vehicle and the natural frequency of the user's head, the motion parameters of the head-mounted augmented reality device may be filtered based on the natural frequency of the vehicle, and the head-mounted augmented reality device The motion parameters of the vehicle are separated from the motion parameters of the augmented reality device.

Step S222: Perform fitting processing on the motion parameters of the vehicle to obtain the driving state of the vehicle.

In some implementations, after the motion parameters of the vehicle are acquired from the motion parameters of the head-mounted augmented reality device, fitting processing may be performed on the acquired motion parameters of the vehicle to obtain the driving state of the vehicle. As a way, after the motion parameters of the vehicle are obtained from the motion parameters of the head-mounted augmented reality device, the motion parameters of the vehicle can be fitted to restore the motion information of the vehicle, so as to obtain the driving state of the vehicle.

It can be understood that the method of obtaining the driving state of the vehicle in this embodiment is active acquisition, relying on the motion sensor on the head-mounted augmented reality device to detect the motion parameters of the head-mounted augmented reality device, and restore the result by filtering algorithm and data fitting The real driving status of the vehicle does not require the vehicle manufacturer to provide a data interface, which can improve universality and promotion.

Step S230: Obtain a virtual road condition image corresponding to the driving state.

Step S240: displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

Wherein, for the specific description of step S230-step S240, please refer to step S120-step S130, which will not be repeated here.

In the display method provided by another embodiment of the present application, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, the motion parameters of the head-mounted augmented reality device are acquired, based on the motion of the head-mounted augmented reality device parameter, acquire the driving state of the vehicle, obtain the virtual road image corresponding to the driving state, display the target content image in the first display area of the display screen, and display the virtual road image in the second display area of the display screen. Compared with the display method shown in Figure 3, this embodiment also acquires the motion parameters of the head-mounted augmented reality device, and determines the driving state of the vehicle according to the motion parameters, thereby relying on the head-mounted augmented reality device to actively detect its motion parameters By obtaining the driving status of the vehicle, the dependence on the data interface of the vehicle manufacturer can be avoided, and the universality and popularization can be improved.

Referring to FIG. 7 , FIG. 7 shows a schematic flowchart of a display method provided by another embodiment of the present application. This method is applied to the above-mentioned head-mounted augmented reality device. The following will elaborate on the process shown in FIG. 7 in detail. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include The following steps:

Step S310: Displaying the target content image in a display area of the display screen, wherein the target content image completely covers the display area, and the display area includes the first display area and the second display area .

In this embodiment, when the head-mounted augmented reality device is turned on, if it is not in the vehicle in the driving state, it can display the target content image in full screen in the display area of the display screen, that is, it can display the target content image in the display area including the first The display areas of the first display area and the second display area display the target content image, and the target content image completely covers the display area.

In some embodiments, when the head-mounted augmented reality device is turned on, it can detect whether it is in the vehicle. Among them, when it is detected that it is not in the vehicle, the target content image can be displayed in full screen in the display area of the display screen; when it is detected that it is in the vehicle, the state of the vehicle can be detected; When the vehicle starts to drive, the target content image originally displayed in full screen in the display area can be switched to be displayed in the first display area of the display area, and a virtual road condition image can be displayed in the second display area of the display area.

As an implementable manner, during the full-screen display of the target content image by the head-mounted augmented reality device, if it is detected that the vehicle in which the head-mounted augmented reality device is located starts to run, prompt information can be output, wherein the The prompt information is used to prompt the user to select whether to switch the target content image from full-screen display to display in the first display area. Wherein, when the instruction information for confirming switching output by the user based on the prompt information is received, the target content image originally displayed in the display area in full screen may be switched to be displayed in the first display area in the display area, and displayed in the display area The virtual road condition image is displayed in the second display area; when receiving the instruction information of rejecting switching output by the user based on the prompt information, the target content image can be displayed in full screen in the display area.

Step S320: When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire the driving state of the vehicle.

Step S330: Obtain a virtual road condition image corresponding to the driving state.

Step S340: displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

Wherein, for the specific description of step S320-step S340, please refer to step S110-step S130, which will not be repeated here.

In the display method provided by another embodiment of the present application, the target content image is displayed in the display area of the display screen, wherein the target content image completely covers the display area, and the display area includes the first display area and the second display area. When the wearable augmented reality device detects that the vehicle starts to drive, it obtains the driving state of the vehicle, obtains the virtual road condition image corresponding to the driving state, displays the target content image in the first display area of the display screen, and displays the target content image in the second display area of the display screen. Virtual traffic image. Compared with the display method shown in FIG. 3, this embodiment can also display the target content image in full screen in advance, and then switch to the first display area to display the target content image, and display the virtual road condition image in the second display area to suit Match different usage scenarios.

Please refer to FIG. 8 , which shows a schematic flowchart of a display method provided by another embodiment of the present application. This method is applied to the above-mentioned head-mounted augmented reality device. The process shown in FIG. 8 will be described in detail below. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include The following steps:

Step S410: When the head-mounted augmented reality device is in the first mode, display the target content image in the display area of the display screen, wherein the target content image completely covers the display area, and the display The regions include the first display region and the second display region.

In this embodiment, when the head-mounted augmented reality device is turned on, it can detect the mode it is in, and when it is detected that the mode it is in is the first mode (normal mode), it can display The target content image can be displayed in full screen in the display area of the screen, that is, the target content image can be displayed in the display area including the first display area and the second display area, and the target content image completely covers the display area.

In some implementations, when the head-mounted augmented reality device is turned on, if it is not in a vehicle in a driving state, its mode can be automatically set to the first mode, or a prompt message can be output to prompt the user to set the Its mode is set to the first mode, and when it is determined that the head-mounted augmented reality device is in the first mode, the target content image can be displayed in full screen in the display area of the display screen.

In some embodiments, when the head-mounted augmented reality device is turned on, it can detect whether it is in the vehicle. Wherein, when it is detected that it is not in the vehicle, its mode can be automatically set to the first mode, or prompt information can be output to prompt the user to set its mode to the first mode. In the first mode, the target content image can be displayed in full screen in the display area of the display screen. When it is detected that it is in the vehicle, the state of the vehicle can be detected, and when it is detected that the state of the vehicle is a parking state, its mode can be automatically set to the first mode, or a prompt message can be output To prompt the user to set its mode to the first mode, when it is determined that the head-mounted augmented reality device is in the first mode, the target content image may be displayed in full screen in the display area of the display screen.

Step S420: When it is detected that the vehicle starts to drive and the head-mounted augmented reality device is switched from the first mode to the second mode, acquire the driving state of the vehicle.

As a method, when it is detected that the vehicle in which the head-mounted augmented reality device is located starts to drive, the mode in which the head-mounted augmented reality device is located can be automatically set to the second mode (vehicle mode). When the augmented reality device is in the second mode, it can acquire the driving state of the vehicle, display the target content image in the first display area of the display area, and display the virtual road conditions corresponding to the driving state in the second display area of the display area image.

As yet another way, when it is detected that the vehicle in which the head-mounted augmented reality device is located starts to drive, a prompt message may be output to prompt the user to set the mode of the head-mounted augmented reality device as the second mode, and after determining When the head-mounted augmented reality device is in the second mode, it can acquire the driving state of the vehicle, display the target content image in the first display area of the display area, and display the corresponding information corresponding to the driving state in the second display area of the display area. Virtual traffic image.

As yet another way, when it is detected that the vehicle in which the head-mounted augmented reality device is located starts to drive and the mode in which the head-mounted augmented reality device is located is the first mode, then the The mode is switched from the first mode (normal mode) to the second mode (vehicle mode). When it is determined that the head-mounted augmented reality device is in the second mode, the driving state of the vehicle can be obtained, and the target that was originally displayed in the display area in full screen The content image is switched to be displayed in the first display area of the display area, and the virtual road condition image corresponding to the driving state is displayed in the second display area of the display area.

Step S430: Obtain a virtual road condition image corresponding to the driving state.

Step S440: displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

Wherein, for the specific description of step S430-step S440, please refer to step S120-step S130, which will not be repeated here.

In the display method provided by another embodiment of the present application, when the head-mounted augmented reality device is in the first mode, the target content image is displayed in the display area of the display screen, wherein the target content image completely covers the display area, and the display area includes the first In the display area and the second display area, when it is detected that the vehicle starts to drive and the head-mounted augmented reality device is switched from the first mode to the second mode, the driving state of the vehicle is obtained, and the virtual road condition image corresponding to the driving state is obtained. The target content image is displayed in the first display area of the display screen, and the virtual road condition image is displayed in the second display area of the display screen. Compared with the display method shown in FIG. 3 , this embodiment also adopts different display strategies in different modes, so that users can obtain good experience in different usage scenarios.

Referring to FIG. 9 , FIG. 9 shows a schematic flowchart of a display method provided by still another embodiment of the present application. This method is applied to the above-mentioned head-mounted augmented reality device. The flow shown in FIG. 9 will be described in detail below. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include The following steps:

Step S510: When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire the driving state of the vehicle.

Wherein, for the specific description of step S510, please refer to step S110, which will not be repeated here.

Step S520: Select a virtual road from multiple virtual roads as the target virtual road in response to the virtual road selection instruction.

As an implementable manner, the plurality of virtual roads may include, but is not limited to: highways, country lanes, mountain roads, and urban roads.

In some implementations, the head-mounted augmented reality device can be preset and stored with multiple virtual roads, and when the head-mounted augmented reality device detects that the vehicle it is in starts to drive, it can display a virtual road selection control, which The virtual road selection control is used for the user to select one virtual road from multiple virtual roads as the target virtual road, so as to increase the user's selection diversity of virtual roads. Wherein, during the process of displaying the virtual road selection control, when a virtual road selection instruction is received, a virtual road may be selected from a plurality of virtual roads as the target virtual road in response to the virtual road selection instruction.

In some implementations, the head-mounted augmented reality device can be preset and stored with multiple virtual roads, and when the head-mounted augmented reality device detects that the vehicle it is on starts to drive, Detecting, determining a virtual road matching the detected road as a target virtual road from a plurality of virtual roads according to the detected road. For example, when it is detected that the road on which the vehicle is driving is a highway, then "highway" can be determined from multiple virtual roads as the target virtual road; Among the virtual roads, "country lane" is determined as the target virtual road.

Step S530: Obtain the virtual road condition image based on the driving state and the target virtual road.

In some implementations, after the target virtual road is acquired, a virtual road condition image may be acquired based on the driving state of the vehicle and the target virtual road. For example, when the target virtual road is a highway and the driving state of the vehicle is accelerating forward, then the corresponding virtual road condition image can be acquired as expressway accelerating backward; when the target virtual road is a country lane and the driving state of the vehicle is backward, Then the corresponding virtual road condition image can be obtained as a country road; when the target virtual road is a winding road and the driving state of the vehicle is turning left, the corresponding virtual road condition image can be obtained as a winding road winding to the left, and the virtual At the same time, the road condition image will be switched to the content seen from the right perspective; when the target virtual road is a winding road and the driving state of the vehicle is turning right, the corresponding virtual road condition image can be obtained as the winding road winding to the right, and the virtual The traffic image will also switch to the content seen from the left perspective.

Step S540: displaying the target content image in the first display area of the display screen, and displaying the virtual road condition image in the second display area of the display screen.

Wherein, for the specific description of step S540, please refer to step S130, which will not be repeated here.

In the display method provided by still another embodiment of the present application, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, it acquires the driving state of the vehicle, and selects from multiple virtual roads in response to a virtual road selection instruction. A virtual road is used as the target virtual road, based on the driving state and the target virtual road, a virtual road condition image is obtained, the target content image is displayed in the first display area of the display screen, and the virtual road condition image is displayed in the second display area of the display screen. Compared with the display method shown in FIG. 3 , this embodiment also selects a target virtual road from a plurality of virtual roads according to a virtual road selection instruction, so as to improve the user's autonomous choice.

Referring to FIG. 10 , FIG. 10 shows a schematic flowchart of a display method provided in yet another embodiment of the present application. This method is applied to the above-mentioned head-mounted augmented reality device. The following will elaborate on the flow shown in FIG. 10 in detail. In this embodiment, the head-mounted augmented reality device includes a display screen, and the display method may specifically include The following steps:

Step S610: When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire the driving state of the vehicle.

Step S620: Obtain a virtual road condition image corresponding to the driving state.

Wherein, for the specific description of step S610-step S620, please refer to step S110-step S130, which will not be repeated here.

Step S630: Select a content image from multiple content images as a target content image in response to the content image selection instruction.

As an implementable manner, the plurality of content images may include, but is not limited to: game content images, video content images, news content images, and chat content images.

In some implementations, the head-mounted augmented reality device or the electronic device connected to the head-mounted augmented reality device can be preset and stored with multiple content images, then when the head-mounted augmented reality device is turned on, the content images can be displayed A selection control, the content image selection control is used for the user to select a content image from multiple content images as a target content image, so as to increase the diversity of selection of content images by the user. Wherein, during the process of displaying the content image selection control, when a content image selection instruction is received, a content image may be selected from multiple content images as the target content image in response to the content image selection instruction.

In some implementations, the head-mounted augmented reality device or the electronic device connected to the head-mounted augmented reality device can be preset and stored with a plurality of content images, then when the head-mounted augmented reality device is turned on, the head-mounted augmented reality device can The content image displayed last time by the augmented reality device is detected, and the content image matching the content image displayed last time is selected from the plurality of content images as the target content image.

In some implementations, the head-mounted augmented reality device or the electronic device connected to the head-mounted augmented reality device can be preset and stored with a plurality of content images, then when the head-mounted augmented reality device is turned on, the head-mounted augmented reality device can The augmented reality device detects the content image that is displayed the most times, and selects the content image that matches the content image that is displayed the most times from the plurality of content images as the target content image.

Step S640: Displaying the target content image in the first display area of the display screen.

Step S650: Displaying the virtual road condition image in the second display area of the display screen.

Wherein, for the specific description of step S640-step S650, please refer to step S130, which will not be repeated here.

In the display method provided by yet another embodiment of the present application, when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, it acquires the driving state of the vehicle, acquires a virtual road condition image corresponding to the driving state, and responds to the content selection An instruction is to select a content image from multiple content images as a target content image, display the target content image in the first display area of the display screen, and display a virtual road condition image in the second display area of the display screen. Compared with the display method shown in FIG. 3 , this embodiment also selects a target content image from a plurality of content images according to a content image selection instruction, so as to improve the user's autonomous selectivity.

Please refer to FIG. 11 , which shows a block diagram of modules of a display device provided by an embodiment of the present application. The display device 300 is applied to the above-mentioned head-mounted augmented reality device. The block diagram shown in FIG. 11 will be described below. In this embodiment, the head-mounted augmented reality device includes a display screen, and the displayed display device 300 includes: Driving state acquisition module 310, virtual road condition image acquisition module 320 and display module 330, wherein:

The driving state acquisition module 310 is configured to acquire the driving state of the vehicle when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive.

Further, the driving state acquisition module 310 includes: a motion parameter acquisition submodule and a first driving state acquisition submodule, wherein:

The motion parameter acquisition sub-module is used to acquire the motion parameters of the head-mounted augmented reality device.

The first driving state acquiring submodule is configured to acquire the driving state of the vehicle based on the motion parameters of the head-mounted augmented reality device.

Further, the driving state acquisition sub-module includes: a motion parameter acquisition unit and a driving state acquisition unit, wherein:

The motion parameter acquisition unit is configured to filter the motion parameters of the head-mounted augmented reality device, and separate the motion parameters of the vehicle from the motion parameters of the head-mounted augmented reality device.

Further, the motion parameter acquisition unit includes: a motion parameter acquisition unit, wherein:

A motion parameter acquisition unit, configured to filter the motion parameters of the head-mounted augmented reality device based on the natural frequency of the vehicle, and separate the motion of the vehicle from the motion parameters of the head-mounted augmented reality device parameter.

The driving state acquisition unit is configured to perform fitting processing on the motion parameters of the vehicle to obtain the driving state of the vehicle.

Further, the driving state obtaining module 310 includes: a second driving state obtaining sub-module, wherein:

The second driving state acquisition sub-module is configured to acquire the driving state of the vehicle when it is detected that the vehicle starts to drive and the head-mounted augmented reality device is switched from the first mode to the second mode.

The virtual road condition image acquisition module 320 is configured to acquire a virtual road condition image corresponding to the driving state.

Further, the virtual road condition image acquisition module 320 includes: a target virtual road selection submodule and a virtual road condition image acquisition submodule, wherein:

The target virtual road selection submodule is configured to select a virtual road from a plurality of virtual roads as the target virtual road in response to a virtual road selection instruction.

The virtual road condition image acquisition submodule is configured to obtain the virtual road condition image based on the driving state and the target virtual road.

The display module 330 is configured to display the target content image in the first display area of the display screen, and display the virtual road condition image in the second display area of the display screen.

Further, the display module 330 includes: a target content image selection submodule and a display submodule, wherein:

The target content image selection submodule is configured to select a content image from multiple content images as the target content image in response to the content image selection instruction.

A display submodule, configured to display the target content image in the first display area of the display screen.

Further, the display device 300 also includes: a target content image display module, wherein:

A target content image display module, configured to display the target content image in a display area of the display screen, wherein the target content image completely covers the display area, and the display area includes the first display area and the the second display area.

Further, the target content image display module includes: a target content image display submodule, wherein:

The target content image display submodule is configured to display the target content image in the display area of the display screen when the head-mounted augmented reality device is in the first mode.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the devices and modules described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.

Please refer to FIG. 12 , which shows a structural block diagram of a head-mounted augmented reality device 100 provided by an embodiment of the present application. The head-mounted augmented reality device 100 may be an electronic device capable of running application programs, such as a smart phone, a tablet computer, and an e-book. The head-mounted augmented reality device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, a display screen 130, and one or more application programs, wherein one or more application programs may be stored in the memory 120 and is configured to be executed by one or more processors 110, and one or more programs are configured to execute the methods described in the foregoing method embodiments.

Wherein, the processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts in the whole head-mounted augmented reality device 100, and runs or executes instructions, programs, code sets or instruction sets stored in the memory 120, and calls stored in the memory 120. Execute various functions of the head-mounted augmented reality device 100 and process data. Optionally, the processor 110 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the content to be displayed; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 110, but may be realized by a communication chip alone.

The memory 120 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area can also store data created by the head-mounted augmented reality device 100 during use (such as phonebook, audio and video data, chat record data) and the like.

The display screen 130 is used to display information input by the user, information provided to the user, and various graphical user interfaces of the head-mounted augmented reality device 100. These graphical user interfaces can be composed of graphics, text, icons, numbers, videos, and In one example, the display screen 130 can be a liquid crystal display (Liquid Crystal Display, LCD) or an organic light-emitting diode (Organic Light-Emitting Diode, OLED), which is not limited here.

Please refer to FIG. 13 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program codes are stored in the computer-readable medium 400, and the program codes can be invoked by a processor to execute the methods described in the foregoing method embodiments.

The computer readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 400 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium). The computer-readable storage medium 400 has a storage space for program code 410 for executing any method steps in the above methods. These program codes can be read from or written into one or more computer program products. Program code 410 may, for example, be compressed in a suitable form.

To sum up, the display method, device, head-mounted augmented reality device, and storage medium provided by the embodiments of the present application divide the display screen of the head-mounted augmented reality device into two areas to display The target content image and the virtual road condition image corresponding to the driving state of the vehicle, so that the user can watch the virtual road condition image corresponding to the driving state of the vehicle while viewing the target content image, so that the visual information and the inner ear information are synchronized to prevent motion sickness Effect.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not drive the essence of the corresponding technical solutions away from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

A display method, characterized in that it is applied to a head-mounted augmented reality device, the head-mounted augmented reality device includes a display screen, and the method includes:

When the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, acquire the driving state of the vehicle;

Acquiring a virtual road condition image corresponding to the driving state;

The target content image is displayed in the first display area of the display screen, and the virtual road condition image is displayed in the second display area of the display screen.
The method according to claim 1, wherein the acquiring the driving state of the vehicle comprises:

Acquiring motion parameters of the head-mounted augmented reality device;

Acquiring the driving state of the vehicle based on the motion parameters of the head-mounted augmented reality device.
The method according to claim 2, wherein the acquiring the driving state of the vehicle based on the motion parameters of the head-mounted augmented reality device comprises:

performing filtering processing on the motion parameters of the head-mounted augmented reality device, and separating the motion parameters of the vehicle from the motion parameters of the head-mounted augmented reality device;

Fitting processing is performed on the motion parameters of the vehicle to obtain the driving state of the vehicle.
The method according to claim 3, wherein the filtering process is performed on the motion parameters of the head-mounted augmented reality device, and the motion parameters of the vehicle are separated from the motion parameters of the head-mounted augmented reality device. Movement parameters, including:

The motion parameters of the head-mounted augmented reality device are filtered based on the natural frequency of the vehicle, and the motion parameters of the vehicle are separated from the motion parameters of the head-mounted augmented reality device.
The method according to claim 2, wherein the head-mounted augmented reality device includes a motion sensor, and the acquiring motion parameters of the head-mounted augmented reality device comprises:

Acquiring motion parameters of the head-mounted augmented reality device through the motion sensor.
The method according to claim 1, wherein the acquiring the driving state of the vehicle comprises:

Detecting the position of the head-mounted augmented reality device;

When it is detected that the head-mounted augmented reality device is in the vehicle, the driving state of the vehicle is acquired.
The method according to claim 6, wherein the detecting the position of the head-mounted augmented reality device comprises:

Detecting the state of the head-mounted augmented reality device;

When it is detected that the head-mounted augmented reality device is in the wearing state, the position of the head-mounted augmented reality device is detected.
The method according to any one of claims 1-7, wherein when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive, before acquiring the driving state of the vehicle, Also includes:

The target content image is displayed in a display area of the display screen, wherein the target content image completely covers the display area, and the display area includes the first display area and the second display area.
The method according to claim 8, wherein the displaying the target content image in the display area of the display screen comprises:

When the head-mounted augmented reality device is in the first mode, displaying the target content image in the display area of the display screen;

The acquiring the driving state of the vehicle when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive includes:

When it is detected that the vehicle starts to drive and the head-mounted augmented reality device is switched from the first mode to the second mode, the driving state of the vehicle is acquired.
The method according to any one of claims 1-7, wherein said acquiring a virtual road condition image corresponding to said driving state comprises:

Selecting a virtual road from a plurality of virtual roads as a target virtual road in response to a virtual road selection instruction;

The virtual road condition image is obtained based on the driving state and the target virtual road.
The method according to any one of claims 1-7, wherein the displaying the target content image in the first display area of the display screen comprises:

selecting a content image from a plurality of content images as a target content image in response to a content image selection instruction;

The target content image is displayed in the first display area of the display screen.
The method according to claim 11, wherein the selecting a content image from a plurality of content images as the target content image in response to the content image selection instruction comprises:

Display content image selection control;

Selecting a content image from the plurality of content images as the target content image in response to a content image selection instruction triggered based on the content image selection control.
The method according to claim 11, wherein the plurality of content images include at least one of game content images, video content images, news content images and chat content images.
The method according to any one of claims 1-7, wherein the second display area is located around the first display area.
The method according to claim 14, wherein when the head-mounted augmented reality device is in a wearing state, the second display area is located on both sides and the bottom side of the first display area.
The method according to any one of claims 1-7, wherein the acquiring the driving state of the vehicle comprises:

Acquiring the forward distance of the vehicle within a preset time period;

Based on the forward distance of the vehicle within a preset time period, the driving state of the vehicle is acquired.
The method according to any one of claims 1-7, wherein the driving state of the vehicle includes at least one of a speed change of the vehicle and a driving direction of the vehicle.
A display device, characterized in that it is applied to a head-mounted augmented reality device, the head-mounted augmented reality device includes a display screen, and the device includes:

A driving state acquiring module, configured to acquire the driving state of the vehicle when the head-mounted augmented reality device in the vehicle detects that the vehicle starts to drive;

A virtual road condition image acquisition module, configured to acquire a virtual road condition image corresponding to the driving state;

A display module, configured to display the target content image in the first display area of the display screen, and display the virtual road condition image in the second display area of the display screen.
A head-mounted augmented reality device is characterized in that it includes a display screen, a memory and a processor, the display screen and the memory are coupled to the processor, the memory stores instructions, and when the instructions are executed by the When the processor executes, the processor executes the method according to any one of claims 1-17.
A computer-readable storage medium, characterized in that program codes are stored in the computer-readable storage medium, and the program codes can be invoked by a processor to execute the method according to any one of claims 1-17 .