WO2022198980A1

WO2022198980A1 - Control method for head-mounted display device, head-mounted display device, and storage medium

Info

Publication number: WO2022198980A1
Application number: PCT/CN2021/121998
Authority: WO
Inventors: 段涛
Original assignee: 歌尔股份有限公司
Priority date: 2021-03-26
Filing date: 2021-09-30
Publication date: 2022-09-29
Also published as: CN112887703A

Abstract

The present application provides a control method for a head-mounted display device, the head-mounted display device, and a storage medium. The head-mounted display device comprises a signal processing module, a communication module in signal connection with the signal processing module, a sound production frame, and a display lens module. The method comprises: obtaining an operating state of the display lens module; when it is determined that the operating state of the display lens is a working state, controlling a second communication unit to receive, by means of a second preset communication protocol, image data sent by an external terminal, and send the image data to the processing module; controlling a first communication unit to receive, by means of a first preset communication protocol, audio data sent by an external terminal, and send the audio data to the processing module; and controlling the signal processing module to parse the audio data sent by a first communication module and the image data sent by a second communication module. The head-mounted display device provided by the present application can realize real-time transmission of audio and video data by means of wireless transmission.

Description

Control method of head-mounted display device, head-mounted display device, and storage medium

This application claims the priority of the Chinese patent application filed on March 26, 2021 with the application number 202110329828.1 and the invention titled "Control Method of Head Mounted Display Device, Head Mounted Display Device and Storage Medium", The entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of wearable setting control, and in particular, to a control method of a head-mounted display device, a head-mounted display device, and a storage medium.

Background technique

Augmented Reality technology is a technology that integrates virtual information with the real world. Through multimedia, 3D modeling, real-time tracking, intelligent interaction, sensing and other technical means, Models, music, videos and other virtual information are simulated and applied to the real world, and the two kinds of information complement each other, thereby realizing the "enhancement" of the real world.

In the prior art, AR technology is implemented by wirelessly connecting one or more external devices through a head-mounted display device. Since the existing head-mounted display device is limited by factors such as its own size and weight, the traditional wireless transmission method has high power consumption, The signal processing equipment cannot be used due to the large size and other reasons. At the same time, due to the increasing requirements for the content displayed by the AR wearable device, the traditional wireless transmission method cannot meet the data transmission needs of the AR wearable device.

SUMMARY OF THE INVENTION

The main purpose of this application is to propose a control method for a head-mounted display device, a head-mounted display device, and a computer-readable storage medium, aiming at solving the data transmission volume of the head-mounted display device using traditional wireless transmission in the prior art Technical issues that cannot meet the working demands of the head-mounted display device.

In order to achieve the above purpose, the present application provides a control method for a head-mounted display device, the head-mounted display device includes a signal processing module, a communication module signally connected to the signal processing module, a sounding stand and a display lens module , the sounding stand is adapted to fit the user's head, the communication module includes a first communication unit and a second communication unit; the method includes:

obtaining the running status of the display lens module;

It is determined that the operating state of the display lens module is a working state, and then the second communication unit is controlled to establish a communication connection with the external terminal through the second preset communication protocol, and the image data sent by the external terminal is received through the second preset communication protocol. and send it to the signal processing module; control the first communication unit to establish a communication connection with the external terminal through the first preset communication protocol, receive the audio data sent by the external terminal through the first preset communication protocol and send it to the signal processing module;

The signal processing module is controlled to parse the audio data sent by the first communication module and the image data sent by the second communication module.

Optionally, the signal processing module includes a first signal processing unit arranged in the housing of the sounding frame and a second signal processing unit arranged in the housing of the display lens module; the control of the signal processing module The step of parsing the audio data sent by the first communication module and the image data sent by the second communication module includes:

controlling the communication module to send the image data to the second signal processing unit, controlling the second signal processing unit to parse the image data, and sending the parsed image data to the display lens module for display;

The communication module is controlled to send audio data to the first signal processing unit, the first signal processing unit is controlled to parse the audio data, and the parsed audio data is sent to the sounding stand for playback.

Optionally, after the step of acquiring the operating state of the display lens module, the method further includes:

determining that the operating state of the display lens module is a standby state;

Control the communication module to establish a communication connection with the external terminal, and control the communication module to receive the audio data sent by the external terminal;

Execute the step of controlling the communication module to send audio data to the first signal processing unit, controlling the first signal processing unit to parse the audio data, and sending the parsed audio data to the sounding stand for playback .

Optionally, the sounding frame and the display lens module are detachably connected; the step of acquiring the operating state of the display lens module includes:

Detecting in real time whether the display lens module is connected with the sounding frame;

If the display lens module is connected to the sounding frame, the running state of the display lens module is obtained.

Optionally, if the operating state of the display lens module is a standby state, the control communication module establishes a communication connection with the external terminal, and the step of controlling the communication module to receive the audio data sent by the external terminal also includes:

The second signal processing unit is controlled to enter the standby mode to operate.

The present application also provides a head-mounted display device, the head-mounted display device includes a signal processing module, a control module, a communication module, a storage module, a sounding frame and a display lens module, and the sounding frame is used to communicate with The user's head is adapted, the communication module includes a first communication unit and a second communication unit, the storage module stores a computer program that can run on the control module, and the computer program is controlled by the control module When executed, the steps of the control method of the head mounted display device as described above are realized.

Optionally, the signal processing module includes a first signal processing unit arranged in the housing of the sounding frame and a second signal processing unit arranged in the housing of the display lens module, the first signal processing units are respectively Signal connection with the first communication unit and the speaker and microphone of the sounding stand, and the second signal processing unit is respectively connected with the second communication unit and the display module of the display lens module.

Optionally, the sounding stand includes a connector detachably connected to the display lens module and a temple connected to the connector.

Optionally, the connector is provided with a first data transmission terminal connected to the communication module, and the display lens module is provided with a second data transmission terminal for contacting the first data transmission terminal , the communication module transmits the image information through the contacted first data transmission terminal and the second data transmission terminal.

The present application also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the control module, the steps of the above-mentioned control method for a head-mounted display device are implemented .

Optionally, the sounding stand further includes a battery connected to the signal processing module, and the first data transmission terminal is also used for connecting an external power source to charge the battery.

A control method of a head-mounted display device, a head-mounted display device and a computer-readable storage medium proposed in the present application, by setting a first communication module and a second communication module to receive audio data and image data sent by an external terminal, respectively, The data interacting with the external terminal during the operation of the head-mounted display device can be transmitted through two transmission routes, thereby reducing the data throughput of a single transmission route, and making it possible to realize wireless transmission on the head-mounted display device.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is a part of the drawings of the present application. For those of ordinary skill in the art, other drawings can also be obtained from the provided drawings without any creative effort.

FIG. 1 is a schematic flowchart of a first embodiment of a control method for a head-mounted display device of the present application;

2 is a schematic flowchart of a second embodiment of a control method for a head-mounted display device of the present application;

3 is a schematic flowchart of a third embodiment of a control method for a head-mounted display device of the present application;

4 is a schematic structural diagram of a module of the head-mounted display device of the present application;

FIG. 5 is a schematic three-dimensional structural diagram of the head-mounted display device of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The present application provides a control method for a head-mounted display device. Referring to FIG. 1 , FIG. 1 is a schematic flowchart of a first embodiment of the control method for a head-mounted display device of the present application. The head-mounted display device includes a signal processing module. , a communication module, a sounding stand and a display lens module that are signally connected to the signal processing module, the sounding stand is adapted to fit the user's head, and the communication module includes a first communication unit and a second communication unit; The method includes:

Step S10, obtaining the running state of the display lens module;

In this embodiment, the head-mounted display device may be in the shape of glasses as shown in FIG. 1 , or may be in the shape of a mask with two supporting legs, a headband, or the like. In this embodiment, the sounding stand has two support legs, the sounding stand is supported on both sides of the user's head through the two supporting legs, and the display lens module is directly or indirectly connected to the sounding stand. When the user wears the sounding stand, the display lens The module is placed in front of the user's eyes, and the two support legs are encircled to form a shape that fits the user's head.

In this embodiment, the control module, power supply structure, speaker, microphone, first communication unit, etc. that may be involved in the control method are placed in the housing of the sounding frame, the power supply structure supplies power to each electronic component, and the control module and each electronic component are connected to each other. signal connection. The second communication unit can be placed in the shell of the sounding frame or in the display lens module. The battery in the sounding frame can supply power to the display lens module, or an additional battery can be set in the display lens module to make the sound The frame and display lens modules can work independently. The running state of the display lens module includes working state and standby state, and the standby state refers to the state that the display lens module is powered on but does not perform any substantive work. The working state is the state in which the display module in the display lens module is displaying. In one embodiment, the display lens module and the sounding frame are detachably connected, and an inductive switch is arranged between the display lens module and the sounding frame. When the display lens module and the sounding frame are connected, the display lens module enters the working state.

In one embodiment, the sounding frame and the display lens module are detachably connected; the step of acquiring the operating state of the display lens module includes:

If the display lens module is connected to the sounding frame, obtain the running state of the display lens module;

If the display lens module and the sounding stand are not connected, execute: the control of the first communication unit to establish a communication connection with an external terminal through a first preset communication protocol, and to receive an external terminal through the first preset communication protocol. The audio data sent by the terminal is sent to the signal processing module.

Step S20, determining that the operating state of the display lens module is a working state, then controlling the second communication unit to establish a communication connection with an external terminal through a second preset communication protocol, and receiving the transmission from the external terminal through the second preset communication protocol The image data is sent to the signal processing module;

Step S30, controlling the first communication unit to establish a communication connection with an external terminal through a first preset communication protocol, and receiving audio data sent by the external terminal through the first preset communication protocol and sending it to the signal processing module;

Steps S30 and S40 are in no particular order. If the running state of the display lens module is obtained and the running state is the working state, it means that the display lens module needs to perform data transmission with the external terminal. The first communication unit and the second communication unit perform data transmission with the external terminal. The external terminal may specifically be a mobile phone, a tablet computer, or the like. Of course, those skilled in the art can understand that the first communication unit and the second communication unit can not only receive data sent by the external terminal, but can also feed back data to the external terminal, and the communication between the communication module and the external terminal is bidirectional. , and will not be repeated here. The first preset communication protocol and the second preset communication protocol may be different low-power communication protocols, and specifically may be Bluetooth Low Energy (Bluetooth Low Energy), ZigBee, and the like. The first preset communication protocol and the second preset communication protocol may also be the same low power consumption communication protocol, and are specifically distinguished by different identification codes or different frequency bands.

Electronic components such as touch switches, cameras, and acceleration sensors may also be provided in the head-mounted display device, and the first communication unit and/or the second communication unit may also send information collected by each electronic component to an external terminal.

Step S40, controlling the signal processing module to parse the audio data sent by the first communication module and the image data sent by the second communication module.

The signal processing module can analyze the received audio data and image data at the same time, and optimize the audio data and image data through a preset algorithm, so that the audio data and image data are synchronized on the time axis, and the sound quality and image of the audio data are improved at the same time. The quality of the data.

In the present application, by setting the first communication module and the second communication module to respectively receive audio data and image data sent by the external terminal, the data interacted with the external terminal during the operation of the head-mounted display device can be transmitted through two transmission routes. transmission, thereby reducing the data throughput of a single transmission route, making it possible to implement wireless transmission on head-mounted display devices.

Referring to FIG. 2 , in the second embodiment of the control method for the head-mounted display device of the present application proposed based on the first embodiment of the present application, the signal processing module includes a first A signal processing unit and a second signal processing unit arranged in the housing of the display lens module; the S40 includes:

Step S41, controlling the communication module to send the image data to the second signal processing unit, controlling the second signal processing unit to parse the image data, and sending the parsed image data to the display lens module for display;

Step S42, controlling the communication module to send the audio data to the first signal processing unit, controlling the first signal processing unit to parse the audio data, and sending the parsed audio data to the sounding stand for playback.

The same steps S41 and S42 are executed in no particular order. The second signal processing unit encodes and decodes the image data, so that the display lens module can display the picture according to the parsed image data, so as to realize the integration with the real environment. The first signal processing unit encodes and decodes the audio data, so that the speaker can play sound according to the parsed audio data, and at the same time, the first signal processing unit can also encode the audio data collected by the microphone to be sent to the audio data through the first communication unit. external terminal. By arranging the first signal processing unit and the second signal processing unit, the workload and heat dissipation of a single processing unit can be reduced, and the signal processing speed can be improved.

In another embodiment, the second signal processing unit is arranged in the sounding frame to reduce the volume and weight of the display lens module. Compared with the first and second signal processing units being placed in the same housing, the first signal processing unit and the second signal processing unit are respectively placed in the outer shell of the sounding frame and the outer casing of the display lens module, so that the sounding frame and the display It is possible for the lens module to work independently, and the user can only use the sounding stand or display the lens module.

Referring to FIG. 3, in the third embodiment of the control method for the head-mounted display device of the present application proposed based on the second embodiment of the present application, after the step S10, the method further includes:

It is determined that the running state of the display lens module is a standby state; step S50 is executed, the communication module is controlled to establish a communication connection with the external terminal, and the communication module is controlled to receive audio data sent by the external terminal; and step S42 is executed.

Further, while executing: determining that the operating state of the display lens module is a standby state; controlling the communication module to establish a communication connection with the external terminal, and controlling the communication module to receive the audio data sent by the external terminal, while controlling the second signal processing unit to enter. Standby mode operation.

When the display lens module is in the standby state, it proves that the display lens module does not need to transmit image data. In order to reduce power consumption, only the first communication unit is used to establish a connection with the external terminal, and the second communication unit can enter the In standby or off, the first communication unit can transmit an instruction to wake up the display lens module. For example: when the first communication unit receives a wake-up command from the external terminal, the first communication unit sends a wake-up command to the display lens module and the second communication unit, so that the display lens module enters the working state, and the second communication unit Establish a connection with an external terminal.

Please refer to FIG. 4 and FIG. 5 , in terms of hardware structure, the head-mounted display device includes a signal processing module 60 , a control module 30 , a communication module 10 , a storage module 20 , a sounding frame 40 and a display lens module 50 . The sounding stand 40 is adapted to fit the user's head, the communication module 10 includes a first communication unit and a second communication unit, the storage module 20 stores a computer program that can run on the control module 30, When the computer program is executed by the control module 30, the steps of the control method of the head-mounted display device as described above are implemented.

The communication module 10 can be connected with an external communication device through a network. The communication module 10 can receive requests from external devices, and can also send requests, instructions and information to the external devices. The first communication unit and the second communication unit are respectively used for connecting with the external device through the first preset communication protocol and the second preset communication protocol.

The storage module 20 can be used to store software programs and various data. The storage module 20 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, etc.; Created data or information, etc. Additionally, the memory modules 20 may include high-speed random access memory modules, and may also include non-volatile memory modules, such as at least one magnetic disk memory module, flash memory device, or other volatile solid state memory module.

The control module 30 is the control center of the head-mounted display device 100, and uses various interfaces and lines to connect various parts of the entire head-mounted display device 100, by running or executing the software programs and/or modules stored in the storage module 20. , and call the data stored in the storage module 20 , perform various functions of the head-mounted display device 100 and process data, so as to perform overall monitoring of the head-mounted display device 100 . The control module 30 may include one or more processing units; optionally, the control module 30 may integrate an application control module and a modulation and demodulation control module, wherein the application control module mainly processes the operating system, user interface and application programs, etc. The adjustment control module mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation control module may also not be integrated into the control module 30 .

Further, the signal processing module 60 includes a first signal processing unit arranged in the housing of the sounding frame 40 and a second signal processing unit arranged in the housing of the display lens module 50. The first signal processing unit The units are respectively connected with the first communication unit, the speaker and the microphone of the sounding frame 40, and the second signal processing unit is respectively connected with the second communication unit and the display module of the display lens module 50. signal connection. By disposing the first signal processing unit and the second signal processing unit respectively, it is possible for the sounding frame 40 and the display lens module 50 to work independently. Users can only use the sounding frame 40 or the display lens module 50, which can reduce the cost of a single processing unit. Workload and cooling, increase signal processing speed.

In one embodiment, the sounding stand 40 includes a connecting member 41 detachably connected with the display lens module 50 and a temple 43 connected with the connecting member 41 . The electronic components in the sounding frame 40 and the electronic components in the display lens module 50 can be connected by a circuit such as a flexible circuit board or a data line, and the circuit can be accommodated in the connector 41 . The connecting piece 41 and the temple 43 can also be detachably connected, so that the temple 43 can be used alone as an earphone.

The connector 41 is provided with a first data transmission terminal connected to the communication module 10, the display lens module 50 is provided with a second data transmission terminal for contacting with the first data transmission terminal, The communication module 10 transmits the image information through the contacted first data transmission terminal and the second data transmission terminal. The first data transmission terminal and the second data transmission terminal may be matched male plugs and female plugs. When the connector 41 is connected to the display lens module 50, the first data transmission terminal and the second data transmission terminal are connected, thereby realizing data transmission.

Further, the sounding stand 40 further includes a battery connected to the signal processing module 60 , and the first data transmission terminal is also used for connecting an external power source to charge the battery.

Those skilled in the art can understand that the structure of the head-mounted display device 100 shown in FIG. 3 does not constitute a limitation on the head-mounted display device 100, and may include more or less components than those shown in the figure, or combine some components, or a different arrangement of components.

The present application also proposes a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium can be the storage module 20 in the head-mounted display device of FIG. 4, or can be, as ROM (Read-Only Memory, read-only memory module)/RAM (Random Access Memory, random access storage. module), a magnetic disk, and an optical disk, the computer-readable storage medium includes several instructions for causing a head-mounted display device with a control module to execute the methods described in the various embodiments of the present application.

In this application, the terms "first", "second", "third", "fourth" and "fifth" are only used for the purpose of description, and should not be construed as indicating or implying relative importance to ordinary skills in the art Personnel can understand the specific meanings of the above terms in this application according to specific situations.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Although the embodiments of the present application have been shown and described above, the scope of protection of the present application is not limited thereto. It should be understood that the above embodiments are exemplary and should not be construed as limitations on the present application. Those skilled in the art can make changes, modifications and substitutions to the above-mentioned embodiments within the scope of the present application, and these changes, modifications and substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A control method for a head-mounted display device, characterized in that the head-mounted display device comprises a signal processing module, a communication module signally connected to the signal processing module, a sounding frame and a display lens module, the communication module The module includes a first communication unit and a second communication unit; the method includes:

obtaining the running status of the display lens module;

determining that the operating state of the display lens module is a working state;

Controlling the second communication unit to establish a communication connection with an external terminal through a second preset communication protocol, receiving image data sent by the external terminal through the second preset communication protocol and sending it to the signal processing module; controlling the first communication The unit establishes a communication connection with the external terminal through the first preset communication protocol, receives the audio data sent by the external terminal through the first preset communication protocol, and sends it to the signal processing module;

The signal processing module is controlled to parse the audio data sent by the first communication module and the image data sent by the second communication module.
The control method of a head-mounted display device according to claim 1, wherein the signal processing module comprises a first signal processing unit and a second signal processing unit; the control of the signal processing module to parse the first signal processing module The steps of the audio data sent by a communication module and the image data sent by the second communication module include:

controlling the communication module to send image data to the second signal processing unit, controlling the second signal processing unit to parse the image data, and sending the parsed image data to the display lens module;

The communication module is controlled to send audio data to the first signal processing unit, the first signal processing unit is controlled to parse the audio data, and the parsed audio data is sent to the sounding stand.
The control method of the head-mounted display device according to claim 2, wherein after the step of acquiring the operating state of the display lens module, the control method further comprises:

determining that the operating state of the display lens module is a standby state;

controlling the communication module to establish a communication connection with an external terminal, and controlling the communication module to receive audio data sent by the external terminal;

The communication module is controlled to send audio data to the first signal processing unit, the first signal processing unit is controlled to parse the audio data, and the parsed audio data is sent to the sounding stand.
The control method of the head-mounted display device according to claim 3, wherein the sounding frame and the display lens module are detachably connected; the step of acquiring the operating state of the display lens module comprises the following steps: :

Detecting whether the display lens module is connected with the sounding frame;

If the display lens module is connected to the sounding stand, the running state of the display lens module is acquired.
3. The control method of the head-mounted display device according to claim 3, wherein, if the operating state of the display lens module is a standby state, the communication module is controlled to establish a communication connection with the external terminal, and the communication module is controlled to receive the external terminal. The step of sending the audio data also includes:

The second signal processing unit is controlled to enter a standby mode to operate.
A head-mounted display device, characterized in that the head-mounted display device includes a signal processing module, a control module, a communication module, a storage module, a sounding frame and a display lens module, and the sounding frame is used to communicate with a user The head is adapted, the communication module includes a first communication unit and a second communication unit, a computer program that can be run on the control module is stored on the storage module, and the computer program is executed by the control module When implementing the control method of the head-mounted display device according to any one of claims 1 to 5.
The head-mounted display device according to claim 6, wherein the signal processing module comprises a first signal processing unit arranged in the housing of the sounding frame and a first signal processing unit arranged in the housing of the display lens module Two signal processing units, the first signal processing unit is respectively connected to the first communication unit, the speaker and the microphone of the sounding stand, and the second signal processing unit is respectively connected to the second communication unit and the Describe the signal connection of the display lens module.
The head-mounted display device according to claim 6, wherein the sound-emitting stand comprises a connecting piece detachably connected with the display lens module and a temple connected with the connecting piece.
The head-mounted display device according to claim 8, wherein a first data transmission terminal connected to the communication module is disposed on the connector, and a first data transmission terminal is disposed on the display lens module for connecting with the communication module. The first data transmission terminal is in contact with the second data transmission terminal, and the communication module transmits image information through the contacted first data transmission terminal and the second data transmission terminal.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a control module, the head-mounted device according to any one of claims 1 to 5 is realized. The steps of the control method of the type display device.