WO2021052131A1

WO2021052131A1 - Head-mounted device, control method, apparatus and system therefor, and storage medium

Info

Publication number: WO2021052131A1
Application number: PCT/CN2020/111465
Authority: WO
Inventors: 黄凯
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-09-16
Filing date: 2020-08-26
Publication date: 2021-03-25
Also published as: CN112506335B; CN112506335A

Abstract

Disclosed are a control method, apparatus and system for a head-mounted device, a head-mounted device, and a computer-readable storage medium. The control method for a head-mounted device comprises: acquiring display information output from an opposite direction of a wearing side of a second head-mounted device, wherein the display information at least comprises operation information; the opposite direction is a direction away from a wearer when the second head-mounted device is worn; and the second head-mounted device is in communication connection with a first head-mounted device; and responding to the operation information, and executing an action corresponding to the operation information.

Description

Head-mounted equipment and its control method, device, system and storage medium

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on September 16, 2019, the application number is 201910874754.2, and the invention title is "head-mounted device and its control method, device, system and storage medium", all of which The content is incorporated in the application by reference.

Technical field

This application relates to the field of augmented reality, and in particular to a method, device, system, head-mounted device, and computer-readable storage medium for controlling a head-mounted device.

Background technique

Augmented reality (AR, Augmented Reality) technology is a technology that ingeniously integrates virtual information with the real world. Current AR products, for example, AR glasses. When the user uses AR glasses, the AR glasses can only see the enhanced display content on the wearing side, or receive the user's button operation on the head-mounted device to control the AR glasses, but the AR glasses cannot be used to control the operation of other devices, resulting in AR The use of equipment is limited.

Technical solutions

The main purpose of this application is to provide a head-mounted device control method, device, system, head-mounted device, and computer-readable storage medium, aiming to solve the problem that the head-mounted device cannot be used to control other devices in the prior art. Operation, leading to technical problems with limited use of head-mounted devices.

To achieve the above objective, the present application provides a method for controlling a head-mounted device, including:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer when the second head-mounted device is worn; Wherein, the second head-mounted device is in communication connection with the first head-mounted device;

Identify the operation information of the display information;

In response to the operation information, an action corresponding to the operation information is executed.

To achieve the foregoing objective, the present application provides a display control method of a head-mounted device, including:

Obtain the display content of the augmented reality to be displayed;

The display content of the augmented reality is output in the direction of the wearing side and the opposite direction of the wearing side, so that the first head-mounted device can obtain the display content output in the opposite direction to perform corresponding actions, and the direction of the wearing side is When the head-mounted device is worn, the direction close to the wearer, and the opposite direction is the direction away from the wearer; wherein the head-mounted device is in communication connection with the first head-mounted device.

In order to achieve the above objective, the present application provides a head-mounted device control device. The head-mounted device control device includes a memory, a processor, and a head that is stored in the memory and can run on the processor. A control program of a wearable device, when the control program of the head-mounted device is executed by the processor, the following steps are implemented:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer after the second head-mounted device is worn; Wherein, the second head-mounted device is in communication connection with the first head-mounted device;

Identify the operation information of the display information;

To achieve the foregoing objective, the present application provides a head-mounted device, the head-mounted device includes: a processor, a display device connected to the processor,

The processor acquires the display content of the augmented reality to be displayed;

The display device receives the display content sent by the processor, and outputs the display content in the direction of the wearing side and the opposite direction of the wearing side, so that the first head-mounted device can obtain the display output in the opposite direction The content executes the corresponding action, the wearing side direction is the direction close to the wearer when the head-mounted device is worn, and the opposite direction is the direction away from the wearer; wherein, the head-mounted device and the first A headset communication connection.

To achieve the above objective, the present application provides a head-mounted device control system, including: a first head-mounted device and a second head-mounted device, the second head-mounted device and the first head-mounted device Device communication connection,

The second head-mounted device outputs the display content of the augmented reality in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side;

The first head-mounted device acquires the display information output by the second head-mounted device in the opposite direction of the wearing side; recognizes the operation information of the display information; responds to the operation information, and executes the Action corresponding to the operation information.

In order to achieve the above objective, the present application provides a computer-readable storage medium having a control program of a head-mounted device stored on the computer-readable storage medium. When the control program of the head-mounted device is executed by a processor, the following is achieved step:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer after the second head-mounted device is worn;

Identify the operation information of the display information;

The second head-mounted device of the present application outputs display information in the direction of the wearing side and the direction opposite to the wearing side. The first head-mounted device performs corresponding actions according to the displayed information, and the second head-mounted device outputs in the opposite direction The display information of is acquired by the first head-mounted device, and the first head-mounted device executes the corresponding action, which realizes the control of the second head-mounted device to the first head-mounted device, instead of a single user operating the head The control of the head-mounted device by the head-mounted device is no longer the only thing that the user can watch the augmented reality content displayed by the head-mounted device by wearing the head-mounted device, which can realize the head-mounted device to control other head-mounted devices and expand The application of head-mounted devices.

Description of the drawings

FIG. 1 is a schematic diagram of functional modules of a control device of a head-mounted device of this application;

2 is a schematic flowchart of an exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 3 is a structural diagram of an exemplary embodiment of AR glasses as the head-mounted device of this application;

4 is a schematic flowchart of another exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 5 is a schematic flowchart of another exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 6 is a schematic flowchart of another exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 7 is a schematic flowchart of another exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 8 is a schematic flowchart of another exemplary embodiment of a method for controlling a head-mounted device according to the present application;

FIG. 9 is a schematic structural diagram of an exemplary embodiment of the control process of the head-mounted device of this application;

FIG. 10 is a schematic structural diagram of another exemplary embodiment of the control process of the head-mounted device of this application;

FIG. 11 is a structural diagram of an exemplary embodiment of a head-mounted device of this application;

FIG. 12 is a structural diagram of another exemplary embodiment of a head-mounted device of this application.

Embodiments of the present invention

It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

As shown in FIG. 1, FIG. 1 is a schematic structural diagram of a control device of a head-mounted device in a hardware operating environment involved in a solution of an embodiment of the present application.

As shown in FIG. 1, the control apparatus of the head-mounted device may include: a processor 1001, such as a CPU (central processing unit), a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input device such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Optionally, the control device of the head-mounted device may further include a camera, an RF (Radio Frequency, radio frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on. The head-mounted device scans the display information displayed by the head-mounted device through a camera, and completes communication operations with other devices through the network interface 1004.

Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the control device of the head-mounted device, and may include more or less components than shown in the figure, or a combination of certain components, or different components. Component arrangement. The control device of the head-mounted device may be a mobile terminal, such as a mobile phone or a pad; it may also be a head-mounted device, such as AR glasses.

As shown in FIG. 1, the memory 1005 may include an operating system, a network communication module, a user interface module, and a control application program of a head-mounted device.

In the terminal shown in FIG. 1, the network interface 1004 is mainly used to connect to the back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to the client (user side) and communicate with the client; and the processor 1001 can be used to call the control application of the head-mounted device stored in the memory 1005, and perform the following operations:

Identify the operation information of the display information;

Referring to FIG. 2, FIG. 2 is a schematic flowchart of an exemplary embodiment of a method for controlling a head-mounted device of this application. In this embodiment, the method for controlling the head-mounted device includes the following steps:

S10. Obtain display information output by the second head-mounted device in a direction opposite to the wearing side, where the display information includes at least operation information, and the reverse direction is that when the second head-mounted device is worn, it is far away from the wearer. Direction; wherein the second head-mounted device is in communication connection with the first head-mounted device;

In this embodiment, the first and second head-mounted devices are devices that can display augmented reality content. The first headset establishes a connection with the second headset, which can be a wireless connection or a wired connection; of course, the connection mentioned here can be that the first headset is authorized by the second headset , The display information can be obtained only after authorization. The first head-mounted device acquires the display information displayed on the side of the second head-mounted device facing the first head-mounted device, that is, the direction away from the wearer when the second head-mounted device is worn.

The head-mounted device takes AR glasses as an example. Referring to FIG. 3, the AR glasses include a display device 1 and a camera 2. The AR glasses have two display directions. One is the display direction on the wearing side as shown in the figure. The X direction, one is the opposite direction of the wearing side, and the display direction is the Y direction in Figure 3. After the user wears AR glasses, the direction closer to the wearer is the wearing side direction, and away from the wearer (the user's direction) The direction of is the opposite direction of the wearing side; in the judgment of the direction, the Y direction is the opposite direction of the wearing side, which is the direction toward the external device 3.

However, the AR glasses in this embodiment support the display of content in both the display direction of the wearing side and the opposite direction of the wearing side. The first head-mounted device acquires the display information in the opposite direction of the wearing side. The AR glasses can control whether to display in both directions according to user operations. This embodiment needs to control the display direction on the wearing side and the display direction opposite to the wearing side to display information at the same time, and the display on the wearing side The content enters the eyes of the wearing user, and the display information in the opposite direction of the wearing side is directly viewed by the outside user or obtained and viewed through the first head-mounted device.

The first head-mounted device obtains the display information output by the second head-mounted device in the opposite direction of the wearing side. The first head-mounted device may obtain the display information of the second head-mounted device by scanning, and the display information of the second head-mounted device may be an image, text or identification information, for example, a payment code or a two-dimensional Code, etc., the displayed information can be displayed in an enhanced display mode. The display information may carry application download information, device operation information, device setting information, or augmented reality content displayed on the wearing side of the second head-mounted device.

In an embodiment, in order to obtain the display information of the augmented reality content output by the second head-mounted device in the opposite direction of the wearing side through the first head-mounted device, the obtaining of the display device of the second head-mounted device is in the The step of outputting the display information in the opposite direction of the wearing side of the second head-mounted device includes:

Enable at least two camera devices; control the at least two camera devices to scan the display information output by the second head-mounted device in the opposite direction of the wearing side to obtain the enhanced display content in the display information for the local terminal The device user directly consults the enhanced display content. Two cameras. For example, two cameras can replace the two left and right lenses of AR glasses. The augmented reality content output by the second head-mounted device can be directly obtained through the two cameras, and then the device with two cameras can be used to obtain the augmented reality content output by the second head-mounted device. Output augmented reality content. The display information obtained by scanning or shooting by a camera device is only one-dimensional display information, and no augmented reality content is obtained.

S20: Identify the operation information of the display information;

After acquiring the display information output by the second head-mounted device, the first head-mounted device recognizes the operation information of the display information. The operation information may be a payment operation, or may be an application launch, a web page link or adding a WeChat friend, an information synchronization operation, and the like. The method of recognizing the operation information is an image extraction technology, which extracts the operation information or text information in the image. The operation information may also be forwarding the display information to other devices, for example, sending to other head-mounted devices, or sending to a third-party terminal, or displaying the display information on the first head-mounted device.

S30. Respond to the operation information and execute an action corresponding to the operation information.

The actions corresponding to the operation information include operations such as downloading applications, starting applications, adding WeChat friends, obtaining links, payment operations, or information sharing (forwarding to other devices). The first head-mounted device responds to the operation information and performs an action corresponding to the operation information. The first head-mounted device may be replaced by a mobile terminal, and is not limited to a head-mounted manner. The mobile terminal is a mobile phone, a pad, etc., which can display enhanced display content and communicate with the second head-mounted device.

The first head-mounted device responds to the operation information and executes the action corresponding to the operation information including multiple ways to complete, and the details are as follows.

Embodiment 1: The step of responding to the operation information and executing an action corresponding to the operation information includes: executing an application download, application control, or payment action according to the operation information. Through the payment code or payment operation information in the information displayed by the second head-mounted device, the first head-mounted device recognizes it and then completes the payment operation, or by obtaining the download link displayed by the second head-mounted device to execute the first head-mounted device The mobile device completes the action of downloading the application, or the first head-mounted device obtains the control information of an application that displays information on the second head-mounted device to complete the operation of the application, for example, WeChat add friend, Close the chat interface of a friend on WeChat, close the mall interface of a game application, enter the game operation interface, etc., or enable or close an application.

Embodiment 2: The step of responding to the operation information and executing an action corresponding to the operation information includes:

When the operation information is the display operation information for the head-mounted device, the first head-mounted device executes the action of sending the operation information to the third head-mounted device for the third head-mounted device Perform corresponding display control operations on the currently displayed augmented reality content. The display operation information includes the display control operation of the augmented reality content of the head-mounted device, for example, the control of the display brightness, the control of the display angle, etc.; the display information output of the second head-mounted device in the opposite direction of the wearing side , Through the first headset to achieve the interaction between different headsets. The third head-mounted device may optionally be a device that displays augmented reality content that can display augmented display content, and the third head-mounted device is a slave device of the second head-mounted device.

Embodiment 3: The step of responding to the operation information and executing an action corresponding to the operation information includes:

When the third-party terminal cannot establish a data connection with the second head-mounted device, the action of sending the operation information to the third-party terminal is performed to complete the second head-mounted device and the second head-mounted device through the first head-mounted device. Data transmission from three-party terminals. The scenario used is that when the third-party terminal cannot interact with the AR glasses, the AR glasses control the third-party terminal through the first head-mounted device, and the AR glasses control the third-party terminal with which it cannot communicate. Of course, when interaction between the second head-mounted device and the third terminal is possible, the control of the third-party terminal by the AR glasses can also be completed indirectly through the first head-mounted device.

Embodiment 4: The step of responding to the operation information and executing an action corresponding to the operation information includes:

When the operation information is synchronized display between head-mounted devices, the first head-mounted device executes the action of sending the display information to a third head-mounted device so that the third head-mounted device can display The wearing side displays the augmented reality content corresponding to the display information. The operation information here is to realize the sharing and synchronization of the augmented reality content displayed by different head-mounted devices, so that different head-mounted devices can display the same augmented reality content. The third head-mounted device may be the second head-mounted device. The slave device of the wearable device.

Embodiment 5: Complete the interactive operation between the first head-mounted device and the second head-mounted device, for example, complete the sharing of augmented reality content between the head-mounted devices, so that different AR glasses can display the same augmented reality content, AR glasses Scanning the display content of another AR glasses in the opposite direction from the wearing side through its installed camera, and displaying the augmented reality content of the other party's AR glasses on its own display device. The application scenario can be when different users want to watch the same as the other party. The content can be realized by outputting the display information in the opposite direction of the wearing side of the AR glasses.

The second head-mounted device of this embodiment outputs display information in the direction of the wearing side and the direction opposite to the wearing side. The first head-mounted device performs corresponding actions according to the displayed information, and the second head-mounted device is in the opposite direction. The output display information is acquired by the first head-mounted device, and the first head-mounted device executes corresponding actions, realizing the control of the second head-mounted device to the first head-mounted device, instead of a single user operation Head-mounted devices control the head-mounted devices, and the only thing is that the user wears the head-mounted device to watch the augmented reality content displayed by the head-mounted device, which can realize the head-mounted device to control other head-mounted devices. Expanded the application of head-mounted devices.

Referring to FIG. 4, FIG. 4 is a schematic flowchart of a method for controlling a head-mounted device in another exemplary embodiment of this application. The difference between this embodiment and other embodiments is that the user implemented here controls the first head-mounted device through the first head-mounted device. Two head-mounted devices, the method for controlling the head-mounted device in this embodiment further includes the steps:

S40: The first head-mounted device receives the voice information sent by the user;

S50: Convert the voice information into first control information of the second head-mounted device;

S60: Send the first control information to the second head-mounted device to control the second head-mounted device to perform a corresponding action.

The user sends voice information based on the first head-mounted device, such as a mobile terminal, and the voice information is the control of AR glasses that can interact with the first head-mounted device, and the first head-mounted device receives the voice information sent by the user , The voice information is converted into the first control information of the second head-mounted device, and the conversion operation is placed on the first head-mounted device, which can reduce the parts set on the AR glasses and save the cost; The wearable device sends the first control information to the second head-mounted device to control the second head-mounted device to perform corresponding actions, for example, complete the display settings of the second head-mounted device, or complete The switching and adjustment of display content is no longer just the user's direct voice control of the head-mounted device, but also the operation of the second head-mounted device can be controlled through the first head-mounted device, and the first control information can be The display is displayed on the first head-mounted device, and the second head-mounted device is obtained by scanning by a camera, and then performs an operation corresponding to the control information. The user completes the control of the terminal head-mounted device through the local device. The steps in FIG. 4 can be executed on the local device at the same time as the process in FIG. 1, or after the steps in FIG. 1, of course, can also be executed before the steps in FIG. 1, which is a complementary interference execution operation. That is, the operation of the first head-mounted device to obtain the display content output by the second head-mounted device in the opposite direction of the wearing side completes the operation of the first head-mounted device or completes the third-party terminal connected to the first head-mounted device /The operation of the third head-mounted device and the first head-mounted device receiving user voice information to control the second head-mounted device to complete the interaction between the first head-mounted device and the second head-mounted device are not mutually exclusive Conflicting actions.

Referring to FIG. 5, FIG. 5 is a schematic flowchart of a method for controlling a head-mounted device in another exemplary embodiment of this application. The difference between this embodiment and other embodiments is that the third-party terminal uses the first head-mounted device and the first head-mounted device. The two head-mounted devices interact to provide interactive operations between the head-mounted device and other devices in more scenarios; the control method of the head-mounted device further includes the steps:

S70: The first head-mounted device receives a control instruction from the third-party terminal to the second head-mounted device;

S80: Convert the control instruction into second control information of the second head-mounted device;

S90: Send the second control information to the second head-mounted device to control the second head-mounted device to perform a corresponding action.

When the third-party terminal cannot interact with the AR glasses, the first head-mounted device can indirectly complete the interaction with the AR glasses to complete the control of the AR glasses or control the AR glasses to display information shared by the third-party terminal. Of course, when the third-party terminal can interact with the AR glasses, it can also be done through the first head-mounted device.

The interaction between the first head-mounted device and the AR glasses is that the first head-mounted device displays the content that needs to be interacted on the display interface, and the AR glasses use the camera set in the AR glasses to capture the images displayed by the first head-mounted device. The content is then displayed, or the captured images are analyzed to obtain the operations that need to be performed, and the interaction between the third-party terminal and the AR glasses is completed. The execution of the steps in FIG. 5 on the first head-mounted device may be performed at the same time as the process in FIG. 1 or after the steps in FIG. 1, of course, it may also be executed before the steps in FIG. 1 are executed, which is a complementary interference execution operation. That is, the first head-mounted device obtains the display content output by the second head-mounted device in the opposite direction of the wearing side to complete the corresponding operation and receives the control instruction of the third-party terminal to control the second head-mounted device. Conflicting execution actions.

Referring to FIG. 6, FIG. 6 is a schematic flowchart of a display control method of a head-mounted device in an exemplary embodiment of this application. The difference between this embodiment and other embodiments is that the technical solution of this embodiment is based on the second head-mounted device. Device, the AR glasses are written on the profile, while the previous embodiments in Figures 2, 3, 4, and 5 are based on the first head-mounted device side and applied to the second head-mounted device side. The display control method of the mobile device further includes the steps:

S101: The second head-mounted device acquires the display content of the augmented reality to be displayed;

In this embodiment, the second head-mounted device takes AR glasses as an example. The current AR glasses can only display content on the user's wearing side. The user can see the real objects outside through the transmission of the glasses lens. Augmented reality technology enables the user to finally watch the augmented reality content, and the second head-mounted device obtains the augmented reality display content that needs to be displayed, or generates the augmented reality to be displayed based on the augmented reality technology for the content that needs to be displayed The display content can be obtained through external signal access or from the display content stored in advance.

S102. Output the display content of the augmented reality in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side, so that the first head-mounted device can obtain the display content output in the opposite direction and execute the corresponding The wearing side direction is the direction close to the wearer when the second head-mounted device is worn, and the opposite direction is the direction away from the wearer; wherein, the second head-mounted device and the The first headset communication connection.

The AR glasses can output display content in both the wearing side direction and the opposite direction of the wearing side, and control the output of the augmented reality display content in the wearing side direction of the second head-mounted device and the opposite direction of the wearing side, For the first head-mounted device to obtain the display content output in the opposite direction and perform corresponding actions, for example, the first head-mounted device is earphones, glasses, etc., and the AR glasses output the display content in the opposite direction of the human eye, The first head-mounted device acquires the display content displayed by the AR glasses through the camera scan, and completes the corresponding actions of the display content, for example, displays the display content, or sends the display content to a third-party terminal, etc. .

In an embodiment, the AR glasses need to control the output in the opposite direction to the wearing side as follows:

The second head-mounted device receives an output control operation based on the second head-mounted device;

When the output control operation is the first operation, execute the step of outputting the augmented reality display content in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side;

The output control operation may be voice control or physical button control, which varies according to the settings of the AR glasses.

When the output control operation is the second operation, the display content of the augmented reality is output in the direction of the wearing side of the second head-mounted device. The first operation is to turn on the output of the display content in the opposite direction to the wearing side; the second operation is to turn off the output of the display in the opposite direction to the wearing side, and the wearing side direction is when the second head-mounted device is worn, approaching The direction of the wearer, and the opposite direction is the direction away from the wearer.

This embodiment controls whether to output the display content in the opposite direction of the wearing side, and provides multiple output modes, so that the output modes are diversified.

Referring to FIG. 7, FIG. 7 is a schematic flowchart of a display control method of a head-mounted device in another exemplary embodiment of the present application. The difference between this embodiment and other embodiments lies in the control between the head-mounted devices. The display control method of the mobile device further includes the steps:

S103: Receive an operation instruction based on the second head-mounted device;

S104, perform an operation corresponding to the operation instruction;

S105: Output the operation instruction in a direction opposite to the wearing side of the display device, so that the first head-mounted device can obtain the operation instruction and send it to a third head-mounted device to perform an operation corresponding to the operation instruction.

The first head-mounted device needs to operate on the second head-mounted device and issue operation instructions. The method of issuing the operating instructions can be to display the operation instructions on the first head-mounted device; the second head-mounted device scans the first head The operation instruction displayed by the wearable device performs an operation corresponding to the operation instruction, for example, adjusting the brightness of the display content or adjusting the angle of the display, etc. When the second head-mounted device completes the operation corresponding to the operation instruction, at the same time or after that, or when receiving an operation instruction based on the second head-mounted device, the operation instruction is sent to the The display device is output in the opposite direction from the wearing side, so that the first head-mounted device can obtain the operation instruction and send it to the third head-mounted device to perform the operation corresponding to the operation instruction. The third head-mounted device scans To scan the operation instructions output by the first head-mounted device to complete the same control of the third head-mounted device.

For the safety of control, the third head-mounted device is a device associated with the second head-mounted device. For example, if the associated device is set for the second head-mounted device in advance, only when the associated device is Only the device can scan the operating instructions displayed by the second head-mounted device, add information about the associated head-mounted device to the operating instructions, and make verification and identification when the third head-mounted device scans. The scenario used can be that one head-mounted device is the main head-mounted device, and other associated head-mounted devices are its slave head-mounted devices. When viewing AR content, all adjustments are based on the main head-mounted device. Equipment, for example, when teaching, the teacher’s headset and the student’s headset need to be the same in order to better complete the teaching effect. Through the control of the master headset, and then share it to the slave Head-mounted devices to achieve consistency.

Referring to FIG. 8, FIG. 8 is a schematic flowchart of a display control method of a head-mounted device in another exemplary embodiment of this application. The difference between this embodiment and other embodiments is that the head-mounted device is used as a relay to implement different devices. The display control method of the head-mounted device further includes the following steps:

S106: The second head-mounted device receives the information sent by the terminal.

S107: Output the information sent by the terminal in a direction opposite to the wearing side of the second head-mounted device, so that the third-party terminal can obtain the information sent by the terminal.

When the third-party terminal and the terminal cannot share information or cannot perform data transmission, the terminal displays the content it needs to share on the display interface, the second head-mounted device scans the information sent by the terminal, and the second head-mounted device saves the information. The information sent by the terminal is output in the opposite direction of the wearing side of the display device, and the information scanned and output by the third-party terminal achieves indirect information sharing between the terminal and the third terminal, avoiding the inability to complete the operation when the direct sharing is not possible. , The terminal and the third-party terminal may have a master-slave relationship, or an association relationship may be established, and the association relationship between the terminal and the third-party terminal may be stored on the second head-mounted device side before the second head-mounted device After completing the information sharing and transmission, the third-party terminal can be authorized to obtain the information output by the head-mounted device.

Referring to Figure 9, the second head-mounted device is AR glasses, the first head-mounted device is replaced with mobile terminals such as mobile phones, and the third-party terminals are pads and mobile phones; AR glasses 11, mobile phones 12, and pad 13 are used as The example describes the process of the embodiment of this application. The AR glasses 11 can output display information in the opposite direction of the wearing side and the wearing side. The mobile phone can scan the display information output by the AR glasses 11 in the opposite direction of the wearing side. The user can wear the AR glasses 11 at the same time and watch the AR glasses 11 output on the wearing side. Augmented reality content. The AR glasses 11 simultaneously output display information in the direction of the wearing side and the opposite direction of the wearing side. The mobile phone 12 scans the display information displayed on the AR glasses 11, recognizes the operation information corresponding to the display information, and executes corresponding actions. For example, the operation information is sent to pad13, and pad13 completes the execution of the corresponding operation; or the mobile phone directly completes the downloading of the application, opening the application, obtaining links or adding WeChat friends, etc.; or the mobile phone sends the displayed information to pad13 to The display information is displayed on pad13. The mobile phone initiates a control operation, the AR glasses 11 scan the control operation information displayed on the mobile phone, and send the control operation information to the pad 13, the pad 13 performs the control operation, and the mobile phone completes the control of the pad 13 through the AR glasses 11. Based on the same operation, the pad 13 can also complete the control of the AR glasses 11 through the mobile phone 12. Based on the scenario, you can choose that the AR glasses cannot directly complete data transmission or control operations with the pad13. This is through the indirect connection of the mobile phone 12 to complete the operation of the AR glasses 11 on the pad13 or the operation of the pad13 on the AR glasses 11, or both Indirect information sharing between.

10, the second head-mounted device is AR glasses, the first head-mounted device is replaced with a mobile terminal such as a mobile phone, and the third head-mounted device is AR glasses. Hereinafter, the first AR glasses 14 are mobile phones. 15 and the second AR glasses 16 are taken as examples to describe the process of the embodiment of the present application. The first AR glasses 14 receives the user's voice control, converts the voice control into information displayed in the opposite direction of the wearing side of the AR glasses 14, the mobile phone 15 scans and obtains the information, and transmits it to the second AR glasses 16. One AR glasses 14 completes the voice control of the second AR glasses 16 by the user. Display content sharing, establish an association relationship between the first AR glasses 14 and the second AR glasses, for example, a master-slave relationship; the first AR glasses 14 display content, the mobile phone 15 scans the displayed content, and transmits the displayed content to the second AR glasses 16 , So that the first AR glasses 14 and the second AR glasses 16 display the same display content. Similarly, the second AR glasses 16 can perform the same operation to make the first AR glasses 14 and the second AR glasses 16 display the same display content.

The first head-mounted device is the second AR glasses, and the first head-mounted device is the first AR glasses as an example to describe the process of the embodiment of the present application. The first AR glasses output the display content in the opposite direction of the wearing side, and the second AR glasses scan through the camera to display the same display content; or receive the operation content displayed by the first AR glasses, and perform the same operation on the second AR glasses, The second AR glasses can control the first AR glasses in the same way.

This application also proposes a control device for a head-mounted device. In one embodiment, it includes a memory, a processor, and a control program for the head-mounted device that is stored on the memory and can run on the processor. When the control program of the type device is executed by the processor, the following steps are realized:

Identify the operation information of the display information;

In an embodiment, the control device of the head-mounted device includes: an acquisition module, an identification module, and an operation module;

The acquiring module is configured to acquire display information output in a reverse direction of the wearing side of a second head-mounted device, the display information includes at least operation information, and the reverse direction is after the second head-mounted device is worn , A direction away from the wearer; wherein the second head-mounted device is in communication connection with the first head-mounted device;

The identification module is configured to identify the operation information of the display information;

The operation module is configured to respond to the operation information and execute an action corresponding to the operation information.

Optionally, the acquisition module is configured to activate at least two camera devices;

Control the at least two camera devices to scan the display information output in the opposite direction of the wearing side of the second head-mounted device to obtain the enhanced display content in the display information for the first head-mounted device to wear The reader directly consults the enhanced display content.

Further, the operation module is further configured to execute the action of sending the display information to the third-party terminal when the third-party terminal cannot establish a data connection with the second head-mounted device, so as to pass the first head-mounted device. The wireless device completes the data transmission between the second head-mounted device and the third-party terminal.

Optionally, the operation module is further configured to perform an action of sending the display information to a third head-mounted device when the operation information is a synchronous display between head-mounted devices, so as to provide for the second head-mounted device. The three-head mounted device displays the augmented reality content on the wearing side and/or the opposite direction of the wearing side, wherein the third head-mounted device and the second head-mounted device have a master-slave relationship.

Optionally, the operation module is further configured to perform an action of sending the operation information to a third head-mounted device when the operation information includes display operation information on the head-mounted device, so that the The third head-mounted device performs a corresponding display control operation on the currently displayed augmented reality content.

Further, the device further includes: a receiving module and a conversion module,

The receiving module is configured to receive voice information sent by a user;

The conversion module is configured to convert the voice information into first control information of the second head-mounted device;

The sending module is further configured to send the first control information to the second head-mounted device to control the second head-mounted device to perform corresponding actions.

Optionally, the receiving module is further configured to receive a control instruction from the third-party terminal to the second head-mounted device when the third-party terminal cannot control the second head-mounted device;

The conversion module is further configured to convert the control instruction into second control information of the second head-mounted device;

The sending module is further configured to send the second control information to the second head-mounted device to control the second head-mounted device to perform corresponding actions.

The implementation of the functions of each module of the above-mentioned device is similar to the process in the above-mentioned method embodiment, and will not be repeated here.

The present application also proposes a head-mounted device. In one embodiment, referring to FIG. 9, the head-mounted device includes a processor 10, and a display device 20 connected to the processor 10,

The processor 10 obtains the display content of the augmented reality to be displayed;

The display device 20 receives the display content sent by the processor 10, and outputs the display content in the direction of the wearing side and the opposite direction of the wearing side, so that the first head-mounted device can obtain the output in the opposite direction Perform corresponding actions on the display content of the head-mounted device, the wearing side direction is the direction close to the wearer when the head-mounted device is worn, and the opposite direction is the direction away from the wearer; wherein the head-mounted device is Said first head-mounted device communication connection.

The head-mounted device takes AR glasses as an example. The display device 20 is a spectacle lens, which can be an optical waveguide AR lens. The optical waveguide AR lens can emit light independently or with an external light source, and the light is in the optical waveguide AR lens. The transmission is total reflection transmission, and the light is emitted in the direction of the wearing side and the opposite direction of the wearing side at the position of light coupling. For AR glasses, light is emitted in the direction of the human eye and light is emitted in the opposite direction of the human eye. The processor 10 obtains the display content of the augmented reality to be displayed; transmits it to the optical waveguide AR lens, and the optical waveguide AR lens receives the display content sent by the processor 10, and converts the display content to the wearing side direction and the wearing side Output in the opposite direction.

In order to better increase the display diversity of the head-mounted device and meet different needs; the optical waveguide AR lens of the head-mounted device is provided with a unidirectional see-through film in the opposite direction of the wearing side, and only allows light in a single direction. (The outside world is shining toward the human eye), and the outside world cannot see the light leakage of the optical waveguide AR lens in the outward direction (the opposite direction to the wearing side). The unidirectional film is set to control the light transmission in one direction, and to prevent the light from the direction of the human eye from being seen from the outside. The one-way perspective film can be detachably connected to the optical waveguide AR lens, for example, it can be pasted to the lens-mounted frame in a pasting manner; or the one-way perspective film and the frame have a detachable engagement structure , Through the opening and closing of the clamping structure, light transmission and opacity are realized.

In an embodiment, referring to FIG. 10, the head-mounted device further includes a camera device 30, which is connected to the processor 10, and the camera device 30 captures the display content of other devices. The wearable device also includes a transmission control device 40 and a storage device 50 connected to the processor 10, a color-changing see-through film is set on the optical waveguide spectacle lens of the head-mounted device, and the user can control the transmittance of the see-through film through the transmission control device 40 Control (through circuit control), or control see-through or non- see-through, so that the display device can display content in the opposite direction of the wearing side; the storage device 50 is used to store data generated by a head-mounted device or receive data from other Device data or data scanned by the imaging device 30. Through the reasonable control of the transmission control device 40, it is possible to switch between transmission and non-transmission. Under the premise of ensuring interaction with other devices, the content can also be kept secret according to requirements. The setting of the camera device 30 meets the interaction requirements with different devices, and the interactive operation is completed by the camera device 30 to obtain information displayed by other devices, such as mobile terminals or other head-mounted devices.

The present application also proposes a control system for a head-mounted device. In one embodiment, it includes a first head-mounted device and a second head-mounted device. The second head-mounted device and the first head-mounted device Type equipment communication connection;

The head-mounted device can be AR glasses, or head-mounted; the head-mounted device outputs display information in the opposite direction of the wearing side to realize the interaction between the first head-mounted device and the second head-mounted device , Instead of a single button or a single voice to control the head-mounted device, the head-mounted device can also share or control the first head-mounted, more diversified application methods.

In an embodiment, the system may also include a third-party terminal, which is added to the third-party terminal to realize the interaction from the second head-mounted device to the first head-mounted device and then to the third-party terminal, for example, the second head-mounted device Output the display content, after the first head-mounted scan, it is transmitted to the third terminal, or the third-party terminal sends the information to the interactive terminal, the interactive terminal displays, the second head-mounted device scans, and the third-party terminal and the second head-mounted device are scanned. Device interaction. For the specific interaction process, refer to the description in the foregoing embodiments.

The present application also proposes a computer-readable storage medium with a control program of the head-mounted device stored on the computer-readable storage medium, and the following steps are implemented when the control program of the head-mounted device is executed by a processor:

Identify the operation information of the display information;

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, method, article, or system. Without more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium such as ROM/RAM, magnetic A disc or an optical disc) includes a number of instructions to enable a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

The above are only optional embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of this application.

Claims

A control method of a head-mounted device is applied to a first head-mounted device, wherein the control method of the head-mounted device includes:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer when the second head-mounted device is worn; Wherein, the second head-mounted device is in communication connection with the first head-mounted device;

Recognizing the operation information of the display information; and in response to the operation information, and performing an action corresponding to the operation information.
8. The method for controlling a head-mounted device according to claim 1, wherein the step of obtaining the display information output in the opposite direction of the wearing side of the second head-mounted device comprises:

Enabling at least two camera devices; and controlling the at least two camera devices to scan the display information output in the opposite direction of the wearing side of the second head-mounted device, and the device provided with at least two camera devices acquires the display information The enhanced display content in the information outputs the enhanced display content for the wearer of the first head-mounted device to directly consult the enhanced display content.
The method for controlling a head-mounted device according to claim 1, wherein the step of responding to the operation information and executing an action corresponding to the operation information comprises: being unable to interact with the second head-mounted device on a third-party terminal When the wireless device establishes a data connection, it executes the action of sending the display information to the third-party terminal, so as to complete the data transmission between the second head-mounted device and the third-party terminal through the first head-mounted device.
The control method of the head-mounted device according to claim 1, wherein the display information includes augmented reality content, and the step of responding to the operation information and executing an action corresponding to the operation information includes:

When the operation information is a synchronized display between head-mounted devices, an action of sending the display information to a third head-mounted device is performed, so that the third head-mounted device can be worn and/or worn on the side of the third head-mounted device. The augmented reality content is displayed in the opposite direction of the side, wherein the third head-mounted device and the second head-mounted device have a master-slave relationship.
The method for controlling a head-mounted device according to claim 1, wherein the step of responding to the operation information and executing an action corresponding to the operation information comprises:

When the operation information includes the display operation information of the head-mounted device, an action of sending the operation information to a third head-mounted device is executed, so that the third head-mounted device can monitor the currently displayed augmented reality The content performs the corresponding display control operation.
The method for controlling a head-mounted device according to any one of claims 1 to 5, further comprising:

Receive voice messages sent by users;

Converting the voice information into first control information of the second head-mounted device; and sending the first control information to the second head-mounted device to control the second head-mounted device to execute The corresponding action.
The method for controlling a head-mounted device according to any one of claims 1 to 5, further comprising:

When the third-party terminal cannot control the second head-mounted device, receiving a control instruction from the third-party terminal to the second head-mounted device;

Transforming the control instruction into second control information of the second head-mounted device; and sending the second control information to the second head-mounted device to control the second head-mounted device to execute The corresponding action.
A display control method of a head-mounted device applied to a second head-mounted device, which includes:

Obtain the display content of the augmented reality to be displayed; and

The display content of the augmented reality is output in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side, so that the first head-mounted device can obtain the display content output in the opposite direction and execute corresponding actions , The wearing side direction is the direction close to the wearer when the second head-mounted device is worn, and the opposite direction is the direction away from the wearer; wherein, the second head-mounted device and the first Head-mounted device communication connection.
8. The display control method of a head-mounted device according to claim 8, wherein after the step of obtaining the display content of the augmented reality to be displayed, the method further comprises:

Receiving an output control operation based on the second head-mounted device;

When the output control operation is the first operation, perform the step of outputting the augmented reality display content in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side; and in the output control When the operation is the second operation, output the display content of the augmented reality in the direction of the wearing side of the second head-mounted device, wherein the first operation is to turn on the output of the display content in the opposite direction of the wearing side; the second operation In order to close the output of the display content in the opposite direction to the wearing side, the wearing side direction is the direction close to the wearer when the second head-mounted device is worn, and the opposite direction is the direction away from the wearer.
8. The display control method of a head-mounted device according to claim 8, further comprising:

Receiving an operating instruction based on the second head-mounted device; and performing an operation corresponding to the operating instruction, and outputting the operating instruction in the opposite direction of the wearing side of the display device for the first head-mounted device The device acquires the operation instruction and sends it to a third head-mounted device to perform an operation corresponding to the operation instruction.
8. The display control method of a head-mounted device according to claim 8, further comprising:

When the terminal cannot establish a data connection with the third-party terminal, receiving the information sent by the terminal; and outputting the information sent by the terminal in the opposite direction of the wearing side of the second head-mounted device, so that the third-party terminal can obtain the information The information sent by the terminal.
A control device for a head-mounted device, which includes a memory, a processor, and a control program of the head-mounted device that is stored in the memory and can run on the processor. The control of the head-mounted device is The following steps are implemented when the program is executed by the processor:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer after the second head-mounted device is worn; Wherein, the second head-mounted device is in communication connection with the first head-mounted device;

Recognizing the operation information of the display information; and in response to the operation information, and performing an action corresponding to the operation information.
A head-mounted device, including: a processor, a display device connected to the processor,

The processor acquires the display content of the augmented reality to be displayed;

The display device receives the display content sent by the processor, and outputs the display content in the direction of the wearing side and the opposite direction of the wearing side, so that the first head-mounted device can obtain the display output in the opposite direction The content executes the corresponding action, the wearing side direction is the direction close to the wearer when the head-mounted device is worn, and the opposite direction is the direction away from the wearer; wherein, the head-mounted device and the first A headset communication connection.
A control system for a head-mounted device, comprising: a first head-mounted device and a second head-mounted device, the second head-mounted device is in communication connection with the first head-mounted device;

The second head-mounted device outputs the display content of the augmented reality in the direction of the wearing side of the second head-mounted device and the opposite direction of the wearing side;

The first head-mounted device acquires the display information output by the second head-mounted device in the opposite direction of the wearing side; recognizes the operation information of the display information; responds to the operation information, and executes the Action corresponding to the operation information.
A computer-readable storage medium, wherein a control program of a head-mounted device is stored on the computer-readable storage medium, and the following steps are implemented when the control program of the head-mounted device is executed by a processor:

Acquiring display information output in a reverse direction of the wearing side of the second head-mounted device, where the display information includes at least operation information, and the reverse direction is a direction away from the wearer after the second head-mounted device is worn;

Recognizing the operation information of the display information; and in response to the operation information, and performing an action corresponding to the operation information.