WO2021057420A1

WO2021057420A1 - Method for displaying control interface and head-mounted display

Info

Publication number: WO2021057420A1
Application number: PCT/CN2020/113092
Authority: WO
Inventors: 徐越; 邓焯泳
Original assignee: 华为技术有限公司
Priority date: 2019-09-25
Filing date: 2020-09-02
Publication date: 2021-04-01
Also published as: CN112558847A; CN112558847B

Abstract

Embodiments of the present application relate to the technical field of electronics, and provide a method for displaying a control interface and a head-mounted display capable of fusing a binocular image of a user, improving user experience. The specific solution is as follows: the head-mounted display displays a first interface, the first interface comprising a first sub-interface and a second sub-interface, the first sub-interface comprising a first picture, and the second sub-interface comprising a second picture, wherein the first picture and the second picture are the same in size; the head-mounted display detects a first preset operation of a user; in response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture; the head-mounted display detects a second preset operation of the user; in response to the second preset operation, the head-mounted display determines an adjustment parameter according to the size of the adjusted first picture and/or second picture; and the head-mounted display displays a third sub-interface and a fourth sub-interface in a second interface according to the adjustment parameter. The embodiments of the present application are used for displaying the control interface.

Description

Method for controlling interface display and head-mounted display

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on September 25, 2019, the application number is 201910914137.0, and the application name is "Method of Control Interface Display and Head Mounted Display", the entire content of which is incorporated by reference In this application.

Technical field

The embodiments of the present application relate to the field of electronic technology, and in particular, to a method for controlling interface display and a head-mounted display.

Background technique

A head-mounted display (HMD) is a structure that is worn on the user's head and integrates the image source and the visual optical system. The head-mounted display can follow the movement and rotation of the user's head and synchronize to the user. The eyes transmit and display image information. The current head-mounted display is the core solution in the fields of virtual reality (virtual reality, VR), augmented reality (AR), and mixed reality (MR). With high-resolution micro-displays and optical lenses The rapid development of technology has been widely studied and commercialized in recent years, and new architectures and equipment have frequently emerged.

As a visual system, the head-mounted display needs to be compatible with customers with different vision levels. This requires the optical system to have a certain adjustment ability, which can adjust the optical power of the user's eyes separately to make the user's eyes clear. However, there may be differences in the optical power of the user's eyes. For example, the user's eyes have different degrees of myopia. Therefore, after adjusting the left and right optical systems of the head-mounted display according to the optical powers of the user's eyes, the focal lengths of the left and right optical systems are different. of view, FOV) is different. Therefore, it is easy to cause the user's binocular images to not merge, resulting in dizziness and poor immersion when the user uses the head-mounted display; and the user's eyes are prone to fatigue and discomfort, which reduces the user experience.

Summary of the invention

The embodiment of the present application provides a method for controlling interface display and a head-mounted display. The head-mounted display can adjust the size of the picture on the first display screen and/or the second display screen according to the user's instruction, and respond to the user Determine the operation of the same size of the pictures seen through the left and right optical systems of the head-mounted display, determine the adjustment parameters according to the adjusted pictures, and then adjust the subsequent display of the first display and the second display according to the adjustment parameters The size of the interface is such that the size of the interface displayed on the first display screen and the second display screen that the user sees later is consistent, so that the binocular images of the user are merged, and the user experience is improved.

To achieve the foregoing objectives, the following technical solutions are adopted in the embodiments of the present application:

On the one hand, the technical solution of the present application provides a method for controlling interface display, which is applied to a head-mounted display with a first display screen and a second display screen. The method includes: a head-mounted display displaying a first interface, the first interface including a first sub-interface displayed on a first display screen and a second sub-interface displayed on a second display screen, the first sub-interface including a first picture, a second sub-interface The sub-interface includes a second picture; wherein, the first picture and the second picture have the same size. The head-mounted display detects the user's first preset operation; in response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture. The head-mounted display detects the second preset operation of the user; in response to the second preset operation, the head-mounted display determines the adjustment parameter according to the adjusted size of the first picture and/or the second picture. The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters; wherein, the third sub-interface is displayed on the first display screen, and the fourth sub-interface is displayed on the second display screen.

In this solution, the head-mounted display can adjust the size of the first picture or the second picture respectively displayed on the first display screen and/or the second display screen according to the user's instructions, so that the user can see the first picture through the optical system. The size of the picture and the second picture are the same. Furthermore, the head-mounted display can determine the adjustment parameters according to the adjusted first picture and/or the second picture, and adjust the size of the interfaces respectively displayed on the subsequent first display screen and the second display screen according to the adjustment parameters, so that the user can follow up The size of the interface displayed on the first display and the second display is the same, so that the user's binocular images are merged, and the user will not feel dizzy when using the head-mounted display, and the immersion is good, and the user's eyes are not easy Fatigue and discomfort can improve user experience. In addition, the first interface for adjusting the size of the picture is non-customized, easy to use, and does not need to be customized for different users.

In a possible implementation manner, the head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters, including: the head-mounted display determines the third sub-interface and the fourth sub-interface according to the adjustment parameters The size of the interface; the head-mounted display displays the third sub-interface and the fourth sub-interface.

That is to say, the head-mounted display determines the size of the third sub-interface and the fourth sub-interface according to the adjustment parameters, so that the size of the third sub-interface and the fourth sub-interface seen by the user are the same, and then displays the determined first sub-interface The three sub-interfaces and the fourth sub-interface allow the user's eyes to see the third sub-interface and the fourth sub-interface through the head-mounted display to merge.

In another possible implementation manner, before the head-mounted display displays the first interface, the method further includes: the head-mounted display detects a third preset operation of the user; in response to the third preset operation, the head-mounted display The head-mounted display adjusts the optical power of the head-mounted display for the eyesight of the user.

In other words, before the head-mounted display displays the first interface, the user can use the head-mounted display to adjust the optical power of the head-mounted display for the user’s binocular vision, so that the left and right eyes of the head-mounted display can be optically adjusted. The system is adapted to the user's left and right eyes respectively, so that the user's eyes can see clearly through the left and right optical systems. In addition, because the head-mounted display adjusts the head-mounted display's optical power for the user’s binocular vision, there may be over-correction/under-correction problems. The over-correction/under-correction problems will cause the fusion of the first picture and the second picture to change. Therefore, the head-mounted display performs image fusion adjustment after focusing, which can simultaneously adjust the fusion changes caused by over-correction/under-correction problems.

In another possible implementation, the head-mounted display further includes an optical focusing module, and the optical focusing module includes an optical lens; in response to the third preset operation, the head-mounted display adjusts the head-mounted display to the user The optical power of vision includes: in response to the third preset operation, the head-mounted display adjusts the optical lens.

In other words, the head-mounted display can adjust the optical lens in the optical focusing module to fit the left and right optical systems of the user's eyes. The user does not need to know the visual acuity of the eyes accurately, and can make the user's eyes pass the The left and right eye optical systems see clearly.

In another possible implementation manner, before the head-mounted display displays the first interface, the method further includes: the head-mounted display detects a fourth preset operation of the user, and the fourth preset operation is used to indicate the head-mounted display The display shows the first interface.

In other words, the head-mounted display displays the first interface in response to the user's fourth preset operation, and provides the user with an interactive interface that can be used to adjust the size of the picture.

In another possible implementation manner, the first interface includes prompt information, and the prompt information is used to prompt the user to adjust the size of the first picture and/or the second picture.

In other words, the head-mounted display can instruct the user how to operate on the first interface to adjust the size of the first picture and/or the second picture by including prompt information on the first interface, so that the user can finally see the first picture. The size of the picture and the second picture are the same.

In another possible implementation manner, after the head-mounted display displays the first interface, the head-mounted display may prompt the user to adjust the size of the first picture and/or the second picture on the first interface through voice information to Make the size of the first picture and the second picture that the user sees the same.

That is to say, after displaying the first interface, the head-mounted display can prompt the user to adjust the size of the first picture and/or the second picture on the first interface through voice information, so that the first picture and/or the second picture that the user finally sees The size of the second picture is the same.

In another possible implementation manner, in response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture, including: in response to the first preset operation, the head-mounted display zooms out The size of the larger picture in the first picture and the second picture; or, the head-mounted display enlarges the size of the smaller picture in the first picture and the second picture; or, the head-mounted display reduces the size of the first picture and the second picture Enlarge the size of the larger picture, and enlarge the size of the smaller picture in the first picture and the second picture.

That is to say, the head-mounted display can realize that the size of the first picture and the second picture seen by the user are the same by reducing the smaller picture, or enlarging the larger picture, or reducing the smaller picture and enlarging the larger picture.

In another possible implementation manner, the first preset operation includes: the user presses the first button on the handle connected to the head-mounted display; or, the user clicks the first control on the first interface through the mouse; or , The user controls the cursor of the head-mounted display connected to the head-mounted display to stay at the first control on the first interface for a preset time; or, the user presses the second button on the head-mounted display.

In other words, the user can implement the first preset operation of instructing to adjust the size of the first picture and/or the second picture through a handle, a mouse, a head sight, or a button.

In another possible implementation manner, the adjustment parameters include a first ratio between the size of the adjusted first picture and the first picture before adjustment, and/or the ratio between the adjusted second picture and the second picture before adjustment The second ratio of size.

That is, the adjustment parameter determined by the head-mounted display may be an adjustment ratio of the first picture and/or the second picture, so that the head-mounted display displays an image according to the adjustment ratio.

In another possible implementation manner, the colors of the first picture and the second picture are different.

In other words, the head-mounted display can set the first picture and the second picture to different colors, so that the user can distinguish the first picture from the second picture.

The color of the first picture may be red, and the color of the second picture may be green.

In another possible implementation, the head-mounted display also includes a software development kit (SDK) for supporting the first interface, a graphics processing unit (GPU), and a display; The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters, which specifically include: the head-mounted display sends the adjustment parameters to the SDK; the head-mounted display sends the adjustment parameters from the SDK to the GPU; The wearable display renders the third sub-interface and the fourth sub-interface in the second interface in the GPU according to the adjustment parameters; the head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface.

In other words, the head-mounted display sends the adjustment parameters to the SDK, and then to the GPU via the SDK, so that the GPU can render the third and fourth sub-interfaces in the second interface according to the adjusted parameters, and finally, the head-mounted display The third sub-interface and the fourth sub-interface in the rendered second interface are displayed, so that the third sub-interface and the fourth sub-interface seen by the user can be merged, and the user experience is improved. At the same time, the solution does not add a hardware negative feedback module to the head-mounted display, and there is no additional cost.

On the other hand, the technical solution provides a device for controlling interface display, which is included in an electronic device, and the device has the function of realizing the above aspects and the behavior of the electronic device in the possible implementation manners of the above aspects. The function can be realized by hardware, or the corresponding software can be executed by hardware. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a detection module or unit, a display module or unit, and so on.

On the other hand, the technical solution provides a head-mounted display, including: a first display screen and a second display screen; one or more processors; a memory; a plurality of application programs; and one or more computer programs, The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instruction is executed by the electronic device, the electronic device is caused to execute the method for controlling interface display in any possible implementation of any one of the foregoing aspects.

On the other hand, the present technical solution provides a computer storage medium, including computer instructions, when the computer instructions run on an electronic device, the electronic device executes the method for controlling interface display in any one of the possible implementations of the foregoing aspects. .

On the other hand, the present technical solution provides a computer program product, which when the computer program product runs on an electronic device, causes the electronic device to execute the method of controlling interface display in any one of the above-mentioned possible designs.

Description of the drawings

FIG. 1 is a schematic diagram of the hardware structure of a head-mounted display provided by an embodiment of the application;

2 is an image height adjustment system architecture of a head-mounted display provided by an embodiment of the application;

FIG. 3 is a flowchart of a method for controlling interface display according to an embodiment of the application;

4A is a schematic diagram of a picture format in the interactive interface of the head-mounted display provided by an embodiment of the application;

4B is a schematic diagram of another picture format in the interactive interface of the head-mounted display provided by an embodiment of the application;

4C is a schematic diagram of another picture format in the interactive interface of the head-mounted display provided by an embodiment of the application;

FIG. 4D is a visual image of a head-mounted display provided by an embodiment of the application when a user sees an interactive interface;

4E is a visual image of another interactive interface seen by a user of a head-mounted display provided by an embodiment of the application;

4F is a visual image of another interactive interface seen by a user of a head-mounted display provided by an embodiment of the application;

FIG. 5 is a schematic diagram of an interactive interface of a head-mounted display provided by an embodiment of the application;

FIG. 6 is a schematic diagram of another interactive interface of a head-mounted display provided by an embodiment of the application;

FIG. 7 is a schematic diagram of another interactive interface of a head-mounted display provided by an embodiment of the application;

FIG. 8 is a schematic diagram of another interactive interface of a head-mounted display provided by an embodiment of the application;

FIG. 9 is a schematic diagram of another interactive interface of a head-mounted display provided by an embodiment of the application;

10 is a schematic diagram of another interactive interface of a head-mounted display provided by an embodiment of the application;

FIG. 11A is a schematic diagram of an interactive interface after adjusting the picture of the head-mounted display according to an embodiment of the application; FIG.

FIG. 11B is a visual image of another interactive interface seen by a user of a head-mounted display provided by an embodiment of the application;

FIG. 12 is a schematic diagram of a display interface of a head-mounted display provided by an embodiment of the application;

FIG. 13 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

detailed description

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this document is only a description of related objects The association relationship of indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: A alone exists, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

Hereinafter, the terms “first” and “second” are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the present embodiment, unless otherwise specified, "plurality" means two or more.

The optical power of the user's eyes refers to the refractive power of the user's eyes to incident parallel light beams. The unit of optical power is dioptre (dioptre, D). The greater the value of the optical power, the more the parallel beams will bend. When the refractive power is greater than zero, the refraction is convergent, corresponding to the eyes of a nearsighted user; when the refractive power is less than zero, the refraction is divergent, corresponding to the eyes of a hyperopic user. When the optical power is equal to zero, it corresponds to plane refraction. At this time, the beam parallel to the axis is still the beam parallel to the axis after being refracted, and no refraction occurs. Generally, glasses are customized for the user according to the user's binocular refractive power, and the power of the user's glasses=diopter×100. For example, if the diopter of the eyes of the myopic user A is -3D, the myopic power of the myopic user A should be 300 degrees; the diopter of the hyperopic user B is 2D, then the hyperopic power of the hyperopic user B should be 200 degrees.

The relationship between the optical power, FOV, FOV converted image height, and paraxial focal length of the optical system of the electronic device adapted to the user's eyesight may be shown in Table 1 below. It can be seen that in the optical system shown in Table 1, if the refractive power is different, the FOV, the FOV converted image height, and the paraxial focal length are all different. And, the greater the optical power, the smaller the paraxial focal length, the larger the FOV, the greater the FOV converted image height; conversely, the smaller the optical power, the greater the paraxial focal length, the smaller the FOV, and the smaller the FOV converted image height. Therefore, when the refractive powers of the user's eyes are inconsistent, the optical powers of the optical system of the electronic device adapted to the user's eyes are also different, so that the image height of the image seen by the user's eyes is different, that is, the size of the image seen is inconsistent. As a result, the user's binocular images are not fused, which easily leads to dizziness and poor immersion when the user uses the electronic device.

Table 1

光焦度Optical power	FOVFOV	FOV换算像高FOV conversion image height	近轴焦距Paraxial focal length

0D0D	100100	11	24.2263970224.22639702
1D1D	100.3100.3	1.0052497441.005249744	24.1318404724.13184047

2D2D	102102	1.0355303141.035530314	23.9326548123.93265481
3D3D	102.8102.8	1.0501034491.050103449	23.6448326123.64483261
4D4D	104.2104.2	1.0761281641.076128164	23.4076292023.40762920
5D5D	105.2105.2	1.0951397391.095139739	23.1737219923.17372199
6D6D	106.3106.3	1.1164767641.116476764	22.9883621122.98836211
7D7D	107.5107.5	1.1402814581.140281458	22.6718378122.67183781

The embodiment of the present application provides a method for controlling interface display, and the method for controlling interface display can be applied to various electronic devices with display functions. For example, the electronic device may be a head-mounted display, or other devices including a display screen, such as an autonomous vehicle including a display screen. Among them, the user can wear a head-mounted display to achieve different effects such as virtual reality (VR), AR, and mixed reality (MR). For example, the head-mounted display may be glasses, goggles, helmets, or the like. The following description mainly takes the electronic device as a head-mounted display as an example.

In the method of controlling interface display, the head-mounted display displays a first interface, the first interface includes a first sub-interface displayed on the first display screen and a second sub-interface displayed on the second display screen, and the first sub-interface includes a first sub-interface displayed on the second display screen. For a picture, the second sub-interface includes a second picture; wherein the size of the first picture and the second picture are the same. The head-mounted display detects the user's first preset operation; in response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture. The head-mounted display detects the second preset operation of the user; in response to the second preset operation, the head-mounted display determines the adjustment parameter according to the adjusted size of the first picture and/or the second picture. The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters; wherein, the third sub-interface is displayed on the first display screen, and the fourth sub-interface is displayed on the second display screen.

In this solution, the head-mounted display may adjust the size of the first picture and/or the second picture in response to the user's first preset operation. When the user determines that the size of the first picture and the second picture seen are the same and triggers the second preset operation, the head-mounted display can respond to the user's second preset operation according to the adjusted first picture and / Or the size of the second picture to determine the adjustment parameters. Then, the head-mounted display can adjust and display the third sub-interface and the fourth sub-interface according to the adjustment parameters, so that the size of the third sub-interface and the fourth sub-interface seen by the user through the head-mounted display are the same, so that the user can double Target image fusion.

Therefore, when the user’s eyes are adapted to the left and right optical systems with different focal lengths due to inconsistent optical powers, the size of the pictures seen by the user’s eyes through the left and right optical systems are inconsistent, and the user can use the head-mounted display to The size of the first picture and the second picture seen by the user’s eyes are adjusted to be the same. The head-mounted display can adjust the third and fourth sub-interfaces to be displayed later according to the adjustment parameters, so that the user’s eyes can see through the optical system The third sub-interface and the fourth sub-interface have the same size. As a result, the binocular images of users with inconsistent optical powers of the eyes can be merged, and the user will not experience dizziness when using the head-mounted display, and the sense of immersion is better. At the same time, the user's eyes are not prone to fatigue and discomfort, which improves the user experience.

Among them, the user determines that the size of the first picture and the second picture are the same means that the size of the first picture and the second picture that the user sees through the head-mounted display are exactly the same or basically the same, that is, the user’s eyes pass through the head. The wearable display sees that the first picture and the second picture can be merged.

In the embodiments of the present application, the head-mounted display adjusting the user's binocular image fusion may include: adjusting the user's binocular vision due to the user's binocular myopia, monocular myopia, binocular hyperopia, monocular hyperopia, or hyperopia and myopia. The binocular images are not fused due to power problems. It is understandable that, in the embodiments of the present application, the situation where the head-mounted display adjusts the fusion of the binocular images of the user is not limited to this, and may also include other situations where the binocular images of the user's eyes are not fused due to the optical power problem. The embodiment does not limit this.

Please refer to FIG. 1. FIG. 1 is a schematic structural diagram of a head-mounted display provided by an embodiment of the present application. When the head-mounted display is installed on the user's head, the user's eyes can see the image presented by the head-mounted display display. The user's eyes can see the physical object through the display screen, or the user's eyes can see the image displayed by another display device through the display screen.

It is understandable that the embodiment of the present application takes the electronic device as a head-mounted display as an example for introduction, but the embodiment of the present application is not limited to the head-mounted display, and the electronic device may also be other devices.

As shown in FIG. 1, the head-mounted display 100 may include a processor 110, a memory 120, a sensor module 130, a microphone 140, buttons 150, an input and output interface 160, a communication module 170, a camera 180, a battery 190, a display screen 1100, and an optical Focusing module 1300 and so on. The sensor module 130 may include a focal length detecting optical sensor 131, and the focal length detecting optical sensor 131 is used to detect the focal length of the eyeball of the user 200. The sensor module 130 may also include other sensors, such as a sound detector, a proximity light sensor, a distance sensor, a gyroscope sensor, an ambient light sensor, an acceleration sensor, and a temperature sensor.

It can be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the head-mounted display 100. In other embodiments of the present application, the head-mounted display 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange different components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal processor. (image signal processor, ISP), video processing unit (VPU) controller, memory, video codec, digital signal processor (DSP), baseband processor, and/or neural network processing Neural-network processing unit (NPU), etc. Among them, the different processing units may be independent devices or integrated in one or more processors.

The controller may be the nerve center and command center of the head-mounted display 100. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or used cyclically. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

In some embodiments, the processor 110 may include one or more interfaces. The interface can include an integrated circuit (inter-integrated circuit, I2C) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), and a general input and output (general input and output) interface. -purpose input/output, GPIO) interface, subscriber identity module (SIM) interface, and/or universal serial bus (USB) interface, serial peripheral interface (SPI) Interface, etc.

The I2C interface is a bidirectional synchronous serial bus, which includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple sets of I2C buses. The processor 110 may be respectively coupled to the focal length detection optical sensor 131, the battery 190, the camera 180, etc. through different I2C bus interfaces. For example, the processor 110 may couple the focal length detecting optical sensor 131 through an I2C interface, so that the processor 110 and the focal length detecting optical sensor 131 communicate through an I2C bus interface to obtain the user's eyeball focal length. The SPI interface can be used for the connection between the processor and the sensor.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a two-way communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the communication module 170. For example, the processor 110 communicates with the Bluetooth module in the communication module 170 through the UART interface to implement the Bluetooth function.

The MIPI interface can be used to connect the processor 110 with the display screen 1100, the camera 180 and other peripheral devices. The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and so on. In some embodiments, the processor 110 and the camera 180 communicate through a CSI interface to implement the shooting function of the head-mounted display 100. The processor 110 and the display screen 1100 communicate through a DSI interface to realize the display function of the head-mounted display 100.

The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 180, the display screen 1100, the communication module 170, the sensor module 130, the microphone 140, and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface is an interface that complies with the USB standard specifications, and can be a Mini USB interface, a Micro USB interface, and a USB Type C interface. The USB interface can be used to connect a charger to charge the head-mounted display 100, and can also be used to transfer data between the head-mounted display 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. The interface can also be used to connect other electronic devices, such as mobile phones. The USB interface can be USB3.0, which is compatible with high-speed display port (DP) signal transmission, and can transmit high-speed video and audio data.

It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely a schematic description, and does not constitute a structural limitation of the head-mounted display 100. In other embodiments of the present application, the head-mounted display 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.

In addition, the head mounted display 100 may include a wireless communication function. The communication module 170 may include a wireless communication module and a mobile communication module. The wireless communication function can be realized by an antenna (not shown), a mobile communication module (not shown), a modem processor (not shown), a baseband processor (not shown), and the like.

The antenna is used to transmit and receive electromagnetic wave signals. The head-mounted display 100 may include multiple antennas, and each antenna may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example: the antenna can be multiplexed as a diversity antenna for a wireless local area network. In other embodiments, the antenna can be used in combination with a tuning switch.

The mobile communication module can provide a wireless communication solution including 2G/3G/4G/5G and the like applied to the head-mounted display 100. The mobile communication module may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves by the antenna, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module can also amplify the signal modulated by the modem processor, and convert it into electromagnetic waves for radiation by the antenna. In some embodiments, at least part of the functional modules of the mobile communication module may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module and at least part of the modules of the processor 110 may be provided in the same device.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker, etc.), or displays an image or video through the display screen 1100. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module or other functional modules.

The wireless communication module can provide applications on the head-mounted display 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), and global navigation. Satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module may be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, modulates the frequency of the electromagnetic wave signals and filters them, and sends the processed signals to the processor 110. The wireless communication module can also receive the signal to be sent from the processor 110, perform frequency modulation, amplify, and convert it into electromagnetic waves to radiate through the antenna.

In some embodiments, the antenna of the head-mounted display 100 is coupled with the mobile communication module, so that the head-mounted display 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite-based augmentation systems (SBAS).

The head-mounted display 100 implements a display function through a GPU, a display screen 1100, and an application processor. The GPU is an image processing microprocessor, which is connected to the display screen 1100 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information. In the embodiment of the present application, the GPU may render the to-be-displayed interface of the display screen 1100 according to the adjustment parameters.

The number of display screens 1100 in the head-mounted display 100 may be two, corresponding to the two eyeballs of the user 200 respectively. The content displayed on the two displays can be displayed independently. Different images can be displayed on the two display screens to improve the three-dimensional sense of the image. In some possible embodiments, the number of display screens 1100 in the head-mounted display 100 may also be one to correspond to the two eyeballs of the user 200. In the embodiment of the present application, the display screen 1100 includes a first display screen 1101 and a second display screen 1102, which can provide display content for the two eyeballs of the user 200, respectively. For example, the first display screen 1101 provides display content for the user's left eye, and the second display screen 1102 provides display content for the user's right eye; or, the first display screen 1101 provides display content for the user's right eye, and the second display screen 1102 is the user The left eye provides display content.

The head-mounted display 100 can realize a shooting function through an ISP, a camera 180, a video codec, a GPU, a display screen 1100, and an application processor.

The ISP is used to process the data fed back by the camera 180. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element of the camera through the lens, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing and is converted into an image visible to the naked eye. ISP can also optimize the image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 180.

The camera 180 is used to capture still images or videos. The object generates an optical image through the lens and is projected to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal. ISP outputs digital image signals to DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the head-mounted display 100 may include 1 or N cameras 180, and N is a positive integer greater than 1.

As shown in FIG. 1, the camera 180 may be installed on the side of the head-mounted display 100, and may also be installed at a position between two display screens on the head-mounted display 100. The camera 180 is used to capture images and videos within the viewing angle of the user 200 in real time. The head mounted display 100 generates a virtual image according to the captured real-time image and video, and displays the virtual image through the display screen 1100.

The processor 110 may determine the virtual image displayed on the display screen 1100 according to the still image or video image captured by the camera 180, in combination with the data (such as brightness, sound, etc.) acquired by the sensor module 130, to achieve superimposition on real world objects On the virtual image.

Among them, the digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the head-mounted display 100 selects the frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The head mounted display 100 may support one or more video codecs. In this way, the head-mounted display 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, for example, the transfer mode between human brain neurons, it can quickly process input information, and it can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the head-mounted display 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, and so on.

The memory 120 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the head mounted display 100 by running instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function. The data storage area can store data (such as audio data, phone book, etc.) created during the use of the head mounted display 100. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), and the like.

The head-mounted display 100 can implement audio functions through an audio module, a speaker, a microphone 140, a headphone interface, and an application processor. For example, music playback, recording, etc.

The audio module is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module can also be used to encode and decode audio signals. In some embodiments, the audio module may be provided in the processor 110, or part of the functional modules of the audio module may be provided in the processor 110.

Loudspeakers, also called "speakers", are used to convert audio electrical signals into sound signals. The head mounted display 100 can listen to music through a speaker, or listen to a hands-free call.

The microphone 140, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. The head mounted display 100 may be provided with at least one microphone 140. In other embodiments, the head-mounted display 100 can be provided with two microphones 140, which can realize noise reduction function in addition to collecting sound signals. In other embodiments, the head-mounted display 100 can also be equipped with three, four or more microphones 140 to collect sound signals, reduce noise, identify the source of sound, and realize the function of directional recording.

In some embodiments, the head-mounted display 100 may include a sound detector 132, which may detect and process a voice signal used to control a portable electronic device. For example, the sound detector may include a microphone 140. The portable head-mounted display 100 may use the microphone 140 to convert sound into electrical signals. The sound detector 132 can then process the electrical signal and recognize the signal as a command of the head-mounted display system. The processor 110 may be configured to receive a voice signal from the microphone 140. After receiving the voice signal, the processor 110 may run the sound detector 132 to recognize the voice command. For example, when a voice command is received, the head-mounted display 110 can obtain a contact on the stored user contact list, and the head-mounted display 100 can automatically dial the phone number of the contact.

The headphone jack is used to connect wired headphones. The headset interface can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

In some embodiments, the head-mounted display 100 may include one or more buttons 150 that can control the head-mounted display and provide the user with access to functions on the head-mounted display 100. The key 150 may be in the form of a button, a switch, a dial, and a touch or proximity sensing device (such as a touch sensor). Specifically, for example, the user 20 can turn on the display screen 1100 of the head mounted display 100 by pressing a button. The button 150 includes a power button, a volume button, and so on. The button 150 may be a mechanical button. It can also be a touch button. The head mounted display 100 may receive key input, and generate key signal input related to user settings and function control of the head mounted display 100.

In some embodiments, the head-mounted display 100 may include an input-output interface 160, and the input-output interface 160 may connect other devices to the head-mounted display 100 through appropriate components. Components may include audio/video jacks, data connectors, etc., for example.

The sound detector can detect and process the voice signal used to control the portable electronic device.

In some embodiments, the head-mounted display 100 may implement eye tracking. Specifically, an infrared device (such as an infrared transmitter) and an image acquisition device (such as a camera) can be used to detect the gaze direction of the eyeball.

The proximity light sensor may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The head mounted display 100 emits infrared light to the outside through the light emitting diode. The head mounted display 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the head mounted display 100. When insufficient reflected light is detected, the head-mounted display 100 can determine that there is no object near the head-mounted display 100. The head-mounted display 100 may use a proximity light sensor to detect gesture operations at a specific position of the head-mounted display 100 to achieve the purpose of associating the gesture operations with operation commands.

Distance sensor, used to measure distance. The head mounted display 100 can measure the distance by infrared or laser. In some embodiments, the head-mounted display 100 may use a distance sensor to measure distance to achieve fast focusing.

The gyroscope sensor can be used to determine the movement posture of the head mounted display 100. In some embodiments, the angular velocity of the head mounted display 100 around three axes (ie, x, y, and z axes) can be determined by a gyroscope sensor. The gyroscope sensor can also be used for navigation and somatosensory game scenes.

The ambient light sensor is used to sense the brightness of the ambient light. The head mounted display 100 can adaptively adjust the brightness of the display screen 1100 according to the perceived brightness of the ambient light. The ambient light sensor can also be used to automatically adjust the white balance when taking pictures.

The acceleration sensor can detect the magnitude of acceleration of the head mounted display 100 in various directions (generally three-axis). When the head mounted display 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to recognize the head-mounted display posture and apply to applications such as pedometers.

The temperature sensor is used to detect temperature. In some embodiments, the head-mounted display 100 uses the temperature detected by the temperature sensor to execute the temperature processing strategy. For example, when the temperature reported by the temperature sensor exceeds the threshold, the head-mounted display 100 reduces the performance of the processor located near the temperature sensor, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the head-mounted display 100 heats the battery 190 to prevent the head-mounted display 100 from shutting down abnormally due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the head mounted display 100 boosts the output voltage of the battery 190 to avoid abnormal shutdown caused by low temperature.

The focal length detection optical sensor 131 is used to detect the focal length of the eyeball of the user 200. In some embodiments, as shown in FIG. 1, the head-mounted display 100 may further include an infrared light source 1200. The focal length detection optical sensor 131 may cooperate with the infrared light source 1200 to detect the focal length of the eyeball of the user 200.

As shown in FIG. 1, the focal length detecting optical sensor 131 and the infrared light source 1200 may be arranged on the side of the display screen close to the eyeball. The number of the focal length detecting optical sensor 131 and the infrared light source 1200 may both be two, and each eyeball may correspond to a focal length detecting optical sensor 131 and the infrared light source 1200 for detecting the focal length of the eyeball.

In addition, as shown in FIG. 1, the head-mounted display 100 further includes an optical focusing module 1300, and the optical focusing module 1300 may include one or more optical lenses. In the embodiment of the present application, the head-mounted display 100 can adjust the optical power of the optical focusing module 1300 for the user's vision in response to a user's preset operation. Specifically, adjusting the optical power of the optical focusing module 1300 for the eyesight of the user can be accomplished by adjusting the optical lens, or it may be performed in other ways to adjust the optical power of the optical focusing module 1300 for the eyesight of the user. There is no limitation on the way of the optical power of the user's vision.

It can be understood that the positions and numbers of the focal length detection optical sensor 131, the infrared light source 1200 and the camera 180 on the head mounted display 100 shown in FIG. 1 are only for explaining the embodiment of the present application and should not constitute a limitation. The number of the focal length detection optical sensor 131 and the infrared light source 1200 may also be one. The number of one focal length detecting optical sensor 131 and one infrared light source 1200 can be used to detect the focal length of one eyeball, or to detect the focal length of two eyeballs at the same time.

In the embodiment of the present application, the head-mounted display 100 can adjust the optical power of the user's vision by adjusting the optical lens in the optical focusing module 1300. After the optical power is adjusted, there may be a problem that the user’s binocular images are not fused due to the inconsistent optical power of the user’s eyes. The head mounted display 100 can adjust the first display screen 1101 displayed on the first display screen 1101 in response to the user’s preset operation. The size of the picture and the second picture displayed on the second display screen 1102 is such that the user can see that the size of the first picture and the second picture are the same through the adjusted optical focusing module 1300. Then, the head-mounted display 100 may respond to the user's preset operation to determine the adjustment parameters according to the adjusted size of the first picture and/or the second picture, where the user's preset operation may indicate that the user is sure to see The operation where the size of the first picture is the same as that of the second picture. After that, the GPU can render the to-be-displayed interface on the first display screen 1101 and the second display screen 1102 according to the adjustment parameter. Finally, the first display screen 1101 and the second display screen 1102 can display the rendered interface, so that the binocular images of users with inconsistent optical powers of the two eyes can be merged, and the user will not feel dizzy and immersed when using the head-mounted display. The user experience is better, and the user’s eyes are not prone to fatigue and discomfort, which improves the user experience.

The head-mounted display 100 in FIG. 1 may include the image height adjustment system architecture shown in FIG. 2, and the method for controlling interface display provided by the embodiment of the present application may be implemented by the image height adjustment system architecture shown in FIG. 2. Referring to Figure 2, the image height adjustment system architecture is divided into a hardware part and a software part. Among them, the hardware part may include central processing unit (CPU) 100, GPU 101, display screen 102 and optical focusing module 103; the software part of the image height adjustment system may include SDK server (SDK server) 104, SDK 105, an interactive application 106 and a launcher 107.

The CPU 100 is the core hardware unit that controls and allocates all hardware resources of the image height adjustment system and performs general operations, and is the core of the calculation and control of the image height adjustment system.

Wherein, GPU 101 may be the GPU in FIG. 1; GPU 101 may be used to render an interface to be displayed on the display screen 102.

The display screen 102 may be the display screen 1100 in FIG. 1; the display screen 102 may include a first display screen 1101 and a second display screen 1102, which may provide display content for the two eyeballs of the user 200, respectively.

The optical focusing module 103 may be the optical focusing module 1300 in FIG. 1; the optical focusing module 103 may include one or more optical lenses.

Among them, the user can adjust the optical lens in the optical focusing module 103 to adapt the right and left eye optical systems for the user's binocular vision.

The SDK server 104 is the underlying SDK for supporting the functions of the interactive application 106 and the launcher 107.

The SDK 105 is used to support the running of the interactive application 106 and support the interactive application 106 to realize its functions.

The interactive application 106 is an application program for presenting an interactive interface to the user. Among them, the head-mounted display does not have to rely on the interactive application 106 to present the interactive interface to the user. The interactive interface can also be presented to the user in other forms. For example, the interactive interface can be integrated into other applications. Way to present the interactive interface. The embodiment of the present application does not limit the manner in which the interactive interface is presented to the user. In addition, the interactive application 106 may be designed through three-dimensional graphics production software. For example, the three-dimensional graphics production software may include Unity, Blender, and the like. Among them, the interactive interface can present pictures for the left and right eyes of the user, and the head-mounted display can adjust the size of the pictures in the interactive interface in response to user operations, so that the size of the pictures seen by the user through the left and right eye optical systems are consistent.

The launcher 107 is a virtual display interactive interface system, which can be used to display applications that have been installed in the image height adjustment system. For example, the launcher 107 can be a desktop of the image height adjustment system.

In the method for controlling interface display provided by the embodiment of the present application, first, the user can use the optical focusing module 103 to adapt the user's vision to form a left and right eye optical system adapted to the user's vision. Then, the head-mounted display can use the interactive application 106 to present the interactive interface to the user in response to the user's instruction, and adjust the size of the picture in the interactive interface in response to the user's operation, so that the user sees the size of the picture through the left and right eye optical systems. Unanimous. After that, the head-mounted display may determine the adjustment parameters according to the adjusted picture size in response to the user's operation of determining that the picture sizes they see are consistent. The GPU 101 can render the subsequent interface to be displayed according to the adjusted parameters. Finally, the first display 1101 and the second display 1102 display the rendered interface for the left and right eyes of the user, so that the user can pass the left and right optical The size of the interface seen by the system is the same, thus realizing the user's binocular image fusion.

Hereinafter, taking the electronic device as an example of a head-mounted display having the structure shown in FIG. 1 and FIG. 2, the method of controlling interface display provided by the embodiment of the present application will be described in detail with reference to the accompanying drawings. Referring to Fig. 3, the method for displaying the control interface may include:

301. The head-mounted display displays a first interface.

Among them, the first interface is an interactive interface that can be used by the user to adjust the size of the picture. 4A-4C, the first interface includes a first sub-interface 41 displayed on the first display screen and a second sub-interface 42 displayed on the second display screen. The first sub-interface 41 and the second sub-interface 42 may be interactive interfaces presented to the user’s left eye and right eye, respectively; wherein, the first sub-interface 41 may be displayed on the first display screen 1101 in FIG. 1, and the second sub-interface 41 may be displayed on the first display screen 1101 in FIG. The sub-interface 42 may be displayed on the second display screen 1102 in FIG. 1.

The first sub-interface 41 may include picture one, and the second sub-interface 42 may include picture two; the sizes of picture one and picture two displayed on the head-mounted display may be the same. At this time, picture one is the unadjusted initial picture one, and picture two is the unadjusted initial picture two. The embodiment of the present application does not limit the specific sizes of picture one and picture two.

The position of picture one on the first sub-interface 41 is the same as the position of picture two on the second sub-interface 42. For example, the vertical distances between the upper, lower, left, and right frames of the picture one and the first sub-interface 41 and the vertical distances between the upper, lower, left, and right frames of the picture two and the second sub-interface 42, respectively the same.

Among them, picture one and picture two can be various visual pictures, for example, can be a box, a circle, or a dashed and solid line. Among them, in Fig. 4A, picture 1 and picture 2 are squares; in Fig. 4B, picture 1 and picture 2 are circles; in Fig. 4C, picture 1 and picture 2 are dashed and solid lines.

In some embodiments, the colors of picture one and picture two may be different to make it easy for the user to distinguish. Optionally, the color of picture one is red, and the color of picture two is green; since the tones of red and green are quite different, red and green are a set of colors that are not easy to be misunderstood, which is convenient for users to distinguish. The colors of picture one and picture two can also be different color settings of other groups. For example, the group of colors can be red and yellow; or green and yellow; or yellow and blue. The color of picture one and picture two can also be the same, for example, both are red, yellow, green or blue. The embodiment of this application does not limit the colors of picture one and picture two.

After the first sub-interface 41 and the second sub-interface 42 are presented to the eyes of the user, the images of the first sub-interface 41 and the second sub-interface 42 seen by the user can be processed by the brain to form the images shown in Figure 4D-Figure 4F The visual images of the first sub-interface 41 and the second sub-interface 42 are combined. Since the refractive power of the user's eyes may be inconsistent, after the user uses the optical focusing module 103 shown in FIG. 2 to adapt the optical power of the user's eyes, the focal lengths of the left and right eye optical systems formed by the head-mounted display are different. Therefore, in the visual image formed by the user, the size of the picture 1 and the picture 2 may be different, so that the user sees that the picture 1 and the picture 2 do not merge. For example, FIG. 4D shows the visual image formed by the first sub-interface 41 and the second sub-interface 42 shown in FIG. 4A for the user, where the picture 1 and picture 2 seen by the user are not merged, and the picture 1 is larger than the picture 2. . FIG. 4E shows the visual images formed by the first sub-interface 41 and the second sub-interface 42 shown in FIG. 4B for the user, where the user sees that the picture 1 and the picture 2 do not merge, and the picture 1 is smaller than the picture 2. FIG. 4F shows the visual images formed by the first sub-interface 41 and the second sub-interface 42 shown in FIG. 4C for the user, where the user sees that the picture 1 and the picture 2 do not merge, and the picture 1 is larger than the picture 2.

Among them, the first sub-interface 41 and the second sub-interface 42 can have various forms, as illustrated below:

In one embodiment, referring to FIGS. 5-8, in addition to picture one, the first sub-interface 41 may also include a zoom control 511A and a determination control 512A; in addition to picture two, the second sub-interface 42 may also include a zoom control 511B, and determine control 512B.

In some implementations, the zoom control and the determination control on the sub-interface corresponding to the user's left eye may be on the left, and the zoom control and the determination control on the sub-interface corresponding to the user's right eye may be on the right to facilitate user adjustment. Referring to FIG. 5, the first sub-interface 41 may be an interface presented to the left eye of the user, and the zoom control 511A and the determination control 512A may be arranged on the left side of the first sub-interface 41. The second sub-interface 42 may be an interface presented for the right eye of the user, and the zoom control 511B and the determination control 512B may be arranged on the right side of the second sub-interface 42.

In other implementations, the zoom control and the determination control on the sub-interface corresponding to the user's left eye may also be on the right, and the zoom control and the determination control on the sub-interface corresponding to the user's right eye may also be on the left. For example, referring to FIG. 6, the first sub-interface 41 may be an interface presented to the left eye of the user, and the zoom control 511A and the determination control 512A may be arranged on the right side of the first sub-interface 41. The second sub-interface 42 may be an interface presented to the right eye of the user, and the zoom control 511B and the determination control 512B may be arranged on the left side of the second sub-interface 42.

In other implementation manners, referring to FIG. 7, the zoom control 511A and the determination control 512A may be arranged on the left and right sides of the first sub-interface 41. The zoom control 511B and the determination control 512B can be arranged on the left and right sides of the second sub-interface 42. Compared with setting the controls on both sides of the sub-interface, the one-sided setting is not prone to the problem that the controls on the two sub-interfaces are not merged.

It is understandable that the setting methods of the zoom control and the determination control are not limited to the above methods, and there may also be other setting methods of the zoom control and the determination control. For example, the zoom control and the determination control may be set on the upper side of the sub-interface and/or On the bottom side, the embodiment of the present application does not limit this.

Among them, the zoom control 511A may include a "+" control 5111A and a "-" control 5112A; among them, the "+" control 5111A is used to zoom in picture one, and the "-" control 5112A is used to zoom out picture one. The "+" control 5111A and the "-" control 5112A can zoom in and out of picture one by the step size δ. The zoom control 511B may include a "+" control 5111B and a "-" control 5112B; among them, the "+" control 5111B is used to zoom in the second picture, and the "-" control 5112B is used to zoom out the second picture. The "+" control 5111B and the "-" control 5112B can zoom in and out of picture two by the step size δ.

In some embodiments, the step size δ may indicate that the picture one or the picture two is zoomed by δ times. That is, each click of the "+" control 5111A indicates that the size of picture one is enlarged by δ times, and each click of the "-" control 5112A indicates that the size of picture one is reduced by δ times. Each click of the "+" control 5111B indicates that the size of the picture two is enlarged by δ times, and each click of the "-" control 5112B indicates that the size of the picture two is reduced by δ times. Optionally, the step size δ can be 0.01, 0.02, 0.03, etc., and the embodiment of the present application does not limit the size of the step size δ. For example, when δ is 0.01, each click of the "+" control 5111A indicates that the size of picture one is enlarged by 0.01 times; each click of the "-" control 5112A indicates that the size of picture one is reduced by 0.01 times. Among them, the operation in which the user clicks on the "+" control to instruct the picture to zoom in can be called a "+" operation, and the operation in which the user clicks on the "-" control to instruct the picture to zoom out can be called a "-" operation.

In other embodiments, the step size δ may also indicate the degree of increase/decrease for picture one or picture two. Optionally, the step length δ may be 0.5 mm, 0.8 mm, 1 mm, 1.5 mm, etc., and the embodiment of the present application does not limit the size of the step length δ. For example, when picture one is a box, when δ is 1 mm, each click on the "+" control 5111A indicates that the side length of picture one is increased by 1 mm; each click on the "-" control 5112A indicates that the side of picture one The side length is reduced by 1 mm.

In addition, the step length δ may also be in other forms, and the embodiment of the present application does not limit the form of the step length δ.

The determining control 512A can be used to validate the operations of the "+" control 5111A and the "-" control 5112A, and the determining control 512B can be used to validate the operations of the "+" control 5111B and the "-" control 5112B. That is, the changes made by the "+" operation and/or "-" operation to the size of picture one/picture two will only be displayed after clicking the OK control.

In another embodiment, the changes made by the "+" operation and/or "-" operation to the size of picture one/picture two will be displayed without clicking the OK control, that is, the "+" operation and/or " -" The changes made to the picture size by the operation will be displayed in real time. At this time, the confirm control 512A and the confirm control 512B can be used for the user to indicate that the size of the picture 1 and picture 2 they see is the same, that is, when the user is sure that the size of the picture 1 and picture 2 they see is the same, they can click OK Control 512A or confirm control 512B to confirm that the size of picture 1 and picture 2 are the same.

In some implementation manners, there may also be only one determination control on the first sub-interface 41 and the second sub-interface 42. For example, the first sub-interface 41 includes a determination control 512A, and the second sub-interface 42 does not include a determination control 512B. At this time, the confirm control is used for the user to indicate that the size of the picture 1 and the picture 2 seen is the same.

In another embodiment, referring to FIG. 8, neither the first sub-interface 41 nor the second sub-interface 42 includes a determination control. The changes to the picture size made by the "+" operation and/or "-" operation will be displayed in real time.

As an alternative, the left and right sides of the head-mounted display are provided with controls to replace the functions of the controls displayed on the first sub-interface and the second sub-interface; that is, the user can click the settings on the head-mounted display Control to adjust the size of the picture.

Optionally, the first sub-interface 41 may present parameter one indicating the size change of picture one, and the second sub-interface 42 may present parameter two indicating the size change of picture two.

In some implementations, parameter one may indicate how many times the size of picture one is the size of the unadjusted initial picture one, and parameter two may indicate how much the size of picture two is the size of the unadjusted initial picture two Times. As shown in Figure 5-7, parameter 1 and parameter 2 are both 1, indicating that the size of picture 1 and picture 2 is twice the initial picture size, that is, picture 1 and picture 2 are not adjusted at this time The initial picture.

In other implementations, parameter one may indicate the extent to which the size of picture one is increased/decreased compared to the size of the unadjusted initial picture one, and parameter two may indicate that the size of picture two is compared with the size of the unadjusted initial picture one. The extent to which the size of the adjusted initial picture 2 increases/decreases. As shown in FIG. 9, both parameter 1 and parameter 2 are 0, indicating that the sizes of picture 1 and picture 2 are not increased/decreased, that is, picture 1 and picture 2 are both unadjusted initial pictures at this time.

It can be understood that parameter one and parameter two may also be other types of parameters, and the embodiment of the present application does not limit the types of parameter one and parameter two.

In some embodiments, the first interface includes prompt information to prompt the user to adjust the size of the picture one and/or the picture two on the first interface, so that the size of the picture one and the picture two seen by the user are the same. For example, referring to Figure 10, the prompt message can include the text message "Please click the "+" control or the "-" control to adjust the size of the picture until you see the two pictures overlap." Or, after the head-mounted display displays the first interface, the head-mounted display may prompt the user to adjust the size of picture one and/or picture two on the first interface through voice information, so that the user sees picture one and picture two The sizes are the same. The embodiment of the present application does not limit the form of the prompt information.

302. The head-mounted display detects the user's first preset operation.

After the user sees that the picture 1 and the picture 2 are not fused through the left and right eye optical systems with both eyes, the first preset operation can be triggered to instruct the head mounted display to adjust the size of the picture 1 and/or the picture 2.

Among them, the first preset operation can be implemented in a variety of ways. The following exemplarily shows a specific way of implementing the first preset operation.

In an implementation manner, the first preset operation may be that the user presses the first button on the handle connected to the head-mounted display. For example, the user controls the joystick on the handle connected with the head-mounted display to hold the cursor on the "+" control, and then presses the OK button on the handle to complete the click on the "+" control on the interactive interface.

In another implementation manner, the first preset operation may be that the user clicks the first control on the interactive interface through a mouse. The first control may include a "+" control, a "-" control, or a confirm control on the interactive interface. For example, the user uses the mouse to move the cursor to the "+" control and click, double-click, or right-click the mouse to complete the click on the "+" control on the interactive interface.

In another implementation manner, the first preset operation may be that the user controls the cursor of the head sight connected to the head-mounted display to stay at the first control on the first interface for a preset time. For example, the first control is a "+" control. The user moves the cursor of the head sight connected to the head-mounted display to the "+" control on the interactive interface by swinging the head, and stays the cursor for a preset time to complete Click on the "+" control on the interactive interface. The preset time may be 1 second, 2 seconds, etc., and the embodiment of the present application does not limit the preset time.

In another implementation manner, the first preset operation may be that the user presses a second button on the head-mounted display. Among them, the second button can be a built-in button on the head-mounted display. For example, the second button can be a volume button. The user can zoom in on the picture on the interactive interface by pressing the volume "+" button, and pressing the volume The "-" key realizes the reduction of the picture on the interactive interface. That is, pressing the volume "+" key and the volume "-" key is equivalent to clicking the "+" control and "-" control on the interactive interface, so that the picture on the interactive interface can be zoomed in and out. Alternatively, the second button may also be a newly added button on the head-mounted display, and the embodiment of the present application does not limit the type of the second button. Optionally, the second button may also be a knob. For example, two knobs may be provided on the left and right sides of the head-mounted display, and the user can adjust the size of picture one/or picture two by rotating the two knobs.

In other words, the user can perform the first preset operation of instructing to adjust the size of picture one and/or picture two through a handle, a mouse, a head sight, or a button.

303. In response to the first preset operation, the head-mounted display adjusts the size of picture one and/or picture two.

Wherein, in response to the first preset operation, adjusting the size of picture one and/or picture two by the head-mounted display may include: in response to the first preset operation, the head-mounted display reduces the larger picture in picture one and picture two For example, in the case shown in Figure 4D, the head-mounted display can reduce the size of picture one. Or, the head-mounted display enlarges the size of the smaller picture in the first and second pictures; for example, in the case shown in FIG. 4D, the head-mounted display can enlarge the size of the second picture. Or, the head-mounted display reduces the size of the larger picture in picture one and picture two, and enlarges the size of the smaller picture in picture one and picture two. For example, in the case shown in FIG. 4D, the head-mounted display can reduce the size of picture one and enlarge the size of picture two.

That is to say, the head-mounted display can realize that the size of the picture 1 and the picture 2 seen by the user is the same by reducing the smaller picture, or enlarging the larger picture, or reducing the smaller picture and enlarging the larger picture.

The head-mounted display may adjust the size of the picture one and/or the picture two in response to the first preset operation, so that the size of the picture one and the picture two seen by the user through the left and right-eye optical systems are consistent. For example, referring to FIG. 11A, the head-mounted display can adjust the size of picture one to 0.9 times the size of initial picture one, and picture two is one time the size of initial picture two, that is, the size of picture two remains unchanged. After the adjustment shown in FIG. 11A, referring to FIG. 11B, the adjusted picture 1 and picture 2 seen by the user through the left and right eye optical systems overlap.

Wherein, in the above-mentioned "+" operation and/or "-" operation to change the size of the picture in real-time display solutions, the first preset operation may include clicking the "+" control/"-" control. In response to the above-mentioned first preset operation, the head-mounted display adjusts the size of picture one and/or picture two and displays the adjusted size of picture one and/or picture two in real time.

In the above-mentioned “+” operation and/or “-” operation to change the size of the picture, the change of the picture size needs to be validated by pressing the OK button. The first preset operation may include clicking the “+” control/“-” control, and clicking Determine the control. Wherein, the head-mounted display displays the adjusted size of picture one and/or picture two after detecting that the user clicks the OK button.

304. The head-mounted display detects the second preset operation of the user.

Wherein, the second preset operation can be used to trigger the head-mounted display to determine the adjustment parameter. Generally, the user can trigger the second preset operation after seeing the same size of the picture 1 and picture 2 through the left and right optical system. Therefore, the second preset operation can also be used to instruct the user to pass the left and right eye optics. The system sees the operation that the size of picture 1 and picture 2 are the same.

That is to say, when the size of the picture 1 and the picture 2 seen by the user through the left and right eye optical systems are the same, the second preset operation is triggered to instruct the user to confirm that the size of the picture 1 and the picture 2 seen is the same, thereby indicating The head mounted display determines the adjustment parameters.

Wherein, in the solution where the above-mentioned "+" operation and/or "-" operation changes the size of the picture need to be validated by pressing the OK button, the second preset operation may be that the user does not perform any operation within the predetermined time A, and the predetermined The time A may be, for example, 5 seconds, 10 seconds, etc. The embodiment of the present application does not limit the predetermined time A. Alternatively, the second preset operation may be that the user long presses the OK button 412A or 412B; for example, if the user long presses the OK button for more than a predetermined time B, the predetermined time B may be 2 seconds, 3 seconds, etc. B is not limited. Alternatively, the second preset operation may be that the user clicks the OK button 412A or 412B more than a predetermined number of times within the predetermined time C, for example, the user clicks the OK button 412A or 412B more than 3 times within 2 seconds. The embodiment of the present application does not limit the predetermined time C and the predetermined number of times. Or, the second preset operation may be the user's voice, for example, the user utters an operation of "determining that the pictures overlap" the voice content. Or, the second preset operation may be a user's gesture operation, and the fourth preset operation may be a user's gesture operation. For example, a camera is installed on the head-mounted display, and the head-mounted display can detect the user through the camera. "OK" gesture operation. The second preset operation may also be another type of user operation, and the embodiment of the present application does not limit the type of the second preset operation.

Wherein, in the above-mentioned “+” operation and/or “-” operation that changes the size of the picture in real-time display, the second preset operation may include the above-mentioned second preset operation type; or, the second preset operation It may also be that the user clicks the OK button 412A or 412B.

305. In response to the second preset operation, the head-mounted display determines the adjustment parameter according to the adjusted size of the first picture and/or the second picture.

That is, in response to the user's operation of determining that the adjusted picture 1 and picture 2 have the same size through the left and right eye optical systems, the head-mounted display can determine the size of the adjusted picture 1 and/or picture 2 Adjustment parameters.

In some embodiments, the adjustment parameters may include a first ratio of the size of the adjusted picture one to the size of the picture one before adjustment, and/or the second ratio of the size of the adjusted picture two to the size of the picture two before adjustment . In other words, the adjustment parameter may be parameter one and/or parameter two shown in FIGS. 5-7. When the adjustment parameter is a ratio, as shown in FIG. 11B, when the user sees that the picture 1 and the picture 2 overlap through the left and right eye optical systems, the parameter 1 is 0.9 and the parameter 2 is 1. In other words, the size of picture one becomes 0.9 times the size of picture one before adjustment, and the size of picture two remains unchanged.

The adjustment parameter may also be in other forms, and the embodiment of the present application does not limit the form of the adjustment parameter.

306. The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter.

Wherein, the second interface may be all interfaces to be displayed subsequently on the display screen, for example, a video playback interface, a picture viewing interface, a game interface, or other interfaces. The third sub-interface may be displayed on the first display screen 1101, and the fourth sub-interface may be displayed on the second display screen 1102.

In some embodiments, the head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters, which may include: the head-mounted display determines the third sub-interface and the fourth sub-interface according to the adjustment parameters. Size; the head-mounted display displays the third sub-interface and the fourth sub-interface.

That is to say, the head-mounted display determines the size of the third sub-interface and the fourth sub-interface according to the adjustment parameters, so that the size of the third sub-interface and the fourth sub-interface seen by the user are the same, and then displays the determined first sub-interface The three sub-interfaces and the fourth sub-interface allow the user's eyes to see the third sub-interface and the fourth sub-interface through the head-mounted display to merge. According to FIG. 11B, when the user sees that picture one and picture two overlap through the left and right optical systems, the parameter one is 0.9 and the parameter two is 1. Therefore, the size of the third sub-interface may be 0.9 times the preset size, and the size of the fourth sub-interface may be 1 time the preset size. Wherein, the preset size may be the size of the first display screen/the second display screen. For example, referring to FIG. 12, the size of the third sub-interface 121 may be 0.9 times the size of the first display screen, and the size of the fourth sub-interface 122 is 1 times the size of the first display screen. The preset size may also exist in other forms, and the embodiment of the present application does not limit the form of the preset size.

According to FIG. 2, it can be seen that the head-mounted display may also include an SDK, a GPU and a display screen for supporting the first interface. Among them, the head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters, specifically including: the head-mounted display sends the adjustment parameters to the SDK; the head-mounted display sends the adjustment parameters from the SDK to the SDK. GPU; the head-mounted display renders the third sub-interface and the fourth sub-interface in the second interface in the GPU according to the adjustment parameters; the head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface.

In other words, the head-mounted display sends the adjustment parameters to the SDK, and then to the GPU via the SDK, so that the GPU can render the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters. Finally, the second interface will be rendered. The third sub-interface and the fourth sub-interface in the second interface are displayed. At the same time, this solution does not add a hardware negative feedback module to the head-mounted display, and there is no additional cost.

In the solution described in steps 301-306, the head-mounted display can adjust the size of picture one and/or picture two in the first interface in response to the user's first preset operation, so that the user can see through the left and right eye optical systems Until the adjusted picture one and picture two have the same size. Then, the head-mounted display may respond to the second preset operation that the user determines that the sizes of the picture 1 and the picture 2 are the same, and determine the adjustment parameters according to the adjusted sizes of the picture 1 and/or the picture 2. After that, the head-mounted display may display the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameters. Therefore, users with inconsistent optical powers of both eyes can use the head-mounted display to adjust the size of the picture 1 and picture 2 seen by the user’s eyes through the optical system to the same size, and display the third sub in the second interface according to the adjusted parameters. Interface and the fourth sub-interface. The size of the third sub-interface and the fourth sub-interface seen by the user's eyes through the optical system are made to be the same, so that the binocular images of users with inconsistent optical powers of the eyes can be merged. Therefore, the user does not experience dizziness when using the head-mounted display, and the sense of immersion is better. At the same time, the user's eyes are not prone to fatigue and discomfort, which improves the user experience.

In some embodiments, before step 301, the method may further include: the head-mounted display detects a third preset operation of the user; in response to the third preset operation, the head-mounted display adjusts the head-mounted display to the user The optical power of vision.

That is to say, before the head-mounted display displays the first interface, the user can adjust the optical power of the head-mounted display for the user's binocular vision, so that the left and right eye optical systems of the head-mounted display can be adapted to the user respectively. The left and right eyes allow the user to see clearly through the left and right optical systems. In addition, because the head-mounted display adjusts the head-mounted display's optical power for the user's binocular vision, there may be over-correction/under-correction problems. The over-correction/under-correction problems will cause the fusion of picture one and picture two to change. Therefore, The head-mounted display performs image fusion adjustment after focusing, which can simultaneously adjust the fusion changes caused by over-correction/under-correction problems.

Wherein, the head-mounted display may also include an optical focusing module, and the optical focusing module may include an optical lens. In response to the third preset operation, the head-mounted display adjusting the optical power of the head-mounted display for the vision of the user may include: in response to the third preset operation, the head-mounted display adjusts an optical lens to adjust the head-mounted display The optical power for the user's vision.

Wherein, the third preset operation may be an operation in which the user presses a button on the handle connected to the head-mounted display; or, the third preset operation may be an operation in which the user presses a button on the head-mounted display; or, The third preset operation may be an operation in which the user rotates the knob on the head-mounted display. It is understandable that the third preset operation may also be other types of preset operations, and the embodiment of the present application does not limit the type of the third preset operation.

Wherein, when the third preset operation is the operation of the user pressing a button on the handle/head-mounted display, adjusting the optical lens of the head-mounted display may include: the head-mounted display automatically adjusts the optical lens so that the user can adjust the optical lens after the adjustment. The optical lens has a clear vision. Or, the head-mounted display changes the position of the optical lens from the user's eyes, and the process can be repeated until the user sees clearly through the adjusted optical lens. Or, the head-mounted display changes the focal length of the optical lens by applying a voltage (for example, liquid lens voltage-type focusing), and this process can be repeated until the user sees clearly through the adjusted optical lens. When the third preset operation is the operation of the user rotating the knob on the head-mounted display, adjusting the optical lens of the head-mounted display may include: the head-mounted display moves the optical lens away from the user's eyes according to the user's rotation operation ( For example, curved groove type/thread type mechanical focusing) to change the focal length of the optical path system, and this process can be repeated until the user sees clearly through the adjusted optical lens. It can be understood that there are other ways of adjusting the optical lens of the head-mounted display, and the embodiment of the present application does not limit the way of adjusting the optical lens of the head-mounted display.

In other embodiments, before step 301, the method may further include: the head-mounted display detects a fourth preset operation of the user, and the fourth preset operation is used to instruct the head-mounted display to display the first interface.

For example, the fourth preset operation may be an operation of the user clicking the icon of the interactive application 106 shown in FIG. 2. Or, the fourth preset operation may be an operation in which the user clicks an icon of another application program integrated with an interactive interface. Alternatively, the fourth preset operation may be a voice operation of the user, for example, an operation of the user uttering the voice content of “display an interactive interface”. Alternatively, the fourth preset operation may be a user's gesture operation. For example, a camera is installed on the head-mounted display, and the head-mounted display may detect the user's gesture operation of drawing a circle track through the camera. The fourth preset operation may also be other types of user operations, and the embodiment of the present application does not limit the manner of the fourth preset operation.

In other words, the head-mounted display can display the first interface according to the user's active instruction, and provide the user with an interactive interface that can be used to adjust the size of the picture.

In some embodiments, in response to the user's third preset operation, the head-mounted display may automatically display the first interface after adjusting the optical power of the head-mounted display for the user's vision;

In other embodiments, in response to the user's third preset operation, the head-mounted display adjusts the optical power of the head-mounted display for the user's vision, and then detects the user's fourth preset operation, the head-mounted display The first interface is displayed in response to the fourth preset operation.

It can be understood that, in order to implement the above-mentioned functions, the electronic device includes hardware and/or software modules corresponding to each function. In combination with the algorithm steps of the examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Those skilled in the art can use different methods for each specific application in combination with the embodiments to implement the described functions, but such implementation should not be considered as going beyond the scope of the present application.

In this embodiment, the electronic device may be divided into functional modules according to the foregoing method examples. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 13 shows a schematic diagram of a possible composition of the electronic device 1400 involved in the foregoing embodiment. As shown in FIG. 13, the electronic device 1400 may include: a display unit 1401, detection unit 1402, adjustment unit 1403, and determination unit 1404.

Wherein, the display unit 1401 may be used to support the electronic device 1400 to perform the

above steps

301, 306, etc., and/or other processes used in the technology described herein.

The detection unit 1402 may be used to support the electronic device 1400 to perform the

above steps

302, 304, etc., and/or other processes used in the technology described herein.

The adjustment unit 1403 may be used to support the electronic device 1400 to perform the above-mentioned step 303, etc., and/or be used in other processes of the technology described herein.

The determining unit 1404 may be used to support the electronic device 1400 to perform the above-mentioned step 305, etc., and/or other processes used in the technology described herein.

It should be noted that all relevant content of the steps involved in the foregoing method embodiments can be cited in the functional description of the corresponding functional module, and will not be repeated here.

The electronic device 1400 provided in this embodiment is used to execute the above-mentioned method for controlling interface display, and therefore can achieve the same effect as the above-mentioned implementation method.

In the case of adopting an integrated unit, the electronic device 1400 may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device 1400. For example, it can be used to support the electronic device 1400 to execute the steps performed by the display unit 1401, the detection unit 1402, the adjustment unit 1403, and the determination unit 1404. The storage module can be used to support the electronic device 1400 to store program codes and data. The communication module may be used to support communication between the electronic device 1400 and other devices, for example, communication with a wireless access device.

Among them, the processing module may be a processor or a controller. It can implement or execute various exemplary logical blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, and other devices that interact with other electronic devices.

In an embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a head-mounted display having the structure shown in FIG. 1.

The embodiments of the present application also provide a computer storage medium, the computer storage medium stores computer instructions, when the computer instructions run on the electronic device, the electronic device executes the above-mentioned related method steps to implement the control interface in the above-mentioned embodiment Show method.

The embodiment of the present application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute the above-mentioned related steps, so as to realize the method of controlling interface display executed by the electronic device in the above-mentioned embodiment.

In addition, the embodiments of the present application also provide a device. The device may specifically be a chip, component or module. The device may include a processor and a memory connected to each other. The memory is used to store computer execution instructions. When the device is running, The processor can execute the computer-executable instructions stored in the memory, so that the chip executes the method of controlling interface display executed by the electronic device in the foregoing method embodiments.

Among them, the electronic device, computer storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding method provided above. The beneficial effects of the method will not be repeated here.

Through the description of the above embodiments, those skilled in the art can understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function can be assigned to different functions according to needs. The function module is completed, that is, the internal structure of the device is divided into different function modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, for example, multiple units or components may be divided. It can be combined or integrated into another device, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate. The parts displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place, or they may be distributed to multiple different places. . Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product, and the software product is stored in a storage medium. It includes several instructions to make a device (may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage media include: U disk, mobile hard disk, read only memory (read only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

The above content is only the specific implementation of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Covered in the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for controlling interface display, characterized in that it is applied to a head-mounted display having a first display screen and a second display screen; the method includes:

The head-mounted display displays a first interface, and the first interface includes a first sub-interface displayed on the first display screen and a second sub-interface displayed on the second display screen, and the first sub-interface includes The first picture, the second sub-interface includes a second picture; wherein the size of the first picture and the second picture are the same;

The head-mounted display detects the user's first preset operation;

In response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture;

The head-mounted display detects a second preset operation of the user;

In response to the second preset operation, the head-mounted display determines an adjustment parameter according to the adjusted size of the first picture and/or the second picture;

The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter; wherein, the third sub-interface is displayed on the first display screen, and the fourth sub-interface The interface is displayed on the second display screen.
The method according to claim 1, wherein the head-mounted display displaying the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter comprises:

Determining, by the head-mounted display, the sizes of the third sub-interface and the fourth sub-interface according to the adjustment parameter;

The head-mounted display displays the third sub-interface and the fourth sub-interface.
The method according to claim 1 or 2, characterized in that, before the head-mounted display displays the first interface, the method further comprises:

The head-mounted display detects a third preset operation of the user;

In response to the third preset operation, the head-mounted display adjusts the optical power of the head-mounted display for the vision of the user.
The method according to claim 3, wherein the head-mounted display further comprises an optical focusing module, the optical focusing module includes an optical lens; said in response to the third preset operation, The adjusting, by the head-mounted display, the optical power of the head-mounted display for the eyesight of the user includes:

In response to the third preset operation, the head-mounted display adjusts the optical lens.
The method according to claim 1 or 2, characterized in that, before the head-mounted display displays the first interface, the method further comprises:

The head-mounted display detects a fourth preset operation of the user, and the fourth preset operation is used to instruct the head-mounted display to display the first interface.
The method according to any one of claims 1-5, wherein the first interface includes prompt information, and the prompt information is used to prompt the user to adjust the first picture and/or the second picture. The size of the picture.
The method according to any one of claims 1-6, wherein, in response to the first preset operation, the head-mounted display adjusts the size of the first picture and/or the second picture ,include:

In response to the first preset operation, the head-mounted display reduces the size of the larger one of the first picture and the second picture;

Or, the head-mounted display enlarges the size of the smaller one of the first picture and the second picture;

Alternatively, the head-mounted display reduces the size of the larger picture of the first picture and the second picture, and enlarges the size of the smaller picture of the first picture and the second picture.
The method according to any one of claims 1-7, wherein the first preset operation comprises:

The user presses the first button on the handle connected to the head-mounted display; or,

The user clicks the first control on the first interface by using a mouse; or,

The user controls the cursor of the head sight connected to the head-mounted display to stay at the first control on the first interface for a preset time; or,

The user presses a second button on the head-mounted display.
The method according to any one of claims 1-8, wherein the adjustment parameter comprises a first ratio of the size of the first picture after adjustment to the first picture before adjustment, and/or The second ratio of the size of the second picture after adjustment to the size of the second picture before adjustment.
The method according to any one of claims 1-9, wherein the colors of the first picture and the second picture are different.
The method according to any one of claims 1-10, wherein the head-mounted display further comprises a software development kit (SDK) for supporting the first interface, a graphics processor GPU, and a display screen;

The head-mounted display displaying the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter specifically includes:

Sending, by the head-mounted display, the adjustment parameter to the SDK;

Sending, by the head-mounted display, the adjustment parameters from the SDK to the GPU;

Rendering the third sub-interface and the fourth sub-interface in the second interface by the head-mounted display in the GPU according to the adjustment parameter;

The head-mounted display displays the third sub-interface and the fourth sub-interface in the second interface.
A head-mounted display is characterized by comprising:

A first display screen and a second display screen; one or more processors; a memory; a plurality of application programs; and one or more computer programs, wherein the one or more computer programs are stored in the memory, so The one or more computer programs include instructions that, when executed by the head-mounted display, cause the head-mounted display to perform the following steps:

A first interface is displayed, the first interface includes a first sub-interface displayed on the first display screen and a second sub-interface displayed on the second display screen, the first sub-interface includes a first picture, the The second sub-interface includes a second picture; wherein the size of the first picture and the second picture are the same;

The first preset operation of the user is detected;

In response to the first preset operation, adjusting the size of the first picture and/or the second picture;

The second preset operation of the user is detected;

In response to the second preset operation, determining an adjustment parameter according to the adjusted size of the first picture and/or the second picture;

The third sub-interface and the fourth sub-interface in the second interface are displayed according to the adjustment parameters; wherein, the third sub-interface is displayed on the first display screen, and the fourth sub-interface is displayed on the second interface. Appears on the display.
The head-mounted display according to claim 12, wherein the displaying the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter comprises:

Determining the sizes of the third sub-interface and the fourth sub-interface according to the adjustment parameter;

The third sub-interface and the fourth sub-interface are displayed.
The head-mounted display according to claim 12 or 13, wherein before displaying the first interface, the head-mounted display further performs the following steps:

The third preset operation of the user is detected;

In response to the third preset operation, adjusting the optical power of the head-mounted display for the vision of the user.
The head-mounted display of claim 14, wherein the head-mounted display further comprises an optical focusing module, the optical focusing module includes an optical lens; the response to the third preset Suppose the operation to adjust the optical power of the head-mounted display for the eyesight of the user includes:

In response to the third preset operation, the optical lens is adjusted.
The head-mounted display according to claim 12 or 13, wherein before the head-mounted display displays the first interface, the head-mounted display further executes the following steps:

A fourth preset operation of the user is detected, and the fourth preset operation is used to instruct the head-mounted display to display the first interface.
The head-mounted display according to any one of claims 12-16, wherein the first interface includes prompt information, and the prompt information is used to prompt the user to adjust the first picture and/or the State the size of the second picture.
The head-mounted display according to any one of claims 12-17, wherein in response to the first preset operation, adjusting the size of the first picture and/or the second picture comprises:

In response to the first preset operation, reduce the size of the larger one of the first picture and the second picture;

Or, enlarge the size of the smaller one of the first picture and the second picture;

Or, reduce the size of the larger picture of the first picture and the second picture, and enlarge the size of the smaller picture of the first picture and the second picture.
The head-mounted display according to any one of claims 12-18, wherein the first preset operation comprises:

The user presses the first button on the handle connected to the head-mounted display; or,

The user clicks the first control on the first interface by using a mouse; or,

The user controls the cursor of the head sight connected to the head-mounted display to stay at the first control on the first interface for a preset time; or,

The user presses a second button on the head-mounted display.
The head-mounted display according to any one of claims 12-19, wherein the adjustment parameter comprises a first ratio of the size of the first picture after adjustment to the first picture before adjustment, And/or the second ratio of the size of the second picture after adjustment to the size of the second picture before adjustment.
The head-mounted display according to any one of claims 12-20, wherein the colors of the first picture and the second picture are different.
The head-mounted display according to any one of claims 12-21, wherein the head-mounted display further comprises a software development kit SDK for supporting the first interface, a graphics processor GPU and a display Screen;

The displaying the third sub-interface and the fourth sub-interface in the second interface according to the adjustment parameter specifically includes:

Sending the adjustment parameters to the SDK;

Sending the adjustment parameters from the SDK to the GPU;

Rendering the third sub-interface and the fourth sub-interface in the second interface in the GPU according to the adjustment parameter;

The third sub-interface and the fourth sub-interface in the second interface are displayed.
A computer storage medium, characterized by comprising computer instructions, which when the computer instructions run on an electronic device, cause the electronic device to execute the method for controlling interface display according to any one of claims 1-11 .
A computer program product, characterized in that, when the computer program product runs on a computer, the computer is caused to execute the method for controlling interface display according to any one of claims 1-11.