CN113315999A

CN113315999A - Virtual reality optimization method, device, equipment and storage medium

Info

Publication number: CN113315999A
Application number: CN202110580701.7A
Authority: CN
Inventors: 赵斌; 栗霖; 杨秋霞
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2021-05-26
Filing date: 2021-05-26
Publication date: 2021-08-27

Abstract

The application provides a virtual reality optimization method, a device, equipment and a storage medium, wherein the method comprises the steps of obtaining network delay of a VR current frame corresponding to VR equipment, further determining total delay of the current frame according to the network delay, determining a target VR frame rate according to the total delay of the VR current frame, and adjusting display driving parameters of the VR equipment according to the target VR frame rate to enable the VR frame rate of the VR equipment to be equal to the target VR frame rate, so that a VR instruction is sent to the VR equipment to enable the VR equipment to perform VR operation according to the VR instruction. Under the condition of low network delay, the frame rate can be properly improved and the user fluency can be improved by adopting the embodiment of the application.

Description

Virtual reality optimization method, device, equipment and storage medium

Technical Field

The present application relates to the field of virtual reality technologies, and in particular, to a virtual reality optimization method, apparatus, device, and storage medium.

Background

At present, Virtual Reality (VR) technology is widely used in the fields of video entertainment, medical treatment, and the like, for example, VR technology can be used in a myopia training instrument to realize the auxiliary treatment of myopia through the adjustment of the image distance of images.

The current 5G large bandwidth and low latency feature gives VR a larger development space. VR has high delay requirement, and a typical parameter standard in the industry is the total delay from the head rotation to the frame display to the human eye, and if the total delay is large, the user experience may be dizzy. The 5G low latency mainly uses an Edge Computing (MEC) technology to sink Computing resources to the user side, so that uplink and downlink latencies can be effectively reduced.

However, when the MEC carries the service, there is a difference in the sinking position, resulting in a difference in network delay. Even if the sinking position is the same, the network may be unstable for a period of time, and the delay is slightly different from the actual situation. This floating network delay can have a severe impact on VR experience.

Disclosure of Invention

In order to solve the problems in the prior art, the application provides a virtual reality optimization method, a virtual reality optimization device, virtual reality optimization equipment and a storage medium.

In a first aspect, an embodiment of the present application provides a virtual reality optimization method, which includes the following steps:

acquiring network delay of a VR current frame corresponding to VR equipment to be processed, and determining total VR current frame delay according to the network delay;

determining a target VR frame rate according to the total delay of the VR current frame;

adjusting display driving parameters of the VR equipment to be processed according to the target VR frame rate so that the VR frame rate of the VR equipment to be processed is equal to the target VR frame rate;

sending a VR instruction to the VR equipment to be processed so that the VR equipment to be processed performs VR operation according to the VR instruction

In one possible implementation manner, the network delay includes a first time when the to-be-processed VR device uploads the acquired user head pose to a server and a second time when the server transmits the encoded picture to the to-be-processed VR device;

the determining of the total delay of the VR current frame according to the network delay includes:

acquiring third time when the VR device to be processed acquires the head gesture of the user, fourth time when the server processes the head gesture of the user, fifth time when the server renders pictures according to processing results, sixth time when the server encodes rendered pictures, and seventh time when the VR device to be processed decodes the encoded pictures;

and taking the sum of the first time, the second time, the third time, the fourth time, the fifth time, the sixth time and the seventh time as the total time delay of the VR current frame corresponding to the VR device to be processed.

In one possible implementation manner, before the sending the VR instruction to the to-be-processed VR device, the method further includes:

judging whether the target VR frame rate is larger than a preset frame rate threshold value or not;

the sending of the VR instruction to the VR device to be processed comprises:

if the target VR frame rate is smaller than or equal to the preset frame rate threshold, sending a time warping instruction and the VR instruction to the VR equipment to be processed, so that the VR equipment to be processed performs VR operation according to the time warping instruction and the VR instruction.

In one possible implementation manner, the adjusting the display driving parameters of the VR device to be processed according to the target VR frame rate includes:

determining an adjustable parameter in display driving parameters of the VR device to be processed;

and adjusting the adjustable parameter according to the target VR frame rate and the adjustable range corresponding to the adjustable parameter, so that the VR frame rate of the VR equipment to be processed is equal to the target VR frame rate.

In one possible implementation manner, the adjusting the adjustable parameter according to the target VR frame rate and an adjustable range corresponding to the adjustable parameter includes:

according to the expression:

and an adjustable range corresponding to the adjustable parameter, and adjusting the adjustable parameter, where the adjustable parameter includes mipspeed, hsync, hfp, hbp, vsync, vfp, and vbp, where F is the target VR frame rate, mipspeed is the transmission rate of each data channel for video playing by the VR device to be processed, lane _ num is the number of data channels for video playing by the VR device to be processed, width is the width of a display area for video playing by the VR device to be processed, hsync is a horizontal synchronization signal for video playing by the VR device to be processed, hfp is a front shoulder of a horizontal synchronization signal for video playing by the VR device to be processed, hbp is a rear shoulder of a horizontal synchronization signal for video playing by the VR device to be processed, height is the height of a display area for video playing by the VR device to be processed, and vsync is a frame synchronization signal for video playing by the VR device to be processed, vfp is the frame synchronization signal front shoulder of the VR equipment to be processed for video playing, vbp is the frame synchronization signal back shoulder of the VR equipment to be processed for video playing, and bus _ width is the bus width of the VR equipment to be processed for video playing.

In one possible implementation manner, the determining a target VR frame rate according to the VR current frame total delay includes:

according to the expression:

determining the target VR frame rate F, T (total) as the VR current frame total delay.

In a second aspect, an embodiment of the present application provides a virtual reality optimization apparatus, including:

the delay determining module is used for acquiring the network delay of the VR current frame corresponding to the VR equipment to be processed and determining the total delay of the VR current frame according to the network delay;

a frame rate determination module, configured to determine a target VR frame rate according to the VR current frame total delay;

a parameter adjusting module, configured to adjust, according to the target VR frame rate, a display driving parameter of the VR device to be processed, so that the VR frame rate of the VR device to be processed is equal to the target VR frame rate;

and the VR module is used for sending a VR instruction to the VR equipment to be processed so that the VR equipment to be processed performs VR operation according to the VR instruction.

the delay determining module is specifically configured to:

In a possible implementation manner, the VR system further includes a frame rate determining module, configured to determine whether the target VR frame rate is greater than a preset frame rate threshold before the VR module sends a VR instruction to the to-be-processed VR device;

the VR module further to:

In a possible implementation manner, the parameter adjusting module is specifically configured to:

according to the expression:

In a possible implementation manner, the frame rate determining module is specifically configured to:

according to the expression:

In a third aspect, an embodiment of the present application provides a virtual reality optimization apparatus, including:

a processor;

a memory; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program causes a server to execute the method according to the first aspect.

In a fifth aspect, the present application provides a computer program product, which includes computer instructions for executing the method of the first aspect by a processor.

According to the virtual reality optimization method, the virtual reality optimization device, the virtual reality optimization equipment and the storage medium, the network delay of a VR current frame corresponding to VR equipment is obtained, further, the total delay of the current frame is determined according to the network delay, a target VR frame rate is determined according to the total delay of the VR current frame, and the display driving parameters of the VR equipment are adjusted according to the target VR frame rate, so that the VR frame rate of the VR equipment is equal to the target VR frame rate, therefore, a VR instruction is sent to the VR equipment, and the VR equipment carries out VR operation according to the VR instruction. Under the condition of low network delay, the frame rate can be properly improved and the user fluency can be improved by adopting the embodiment of the application. In addition, the embodiment of the application has good self-adaptability, does not need to intervene every time, and ensures the stability and fluency of VR experience of the user due to no perception of the user.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a schematic diagram of a cloud VR provided in an embodiment of the present application;

fig. 2 is a schematic diagram of network delay corresponding to a VR device according to an embodiment of the present application;

fig. 3 is a schematic diagram of a virtual reality optimization system architecture provided in an embodiment of the present application;

fig. 4 is a schematic flowchart of a virtual reality optimization method according to an embodiment of the present application;

fig. 5 is a schematic flowchart of another virtual reality optimization method provided in an embodiment of the present application;

fig. 6 is a schematic structural diagram of a virtual reality optimization apparatus according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of another virtual reality optimizing apparatus provided in an embodiment of the present application;

fig. 8A is a schematic diagram of a basic hardware architecture of a virtual reality optimization apparatus provided in the present application;

fig. 8B is a schematic diagram of a basic hardware architecture of another virtual reality optimizing apparatus provided in the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terms "first," "second," "third," and "fourth," if any, in the description and claims of this application and the above-described figures are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The current 5G large bandwidth and low latency feature gives VR a larger development space. VR has high delay requirement, and a typical parameter standard in the industry is the total delay from the head rotation to the frame display to the human eye, and if the total delay is large, the user experience may be dizzy. The 5G low delay mainly uses the MEC technology to sink the computing resources to the user side, so that the uplink and downlink delay can be effectively reduced.

For the case that the local VR device performs rendering calculation by using a local Personal Computer (PC), because the rendering is local, there is no network delay, and this ensures better user experience under the circumstance of ensuring calculation power. The disadvantage is that the user needs to purchase a high-specification PC or a high-end Graphics Processing Unit (GPU) Graphics card, and the use threshold is high. Cloud VR appears to reduce the user usage threshold, and directly uses light-weight VR devices, and high-load work such as rendering is placed on the MEC side (edge node side), as an example, as shown in fig. 1. However, at the present stage, the MEC may sink to different network nodes due to cost problems or service requirements, and the different network nodes may cause network conditions to be very different. For example, the MEC may place a provincial data center according to the needs of the service, and this delay is relatively high and may also sink to a city data center, so that the delay is further reduced and then further sink to a regional data center, and the delay is further reduced. This is the lowest delay if the final MEC sinks to the 5G base station side.

However, when the MEC carries the traffic, there is a difference in the sinking position, resulting in a difference in network delay. Illustratively, as shown in fig. 2, experiencing a good VR experience requires: cloud VR latency is as small as possible. In the MEC case, the uplink transmission and the downlink transmission will increase the delay, and the increased delay is determined according to the MEC physical location, and the minimum delay that can be achieved is less than or equal to 1 ms. In addition, even if the sinking position is the same, network instability in a period of time can be met, and the delay is slightly different from the actual situation.

This floating network delay can have a severe impact on VR experience. For example, the system frame rate of the current stage VR device is mostly 60Hz to 90Hz, i.e. the delay of each frame is between 11.1ms to 16.6 ms. Therefore, the action capture is required to be finished within 11.1ms-16.6ms after being transmitted through the network, then transmitted back to the device end through the cloud computing rendering and the like, and if the action capture is more than 16.6ms, the VR device cannot receive the current image in the current frame in time, so that no image is displayed on a screen, and the problems of frame loss and frame skipping occur.

However, the existing scheme is fixed in frame rate of the VR-side system regardless of changes in network conditions. Therefore, when the network is not ideal, the device can only passively accept the frame loss problem, and when the network is good, the frame rate is not sufficiently improved, so that the user does not enjoy a sufficiently smooth picture.

In order to solve the above problem, an embodiment of the present application provides a virtual reality optimization method, which can dynamically adjust a VR frame rate of a VR device according to a network delay corresponding to the VR device, and can avoid frame skipping and frame loss caused by an unsatisfactory network or a large delay under the condition of a fixed VR frame rate. Under the condition of low network delay, the frame rate can be properly improved and the user fluency can be improved by adopting the embodiment of the application. In addition, the embodiment of the application has good self-adaptability, does not need to intervene every time, and ensures the stability and fluency of VR experience of the user due to no perception of the user.

Optionally, the virtual reality optimization method provided in the present application may be applied to the virtual reality optimization system architecture diagram shown in fig. 3, and as shown in fig. 3, the system may include at least one of a receiving device 301, a processing device 302, and a display device 303.

In a specific implementation process, the receiving apparatus 301 may be an input/output interface, and may also be a communication interface, and may be configured to receive information of the VR device to be processed, for example, an identity of the VR device to be processed.

The processing device 302 may determine the VR device to be processed through the receiving device 301, and then may dynamically adjust the VR frame rate of the VR device according to the network delay corresponding to the VR device, so that the VR device performs VR operation based on the adjusted VR frame rate, and may avoid the frame skipping and frame losing problems caused by an unsatisfactory network or a large delay under the condition of a fixed VR frame rate. In addition, when the network latency is low, the processing device 302 can appropriately increase the frame rate and improve the user fluency.

The display device 303 may be configured to display a network delay, a VR frame rate, and the like corresponding to the VR device.

The display device may also be a touch display screen for receiving user instructions while displaying the above-mentioned content to enable interaction with a user.

It should be understood that the processing device may be implemented by a processor reading instructions in a memory and executing the instructions, or may be implemented by a chip circuit.

The system is only an exemplary system, and when the system is implemented, the system can be set according to application requirements.

It is to be understood that the illustrated structure of the embodiment of the present application does not form a specific limitation to the architecture of the virtual reality optimization system. In other possible embodiments of the present application, the foregoing architecture may include more or less components than those shown in the drawings, or combine some components, or split some components, or arrange different components, which may be determined according to practical application scenarios, and is not limited herein. The components shown in fig. 2 may be implemented in hardware, software, or a combination of software and hardware.

In addition, the system architecture described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not form a limitation on the technical solution provided in the embodiment of the present application, and it can be known by a person skilled in the art that the technical solution provided in the embodiment of the present application is also applicable to similar technical problems along with the evolution of the system architecture and the appearance of new service scenarios.

The technical solutions of the present application are described below with several embodiments as examples, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 4 is a flowchart of a virtual reality optimization method according to an embodiment of the present application, where the method may be performed by any device that performs the virtual reality optimization method, and the device may be implemented by software and/or hardware. As shown in fig. 4, based on the system architecture shown in fig. 3, the virtual reality optimization provided by the embodiment of the present application may include the following steps:

s401: and acquiring the network delay of the VR current frame corresponding to the VR equipment to be processed, and determining the total delay of the VR current frame according to the network delay.

The VR device to be processed is a VR device that needs to dynamically adjust a VR frame rate according to a network delay corresponding to the VR device, and may specifically be determined according to an actual situation, which is not particularly limited in the embodiments of the present application.

The network delay includes a first time when the to-be-processed VR device uploads the collected head pose of the user to the server and a second time when the server transmits the encoded picture to the to-be-processed VR device.

In the embodiment of the present application, the execution main body is taken as the processing apparatus as an example. When determining the total VR current frame delay according to the network delay, the processing device may first obtain a third time at which the to-be-processed VR device acquires the head pose of the user, a fourth time at which the server processes the head pose of the user, a fifth time at which the server performs image rendering according to a processing result, a sixth time at which the server encodes a rendered image, and a seventh time at which the to-be-processed VR device decodes the encoded image.

Illustratively, as shown in fig. 2, the third time T (motion capture): the method comprises the steps that VR equipment to be processed collects the time of the head gesture of a user;

first time T (uplink delay): time for uploading the head posture of the user to a server by the VR equipment to be processed;

fourth time T (logical calculation): the time for the server to process the head gestures of the user;

fifth time T (real-time rendering): the server carries out picture rendering time according to the processing result;

sixth time T (coding): the time for the server to encode the rendered picture;

second time T (downlink delay): the time for transmitting the coded picture to VR equipment to be processed by the server;

seventh time T (device decode): and time for the VR device to decode the coded picture.

After the processing device obtains the first time, the second time, the third time, the fourth time, the fifth time, the sixth time, and the seventh time, the sum of the first time, the second time, the third time, the fourth time, the fifth time, the sixth time, and the seventh time may be used as the total delay time of the VR current frame corresponding to the to-be-processed VR device. T (total) · T (motion capture) + T (uplink delay) + T (logical calculation) + T (real-time rendering) + T (coding) + T (downlink delay) + T (device decoding).

In addition, the server may be a cloud server.

S402: and determining the target VR frame rate according to the total time delay of the VR current frame.

Here, the processing means may be according to the expression:

and determining the target VR frame rate F, wherein T (total) is the total delay of the VR current frame.

S403: and adjusting the display driving parameters of the VR equipment to be processed according to the target VR frame rate, so that the VR frame rate of the VR equipment to be processed is equal to the target VR frame rate.

In this embodiment of the application, after the processing device obtains the VR frame rate F, the processing device modifies the configuration file at the bottom layer of the VR device based on the F value to change the frame rate, so as to dynamically adjust the VR frame rate of the VR device according to the network delay corresponding to the VR device, and avoid the frame skipping and frame losing problems caused by an unsatisfactory network or a large delay under the condition of a fixed VR frame rate.

For example, the processing device may determine an adjustable parameter in display driving parameters of the VR device to be processed, so as to adjust the adjustable parameter according to the target VR frame rate and an adjustable range corresponding to the adjustable parameter, so that the VR frame rate of the VR device to be processed is equal to the target VR frame rate.

Wherein, the processing device may be according to the expression:

and an adjustable range corresponding to the adjustable parameter, and adjusting the adjustable parameter, where the adjustable parameter includes mipspeed, hsync, hfp, hbp, vsync, vfp, and vbp, where F is the target VR frame rate, mipspeed is the transmission rate of each data channel for video playback by the VR device to be processed, lane _ num is the number of data channels for video playback by the VR device to be processed, width is the width of a display area for video playback by the VR device to be processed, hsync is a horizontal synchronization signal for video playback by the VR device to be processed, hfp is a front shoulder of a horizontal synchronization signal for video playback by the VR device to be processed, hbp is a rear shoulder of a horizontal synchronization signal for video playback by the VR device to be processed, height is the height of a display area for video playback by the VR device to be processed, and vsync is a frame synchronization signal for video playback by the VR device to be processed, vfp denotes the front shoulder of the frame synchronization signal for the VR device to be processed to play video, vbp denotes the rear shoulder of the frame synchronization signal for the VR device to be processed to play video, and bus _ width denotes the bus width for the VR device to be processed to play video, which generally means how much data amount is contained in each pixel, and is normally 24(═ 3 ═ 8, 3 denotes three primary colors RGB, and 8 denotes the number of bits per color).

Wherein, DSC compression parameters are as follows: the video stream compression standard is an industrial-grade video stream compression standard, can compress and decompress data at a very high speed, and transmits the data to a display screen without causing obvious loss of video quality. Typical DSC compression parameters are 1/3 or 1/2.

Here, the adjustable range corresponding to the adjustable parameter may be determined according to a historical adjustment range of the adjustable parameter, or according to an adjustment range preset by a user, and the like, and may be specifically determined according to an actual situation.

S404: and sending a VR instruction to the VR equipment to be processed so that the VR equipment to be processed performs VR operation according to the VR instruction.

Here, the processing device calculates and adapts the frame rate most suitable for the user side VR device end by obtaining the network delay corresponding to the current VR device, and compared with a fixed frame rate mode, the processing device is more intelligent, and brings the experience performance of the VR device into full play by fully utilizing a network environment.

According to the embodiment of the application, network delay of a VR current frame corresponding to VR equipment is obtained, then, total delay of the current frame is determined according to the network delay, a target VR frame rate is determined according to the total delay of the VR current frame, and display driving parameters of the VR equipment are adjusted according to the target VR frame rate, so that the VR frame rate of the VR equipment is equal to the target VR frame rate, and therefore a VR instruction is sent to the VR equipment, so that the VR equipment performs VR operation according to the VR instruction. Under the condition of low network delay, the frame rate can be properly improved and the user fluency can be improved by adopting the embodiment of the application. In addition, the embodiment of the application has good self-adaptability, does not need to intervene every time, and ensures the stability and fluency of VR experience of the user due to no perception of the user.

In addition, in the embodiment of the application, before the VR command is sent to the to-be-processed VR device, the processing device further considers that the network condition is extremely undesirable and needs to rely on a time warping algorithm to remedy the frame loss situation of the VR side. That is, the processing device further considers whether the target VR frame rate is greater than a preset frame rate threshold, and if the target VR frame rate is less than or equal to the preset frame rate threshold, sends a time warp instruction and a VR instruction to the to-be-processed VR device, so that the to-be-processed VR device performs VR operation according to the time warp instruction and the VR instruction. Fig. 5 is a schematic flowchart of another virtual reality optimization method according to an embodiment of the present application. As shown in fig. 5, the method includes:

s501: and acquiring the network delay of the VR current frame corresponding to the VR equipment to be processed, and determining the total delay of the VR current frame according to the network delay.

S502: and determining the target VR frame rate according to the total time delay of the VR current frame.

S503: and adjusting the display driving parameters of the VR equipment to be processed according to the target VR frame rate, so that the VR frame rate of the VR equipment to be processed is equal to the target VR frame rate.

The steps S501 to S503 refer to the related descriptions of the steps S401 to S403, and are not described herein again.

S504: and judging whether the target VR frame rate is greater than a preset frame rate threshold value.

The preset frame rate threshold may be determined according to an actual situation, for example, when a network condition is extremely undesirable, a VR frame rate determined according to a total VR current frame delay corresponding to the VR device is determined.

S505: and if the target VR frame rate is less than or equal to the preset frame rate threshold, sending a time warping instruction and a VR instruction to the VR equipment to be processed, so that the VR equipment to be processed performs VR operation according to the time warping instruction and the VR instruction.

Here, if the target VR frame rate is less than or equal to the preset frame rate threshold, it indicates that the network condition is extremely undesirable at this time, and it needs to rely on a time warp algorithm to remedy the VR side frame loss situation, so that the processing apparatus sends a time warp instruction and a VR instruction to the VR device to be processed, so that the VR device to be processed performs VR operation according to the time warp instruction and the VR instruction.

If the target VR frame rate is greater than the preset frame rate threshold, it is indicated that the network condition is ideal at this time, and the VR side frame loss condition is remedied without depending on a time warping algorithm, so that the processing device sends a VR instruction to the VR equipment to be processed, so that the VR equipment to be processed performs VR operation according to the VR instruction.

In the embodiment of the application, the processing device further considers that the network condition is extremely unsatisfactory, a time warping algorithm is required to be relied on, the condition that the VR side loses frames is remedied, the user experience is improved, and the processing device is suitable for application. Moreover, the processing device can dynamically adjust the VR frame rate of the VR equipment according to the network delay corresponding to the VR equipment, and can avoid the problems of frame skipping and frame loss caused by non-ideal network or larger delay under the condition of fixing the VR frame rate. And under the condition of low network delay, the processing device can properly improve the frame rate and improve the fluency of users. In addition, the embodiment of the application has good self-adaptability, does not need to intervene every time, and ensures the stability and fluency of VR experience of the user due to no perception of the user.

Fig. 6 is a schematic structural diagram of a virtual reality optimizing apparatus provided in the embodiment of the present application, corresponding to the virtual reality optimizing method in the foregoing embodiment. For convenience of explanation, only portions related to the embodiments of the present application are shown. Fig. 6 is a schematic structural diagram of a virtual reality optimization apparatus according to an embodiment of the present application, where the virtual reality optimization apparatus 60 includes: a delay determination module 601, a frame rate determination module 602, a parameter adjustment module 603, and a VR module 604. The virtual reality optimization apparatus may be the processing apparatus itself, or a chip or an integrated circuit that implements the functions of the processing apparatus. It should be noted here that the division of the delay determination module, the frame rate determination module, the parameter adjustment module, and the VR module is only a division of logical functions, and the two may be integrated or independent physically.

The delay determining module 601 is configured to obtain a network delay of a VR current frame corresponding to the VR device to be processed, and determine a total delay of the VR current frame according to the network delay.

A frame rate determining module 602, configured to determine a target VR frame rate according to the VR current frame total delay.

A parameter adjusting module 603, configured to adjust, according to the target VR frame rate, a display driving parameter of the VR device to be processed, so that the VR frame rate of the VR device to be processed is equal to the target VR frame rate.

And a VR module 604, configured to send a VR instruction to the to-be-processed VR device, so that the to-be-processed VR device performs VR operation according to the VR instruction.

In one possible implementation manner, the network delay includes a first time when the to-be-processed VR device uploads the acquired head pose of the user to a server and a second time when the server transmits the encoded picture to the to-be-processed VR device.

The delay determining module 601 is specifically configured to:

In a possible implementation manner, the parameter adjusting module 603 is specifically configured to:

according to the expression:

In a possible implementation manner, the frame rate determining module 602 is specifically configured to:

according to the expression:

The apparatus provided in the embodiment of the present application may be configured to implement the technical solution of the method embodiment shown in fig. 4, which has similar implementation principles and technical effects, and is not described herein again in the embodiment of the present application.

Fig. 7 is a schematic structural diagram of another virtual reality optimization apparatus according to an embodiment of the present application, and based on the embodiment shown in fig. 6, the virtual reality optimization apparatus 60 further includes: a frame rate determining module 605.

The frame rate determining module 605 is configured to determine whether the target VR frame rate is greater than a preset frame rate threshold before the VR module 604 sends the VR instruction to the VR device to be processed.

The VR module 604, further configured to:

The apparatus provided in the embodiment of the present application may be configured to implement the technical solution of the method embodiment shown in fig. 5, which has similar implementation principles and technical effects, and is not described herein again in the embodiment of the present application.

Optionally, fig. 8A and 8B schematically provide a schematic diagram of a possible basic hardware architecture of the virtual reality optimizing apparatus according to the present application.

Referring to fig. 8A and 8B, a virtual reality optimization apparatus 800 includes at least one processor 801 and a communication interface 803. Further optionally, a memory 802 and a bus 804 may also be included.

Among them, in the virtual reality optimizing apparatus 800, the number of the processors 801 may be one or more, and fig. 8A and 8B illustrate only one of the processors 801. Alternatively, the processor 801 may be a Central Processing Unit (CPU), a GPU, or a Digital Signal Processor (DSP). If the virtual reality optimizing device 800 has multiple processors 801, the types of the multiple processors 801 may be different, or may be the same. Optionally, the plurality of processors 801 of the virtual reality optimization device 800 may also be integrated into a multi-core processor.

Memory 802 stores computer instructions and data; the memory 802 may store computer instructions and data necessary to implement the virtual reality optimization methods provided herein, e.g., the memory 802 stores instructions for implementing the steps of the virtual reality optimization methods described above. The memory 802 may be any one or any combination of the following storage media: nonvolatile memory (e.g., Read Only Memory (ROM), Solid State Disk (SSD), hard disk (HDD), optical disk), volatile memory.

The communication interface 803 may provide information input/output for the at least one processor. Any one or any combination of the following devices may also be included: a network interface (e.g., an ethernet interface), a wireless network card, etc. having a network access function.

Optionally, the communication interface 803 may also be used for the virtual reality optimizing device 800 to perform data communication with other computing devices or terminals.

Further alternatively, fig. 8A and 8B show the bus 804 by a thick line. A bus 804 may connect the processor 801 with the memory 802 and the communication interface 803. Thus, via bus 804, processor 801 may access memory 802 and may also interact with other computing devices or terminals using communication interface 803.

In the present application, the virtual reality optimizing apparatus 800 executes computer instructions in the memory 802, so that the virtual reality optimizing apparatus 800 implements the virtual reality optimizing method provided in the present application, or the virtual reality optimizing apparatus 800 deploys the virtual reality optimizing device.

From the viewpoint of logical functional division, as shown in fig. 8A, the memory 802 may include a delay time determination module 601, a frame rate determination module 602, a parameter adjustment module 603, and a VR module 604. The inclusion herein merely refers to that the instructions stored in the memory, when executed, may implement the functions of the delay determining module, the frame rate determining module, the parameter adjusting module, and the VR module, respectively, without limitation to physical structures.

Illustratively, as shown in fig. 8B, the memory 802 may include a frame rate determining module 605. The inclusion herein merely refers to that the instructions stored in the memory can implement the functions of the frame rate judging module when executed, and is not limited to the physical structure.

In addition, the virtual reality optimizing apparatus may be implemented by software as shown in fig. 8A and 8B, or may be implemented by hardware as a hardware module or as a circuit unit.

A computer-readable storage medium is provided, the computer program product comprising computer instructions that instruct a computing device to perform the above-mentioned virtual reality optimization method provided herein.

The present application provides a computer program product comprising computer instructions for executing the above virtual reality optimization method provided herein by a processor.

The present application provides a chip comprising at least one processor and a communication interface providing information input and/or output for the at least one processor. Further, the chip may also include at least one memory for storing computer instructions. The at least one processor is used for calling and executing the computer instructions to execute the virtual reality optimization method provided by the application.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Claims

1. A virtual reality optimization method, comprising:

acquiring network delay of a virtual reality current frame corresponding to virtual reality equipment to be processed, and determining total delay of the virtual reality current frame according to the network delay;

determining a target virtual reality frame rate according to the virtual reality current frame total delay;

adjusting display driving parameters of the virtual reality equipment to be processed according to the target virtual reality frame rate so that the virtual reality frame rate of the virtual reality equipment to be processed is equal to the target virtual reality frame rate;

and sending a virtual reality instruction to the to-be-processed virtual reality equipment so as to enable the to-be-processed virtual reality equipment to perform virtual reality operation according to the virtual reality instruction.

2. The method according to claim 1, wherein the network latency includes a first time when the virtual reality device to be processed uploads the acquired head pose of the user to a server and a second time when the server transmits the encoded picture to the virtual reality device to be processed;

the determining of the total delay of the current frame of the virtual reality according to the network delay comprises:

acquiring third time when the virtual reality equipment to be processed collects the head gesture of the user, fourth time when the server processes the head gesture of the user, fifth time when the server renders pictures according to a processing result, sixth time when the server encodes rendered pictures and seventh time when the virtual reality equipment to be processed decodes the encoded pictures;

and taking the sum of the first time, the second time, the third time, the fourth time, the fifth time, the sixth time and the seventh time as the total virtual reality current frame delay.

3. The method according to claim 1, before the sending virtual reality instructions to the virtual reality device to be processed, further comprising:

judging whether the target virtual reality frame rate is greater than a preset frame rate threshold value or not;

the sending of the virtual reality instruction to the virtual reality device to be processed includes:

if the target virtual reality frame rate is smaller than or equal to the preset frame rate threshold value, sending a time warping instruction and the virtual reality instruction to the to-be-processed virtual reality equipment, so that the to-be-processed virtual reality equipment performs virtual reality operation according to the time warping instruction and the virtual reality instruction.

4. The method according to any one of claims 1 to 3, wherein the adjusting display driving parameters of the virtual reality device to be processed according to the target virtual reality frame rate comprises:

determining adjustable parameters in display driving parameters of the virtual reality equipment to be processed;

and adjusting the adjustable parameters according to the target virtual reality frame rate and the adjustable range corresponding to the adjustable parameters, so that the virtual reality frame rate of the virtual reality equipment to be processed is equal to the target virtual reality frame rate.

5. The method according to claim 4, wherein the adjusting the adjustable parameter according to the target virtual reality frame rate and the adjustable range corresponding to the adjustable parameter includes:

according to the expression:

and the adjustable range corresponding to the adjustable parameter, and adjusting the adjustable parameter, wherein the adjustable parameter includes mipspeed, hsync, hfp, hbp, vsync, vfp, and vbp, F is the target virtual reality frame rate, mipspeed is the transmission rate of each data channel for video playing of the virtual reality device to be processed, lane _ num is the number of data channels for video playing of the virtual reality device to be processed, width is the width of a display area for video playing of the virtual reality device to be processed, hsync is a horizontal synchronization signal for video playing of the virtual reality device to be processed, hfp is a front shoulder of the horizontal synchronization signal for video playing of the virtual reality device to be processed, hbp is a rear shoulder of the horizontal synchronization signal for video playing of the virtual reality device to be processed, and height is the height of the display area for video playing of the virtual reality device to be processed, vsync is a frame synchronization signal of the virtual reality equipment to be processed for video playing, vfp is a front shoulder of the frame synchronization signal of the virtual reality equipment to be processed for video playing, vbp is a rear shoulder of the frame synchronization signal of the virtual reality equipment to be processed for video playing, and bus _ width is a bus width of the virtual reality equipment to be processed for video playing.

6. The method according to any one of claims 1 to 3, wherein the determining a target virtual reality frame rate according to the virtual reality current frame total delay comprises:

according to the expression:

determining the target virtual reality frame rate F, wherein T (total) is the total delay of the current frame of the virtual reality.

7. A virtual reality optimization apparatus, comprising:

the delay determining module is used for acquiring the network delay of the virtual reality current frame corresponding to the virtual reality equipment to be processed and determining the total delay of the virtual reality current frame according to the network delay;

the frame rate determining module is used for determining a target virtual reality frame rate according to the total delay of the current virtual reality frame;

a parameter adjusting module, configured to adjust a display driving parameter of the to-be-processed virtual reality device according to the target virtual reality frame rate, so that the virtual reality frame rate of the to-be-processed virtual reality device is equal to the target virtual reality frame rate;

and the virtual reality module is used for sending a virtual reality instruction to the to-be-processed virtual reality equipment so as to enable the to-be-processed virtual reality equipment to perform virtual reality operation according to the virtual reality instruction.

8. A virtual reality optimization apparatus, comprising:

a processor;

a memory; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of any of claims 1-6.

9. A computer-readable storage medium, characterized in that it stores a computer program that causes a server to execute the method of any one of claims 1-6.

10. A computer program product comprising computer instructions for executing the method of any one of claims 1-6 by a processor.