CN111683077A

CN111683077A - Virtual reality equipment and data processing method

Info

Publication number: CN111683077A
Application number: CN202010490926.9A
Authority: CN
Inventors: 李旭东; 张箭
Original assignee: Silicon Valley Digital Analog Suzhou Semiconductor Co ltd; Analogix International LLC
Current assignee: Silicon Valley Analog Suzhou Semiconductor Co ltd; Analogix International LLC
Priority date: 2020-06-02
Filing date: 2020-06-02
Publication date: 2020-09-18
Anticipated expiration: 2040-06-02
Also published as: CN111683077B

Abstract

The application discloses a virtual reality device and a data processing method. Wherein, this virtual reality equipment includes: the system comprises a processor, a communication module and a display device, wherein the communication module is in communication connection with the processor and a cloud server and is used for realizing communication between the processor and the cloud server; the processor is used for acquiring data to be processed from the cloud server and converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of the display device; the display device is in communication with the processor for displaying the target image data. The application solves the technical problems that the existing AR/VR equipment structure has many defects, for example, the cost is high, and the resolution ratio of a glasses screen is difficult to meet the market demand.

Description

Virtual reality equipment and data processing method

Technical Field

The application relates to the field of architecture design of virtual reality equipment, in particular to virtual reality equipment and a data processing method.

Background

Virtual Reality (VR) technology is a computer simulation system that creates and experiences a Virtual world, using a computer to create a simulated environment into which a user is immersed. Augmented Reality (AR) technology is a new technology that integrates real world information and virtual world information "seamlessly".

Fig. 1 is an architecture diagram of most AR/VR devices currently on the market, and as shown in fig. 1, functions of parts of the architecture are described as follows:

CPU/GPU: and the main control chip is responsible for voice and image processing and human-computer interaction data operation and sends a processing result to the lower-level signal converter.

A memory: and storing scene contents such as games and the like, and simultaneously storing temporary data in the CPU/GPU processing process.

Glasses screen of MIPI (Mobile Industry Processor, MIPI) interface: video data content is received and displayed. MIPI, a mobile industry processor interface, is an open standard and a specification established by the MIPI alliance for mobile application processors.

The signal conversion chip: the content data processed by the CPU/GPU is usually transmitted in two data forms of HDMI (High definition multimedia Interface) and Displayport, and the signal conversion chip converts them into MIPI form and transmits them to the glasses screen of the terminal. HDMI is a fully digital video and audio transmission interface, and Displayport is a standardized digital video interface standard.

The existing framework mainly has the following defects:

AR/VR applications require powerful computing power, and therefore require very advanced CPU/GPU and large memory units, which is costly.

2. At present, AR/VR equipment carries a glasses screen with a general MIPI interface, an MIPI interface TED chip (the TED chip is a chip integrating a time sequence control function and a driving function) is actually embedded in the screen, the screen technology is similar to that of a mobile phone screen, the TED chip usually comprises a partial image processing function to meet the requirement of high resolution of glasses, but the requirements of chip power consumption and size are limited, and the partial function can not completely meet the requirement of a user. For example, the frame rate of a mobile phone screen for one second is 60Hz, the mobile phone screen can work at 90Hz during game playing, the extremely special case that the mobile phone screen works at about 130Hz for a short time is a limit, and AR/VR glasses work at 144Hz is a basic condition.

3. Since the screen actually uses the MIPI interface, a signal conversion chip must be collocated, as described above, and such a signal conversion chip usually also designs an image processing function, and the two functions are overlapped.

4. Larger resolutions are superimposed with higher frame rates as AR/VR products are developing, and the screen of the MIPI interface will be increasingly difficult to meet the needs of users and markets.

In view of the above problems of the existing AR/VR device architecture, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the application provides virtual reality equipment and a data processing method, and aims to at least solve the technical problems that the existing AR/VR equipment architecture has many defects, for example, the cost is high, the resolution of a glasses screen is difficult to meet the market demand, and the like.

According to an aspect of an embodiment of the present application, there is provided a virtual reality device, including: the system comprises a processor, a communication module and a display device, wherein the communication module is in communication connection with the processor and a cloud server and is used for realizing communication between the processor and the cloud server; the processor is used for acquiring data to be processed from the cloud server and converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of the display device; the display device is in communication with the processor for displaying the target image data.

Optionally, the communication module is a 5G communication module.

Optionally, the processor is a RISC-V architecture integrated processor, and is configured to process original image data in the data to be processed, so as to obtain target image data capable of adapting to the resolution of the display device.

Optionally, the processor is further configured to divide a display screen of the display device into a plurality of display areas according to the target image data, and dynamically adjust backlight brightness of the plurality of display areas according to a display position of the target image data on the display screen.

Optionally, the target data further includes target audio data, and the processor is further configured to process original audio data in the data to be processed to obtain the target audio data.

Optionally, the virtual reality device further includes a voice playing device, configured to play the target audio data.

Optionally, the target data further includes target human-computer interaction data, the processor is further configured to process original human-computer interaction data in the data to be processed to obtain target human-computer interaction data, and the target human-computer interaction data is used to control a virtual target in the target image data.

Optionally, the display device is integrated with a display driving chip, and the display driving chip is configured to drive the display device to display the target image data.

According to another aspect of the embodiments of the present application, there is also provided a data processing method, which is applied to the virtual reality device described above, and includes: acquiring to-be-processed data stored on a cloud server; converting the data to be processed into target data, the target data including at least target image data capable of adapting to a resolution of the display device; the target image data is transmitted to the display device.

Optionally, after the target image data is sent to the display device, the method further includes: dividing a display screen of a display device into a plurality of display areas according to target image data; and dynamically adjusting the backlight brightness of the plurality of display areas according to the display position of the target image data on the display screen.

In an embodiment of the present application, there is provided a virtual display device, including: the system comprises a processor, a communication module and a display device, wherein the communication module is in communication connection with the processor and a cloud server and is used for realizing communication between the processor and the cloud server; the processor is used for acquiring data to be processed from the cloud server and converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of the display device; the display device is in communication connection with the processor and is used for displaying target image data, so that the system design of the AR/VR equipment is greatly simplified, and the hardware cost and the power consumption are reduced; the technical effect of improving the resolution of the glasses screen is achieved, and therefore the technical problems that the existing AR/VR equipment structure has many defects, for example, the cost is high, the resolution of the glasses screen cannot meet the market demand easily, and the like are solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is an architectural diagram of an AR/VR device in accordance with embodiments of the present application;

FIG. 2 is a block diagram of a virtual reality device according to an embodiment of the present application;

fig. 3 is a flowchart of a data processing method according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Referring to fig. 1, a schematic diagram of an existing architecture of an AR/VR device is shown, where the existing architecture has many disadvantages, such as high cost, difficulty in meeting market requirements for the resolution of the glasses screen, and the like. Aiming at the defects of the existing AR/VR equipment structure, the application provides a new AR/VR equipment structure.

Fig. 2 is a structural diagram of a virtual reality device to which the above new AR/VR device architecture is applied according to an embodiment of the present application, and as shown in fig. 2, the virtual reality device includes: a processor 20, a communication module 22, and a display device 24, wherein,

the communication module 22 is communicatively connected to the processor 20 and the cloud server, and is configured to implement communication between the processor 20 and the cloud server.

According to an alternative embodiment of the present application, the communication module 22 is a 5G communication module.

In the AR/VR device of this embodiment, all data are processed at the cloud server, and the 5G module is responsible for being used for communicating with the cloud server in real time, acquiring data from the cloud server or returning the processed data to the cloud server.

It should be noted that the communication module 22 may also be a 4G module or other wireless communication modules.

The processor 20 is configured to obtain data to be processed from the cloud server, and convert the data to be processed into target data, where the target data includes at least target image data capable of adapting to a resolution of the display device 24.

The processor 20 is an RISC-V architecture integrated processor for processing the original image data in the data to be processed to obtain the target image data capable of adapting to the resolution of the display device 24.

The processor in the embodiment of the application is a processor integrated with RISC-V architecture, has strong image processing function, and is a control chip most meeting the requirements of AR/VR products. For example, the dynamic frame rate switching function can be directly switched from 1Hz to 240Hz without extra transition time, corresponding to a fast game scene; High-Dynamic Range (HDR) functionality may provide users with a wider color gamut and detailed experience; the backlight partitioning technique can partition according to display content, dynamically adjust the backlight brightness of the glasses to save power consumption while enhancing experience, and the like.

The processor 20 can realize functions of data processing, data enhancement, display screen control and the like, replaces the functions realized by three chips, namely a CPU/GPU + signal conversion chip + MIPI TED in the existing architecture, greatly simplifies the system design, reduces the hardware cost and the power consumption, and can promote the portable design of AR/VR products.

A display device 24 is communicatively coupled to the processor 20 for displaying the target image data.

According to an alternative embodiment of the present application, the display device 24 is integrated with a display driving chip for driving the display device to display the target image data.

In the structure provided by the embodiment of the present application, because the processor 20 can directly process and obtain image data capable of adapting to the resolution of the glasses screen, and all image processing functions can be completed by using the processor integrated with the RISC-V structure, the glasses screen does not need to use a TED chip based on an MIPI interface, but can be changed to a display driving chip only using a P2P interface, thereby reducing the system cost and power consumption.

By the aid of the architecture design, system design of AR/VR equipment is greatly simplified, and hardware cost and power consumption are reduced; and a technical effect of improving the resolution of the glasses screen.

In an alternative embodiment of the present application, the processor 20 is further configured to divide the display screen of the display device 24 into a plurality of display areas according to the target image data, and dynamically adjust the backlight brightness of the plurality of display areas according to the display position of the target image data on the display screen. The processor 20 may implement a backlight partitioning technique that may partition, dynamically adjust the backlight brightness of the glasses according to the display content to save power while enhancing the experience.

In some optional embodiments of the present application, the target data further includes target audio data, and the processor 20 is further configured to process original audio data in the data to be processed to obtain the target audio data. The processor 20 may process audio data in addition to having an image processing function.

According to an optional embodiment of the present application, the virtual reality device further includes a voice playing device, configured to play the target audio data.

In another optional embodiment of the present application, the target data further includes target human-computer interaction data, the processor 20 is further configured to process original human-computer interaction data in the data to be processed to obtain the target human-computer interaction data, and the target human-computer interaction data is used to control a virtual target in the target image data.

In a game scene, a user sends a control instruction for controlling a virtual target in the game scene through a human-computer interaction device, a processor obtains the control instruction sent by the user (namely original human-computer interaction data), processes the control instruction to obtain target human-computer interaction data which can be identified by a computer, and the target human-computer interaction data is utilized to realize the control of the virtual target in the game.

Fig. 3 is a flowchart of a data processing method according to an embodiment of the present application, and as shown in fig. 3, the method is applied to the virtual reality device, and includes the following steps:

step S302, acquiring to-be-processed data stored on a cloud server;

step S304, converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of the display device;

step S306, the target image data is transmitted to the display device.

In some optional embodiments of the present application, after the step S306 is completed, the display screen of the display device may be further divided into a plurality of display areas according to the target image data; and dynamically adjusting the backlight brightness of the plurality of display areas according to the display position of the target image data on the display screen.

It should be noted that, reference may be made to the description related to the embodiment shown in fig. 2 for a preferred implementation of the embodiment shown in fig. 3, and details are not described here again.

The embodiment of the application also provides a storage medium, wherein the storage medium is used for storing the program, and the program controls the equipment where the storage medium is located to execute the processing method of the data when running.

The storage medium stores a program for executing the following functions: acquiring to-be-processed data stored on a cloud server; converting the data to be processed into target data, the target data including at least target image data capable of adapting to a resolution of the display device; the target image data is transmitted to the display device.

The embodiment of the application also provides a processor, wherein the processor is used for running the program, and the processing method for executing the data is executed when the program runs.

The processor is used for running a program for executing the following functions: acquiring to-be-processed data stored on a cloud server; converting the data to be processed into target data, the target data including at least target image data capable of adapting to a resolution of the display device; the target image data is transmitted to the display device.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a read-Only Memory (ROM), a random access Memory (RBJGGM, RBJGGndom BJGGccess Memory), a removable hard disk, a magnetic disk or an optical disk, and various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1.A virtual reality device, comprising: a processor, a communication module, and a display device, wherein,

the communication module is in communication connection with the processor and the cloud server and is used for realizing communication between the processor and the cloud server;

the processor is used for acquiring data to be processed from the cloud server and converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of the display device;

the display device is in communication connection with the processor and is used for displaying the target image data.

2. The device of claim 1, wherein the communication module is a 5G communication module.

3. The apparatus of claim 1, wherein the processor is an RISC-V architecture integrated processor configured to process original image data in the data to be processed to obtain target image data capable of adapting to a resolution of the display device.

4. The apparatus of claim 3, wherein the processor is further configured to divide a display screen of the display device into a plurality of display areas according to the target image data, and dynamically adjust backlight brightness of the plurality of display areas according to a display position of the target image data on the display screen.

5. The apparatus of claim 3, wherein the target data further comprises target audio data, and the processor is further configured to process raw audio data in the data to be processed to obtain the target audio data.

6. The apparatus according to claim 5, wherein the virtual reality apparatus further comprises a voice playing device for playing the target audio data.

7. The device according to claim 3, wherein the target data further includes target human-computer interaction data, and the processor is further configured to process raw human-computer interaction data in the data to be processed to obtain the target human-computer interaction data, and the target human-computer interaction data is used to control a virtual target in the target image data.

8. The apparatus of claim 1, wherein the display device is integrated with a display driver chip, and the display driver chip is configured to drive the display device to display the target image data.

9. A data processing method applied to the virtual reality device of any one of claims 1 to 8, comprising:

acquiring to-be-processed data stored on a cloud server;

converting the data to be processed into target data, wherein the target data at least comprises target image data capable of adapting to the resolution of a display device;

and sending the target image data to the display device.

10. The method of claim 9, wherein after sending the target image data to the display device, the method further comprises:

dividing a display screen of the display device into a plurality of display areas according to the target image data;

and dynamically adjusting the backlight brightness of the plurality of display areas according to the display position of the target image data on the display screen.