WO2017193576A1

WO2017193576A1 - Video resolution adaptation method and apparatus, and virtual reality terminal

Info

Publication number: WO2017193576A1
Application number: PCT/CN2016/109555
Authority: WO
Inventors: 许小飞
Original assignee: 乐视控股（北京）有限公司; 乐视致新电子科技（天津）有限公司
Priority date: 2016-05-13
Filing date: 2016-12-13
Publication date: 2017-11-16
Also published as: CN105933726A

Abstract

Embodiments of the present invention provide a video resolution adaptation method and apparatus, and a virtual reality terminal. The method comprises: obtaining the size of an effective display screen of a display in a virtual reality terminal and obtaining a target resolution of video playing; and scaling the playing size of a video to be played according to the target resolution of video playing and the size of the effective display screen of the display of the virtual reality terminal, so that the target resolution of video playing is adapted to the size of the effective display screen. The embodiments of the present invention can implement adaptive presentation on the screens of terminal devices of various sizes according to the video resolution requirement of a user, so that the user obtains a comfortable and smooth viewing effect.

Description

Method, device and virtual reality terminal for adapting video resolution

The present application claims the priority of the Chinese patent application with the application number CN201610319088.2 and the application date being May 13, 2016, and the invention name is "adaptive method and device for the virtual reality terminal and its video resolution", the Chinese patent application The entire contents of this application are incorporated herein by reference.

[Technical Field]

The embodiments of the present invention belong to the field of image processing, and in particular, to a method and device for adapting video resolution and a virtual reality terminal.

【Background technique】

Virtual Reality (VR) is a virtual world that uses computers to simulate three-dimensional space, allowing users to experience interactive and immersive experiences through vision and hearing. Virtual reality technology is computer graphics. A comprehensive technology that intersects in the fields of human-computer interaction and sensing technology. With the rapid development of computer hardware and network environment, virtual reality technology has developed rapidly and many mature virtual reality systems have emerged. They have wide applications in virtual simulation, geographic information system, user social, game entertainment and so on. .

The visual system is the most important perceptual channel of VR. At the same time, the perceptibility of virtual scene is also a prerequisite for human-computer interaction in the virtual environment. Therefore, the scene display mode and its equipment are the basic components of human-computer interaction in VR system. At present, the commonly used scene display modes are helmet type, desktop type, projection type, hand-held type and free stereoscopic display mode, and each display mode includes multiple types of screen sizes.

In order to present an immersive experience to the user in the virtual reality terminal, the video content needs to be presented in high resolution, which makes the video file large, usually more than 1 GB capacity, and if it is rendered after downloading, the loading on the terminal device is too slow, and it is expensive and complicated. The terminal device, which is the first factor that affects the experience, so consider streaming to the end user via network streaming.

The video is compressed and encoded and stored on the server. It has multiple compression formats and resolution differences, such as SD, HD, and Ultra Clear. In order to be able to present the video streaming media comfortably and smoothly to the user, Video needs to be played in a variety of screen sizes and multiple resolutions. Therefore, there is a need for a method and apparatus for adapting video resolution in a virtual reality terminal.

[Summary of the Invention]

In view of this, the embodiments of the present invention provide a method, a device, and a virtual reality terminal for adapting video resolution, so that the user can perform virtual reality video playback online in a comfortable and smooth manner.

An embodiment of the present invention provides a method for adapting video resolution in a virtual reality terminal, which is applied to an electronic terminal, and includes:

Obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing;

The playback size of the video to be played is scaled to adapt the target resolution of the video playback to the effective display screen size based on the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal.

Further, in an embodiment of the invention, the effective display screen size of the display in the virtual reality terminal is represented by a long and wide pixel value of the effective display screen of the display in the virtual reality terminal.

Further, in an embodiment of the invention, obtaining the target resolution of the video playback comprises: selecting a video resolution from the candidate plurality of video resolutions as the target resolution according to the transmission speed and the available bandwidth.

Further, in an embodiment of the present invention, obtaining the target resolution of the video playback comprises: capturing an operation event of the user, and selecting a fixed video resolution from the plurality of video resolutions as the target resolution according to the captured operation event. rate.

Further, in an embodiment of the present invention, according to the target resolution of the video playing and the effective display screen size of the display in the virtual reality terminal, the playing size of the video to be played is scaled to distinguish the target of the video playing. The rate adaptation to the effective display screen size includes:

Calculating a playback size ratio of the video and a playback size of the scaled video according to the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal The effective display screen size of the one-side filling amount in the longitudinal direction or the one-side filling amount in the width direction;

And scaling the play size of the video to be played according to the calculated zoom ratio, and performing double-side filling or width direction in the length direction of the area on the effective display screen where no video content is played according to the calculated one-side fill amount. Double side fill.

An embodiment of the present invention provides an apparatus for adapting video resolution in a virtual reality terminal, including:

Obtaining a module, obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing;

a video scaling module, configured to scale a play size of the video to be played according to a target resolution of the video play and an effective display screen size of the display in the virtual reality terminal, so that the target resolution of the video play is adapted to the effective display screen size.

Further, in an embodiment of the invention, the acquiring module is further configured to adaptively select a video resolution as the target resolution from the candidate multiple video resolutions according to the transmission speed and the available bandwidth.

Further, in an embodiment of the invention, the obtaining module is further configured to capture an operation event of the user, and select a fixed video resolution as the target resolution from the plurality of video resolutions according to the captured operation event.

Further, in an embodiment of the present invention, the video scaling module is further configured to calculate a scaling ratio of a play size of the video according to a target resolution of the video playing and an effective display screen size of the display in the virtual reality terminal. a playback size of the scaled video relative to the effective display screen size in a one-side fill amount in the length direction or a one-side fill amount in the width direction; and scaling the play size of the video to be played according to the calculated zoom ratio And performing, according to the calculated one-side filling amount, a double-side filling in the longitudinal direction or a double-side filling in the width direction on the area where the video content is not played on the effective display screen.

Embodiments of the present invention provide a virtual reality terminal, which includes a display, a processor, and a processor. Obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing, and playing the video to be played according to the target resolution of the video playing and the effective display screen size of the display in the virtual reality terminal The size is scaled to adapt the target resolution of the video playback to the effective display screen size.

The embodiment of the present invention provides a non-transitory computer readable storage medium, where the non-transitory computer readable storage medium stores program instructions for executing, when the terminal is the program instruction,

An embodiment of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by the terminal, The terminal performs the above method.

In the embodiment of the present invention, by obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing; and according to the target resolution of the video playing and the effective display screen size of the display in the virtual reality terminal, The video to be played is scaled to adapt the target resolution of the video playback to the effective display screen size, thereby being adaptively presented on the screens of various sizes of virtual reality terminal devices according to the user's requirements for video resolution, so that the user is comfortable. Smooth viewing.

[Description of the Drawings]

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a few embodiments described in the present invention, and other drawings can be obtained from those skilled in the art from the drawings.

1 is a schematic flowchart of a method for adapting video resolution in a virtual reality terminal according to an embodiment of the present invention;

2 is a schematic flow chart of selecting one-side filling according to Embodiment 2 of the present invention;

FIG. 3 is a schematic flowchart of adaptively selecting a video resolution as a target resolution from an alternative multiple video resolution according to Embodiment 3 of the present invention; FIG.

4 is a schematic flowchart of performing video adjustment according to Embodiment 4 of the present invention;

5 is a schematic structural diagram of a video multi-resolution adaptive device in a virtual reality terminal according to Embodiment 5 of the present invention;

FIG. 6 is a schematic structural diagram of a virtual reality terminal according to Embodiment 6 of the present invention.

【detailed description】

Obtaining the effective display screen size of the display in the virtual reality terminal and obtaining the target resolution of the video playback; and scaling the video to be played according to the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal In order to adapt the target resolution of the video playback to the effective display screen size, the user can adaptively present on the screen of the virtual reality terminal device of various sizes according to the user's requirement for the video resolution, so that the user can obtain a comfortable and smooth viewing effect. Any technical solution of implementing the embodiments of the present invention necessarily does not necessarily need to achieve all the above advantages at the same time.

For a better understanding of the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention should fall within the scope of the present invention.

The specific implementation of the present invention will be further described below in conjunction with the drawings of the present invention.

The present invention provides a method and device for adapting video resolution in a virtual reality terminal, which is generally applied to a virtual reality video on demand network. If the resolution of the video matches the size of the effective display screen, the video is directly played; if not, Then, the technical solutions of the following embodiments are executed.

FIG. 1 is a schematic flowchart of a method for adapting video resolution in a virtual reality terminal according to an embodiment of the present invention, which may include:

Step S1: Obtain an effective display screen size of the display in the virtual reality terminal and obtain a target resolution of the video playback.

Specifically, the common resolution will vary depending on the ratio (aspect ratio) displayed on the display, as detailed below:

Table I

In addition, the resolution is also related to video clarity. The definitions are standard definition, ultra clear, high definition, broadcast TV and cinema movies. The corresponding resolution is shown in Table 2:

Table II

Step S2: according to the target resolution of the video playing and the effective display screen size of the display in the virtual reality terminal, the playing size of the video to be played is scaled to adapt the target resolution of the video playing to the effective display screen size.

In this embodiment, if the video target resolution does not match the effective display screen size, for example, the target resolution of the video is 1280×720, and the effective display screen size is small or large, so that the video exceeds the effective display screen during playback. The size, or the opposite, only occupies a small area on the effective display screen size, and the display is uncoordinated. Referring to FIG. 2 in step S2, the flow chart of selecting length filling or width filling according to the second embodiment of the present invention; As shown, it includes:

Step S201: Calculate a play size scaling ratio of the video and a play size of the scaled video relative to the effective display screen size according to the target resolution of the video play and the effective display screen size of the display in the virtual reality terminal. One-side filling amount of the direction or one-side filling amount in the width direction;

Step S202: scaling the play size of the video to be played according to the calculated zoom ratio, and performing double-side filling in the length direction on the area on the effective display screen where no video content is played according to the calculated one-side fill amount. Or double-sided filling in the width direction.

3 is a schematic flowchart of adaptively selecting a resolution as a target resolution from an alternative multiple video resolution according to Embodiment 3 of the present invention; in this embodiment, when obtaining a target resolution of video playback A video resolution is adaptively selected from the alternative multiple video resolutions as the target resolution based on the transmission speed and the available bandwidth.

Specifically, for a video file, a plurality of resolutions a/b/c may be corresponding, and a specific code rate that can be used is calculated according to the transmission speed and the available bandwidth, and one of the alternative video resolutions is selected according to the specific code rate value. Video target resolution, which increases or decreases the amount of data transferred. For example, if the network bandwidth is sufficient, the code rate value is large, and the video file with high resolution is used to provide more realistic virtual reality video; if the network bandwidth is decreased, the code rate value is small, and the video file with low resolution is used to reduce the data delay. Reduce discomfort such as dizziness caused by virtual reality video delay.

Alternatively, the target resolution may also be determined according to user selection, that is, the user's operation event is captured when the target resolution of the video playback is obtained in step S1, and a fixed one of the plurality of video resolutions is selected according to the captured operation event. The video resolution is used as the target resolution. For example, there are multiple video target resolution options on the display interface for the user to select, and details are not described in detail.

In this implementation, when the effective display screen size of the display in the virtual reality terminal is obtained in step S1, the effective display screen size is represented by the long and wide pixel values of the effective display screen of the display in the virtual reality terminal.

Set the display screen length pixel value to X, display screen width pixel value to Y, set the video length pixel value to Xv, video width pixel value to Yv, video resolution to Xv*Yv, and scale to scale The target resolution is Scale*Xv*Yv; the one-side filling amount in the length direction is set to offsetX, and the one-side filling amount offsetY in the width direction, step S2 may include steps S401-S404 in FIG. 4, see FIG. A schematic flowchart of performing video adjustment according to Embodiment 4 of the present invention; as shown in FIG. 4, the method includes:

Step S401: Calculate Sw=X Yv/Xv,

Step S402: determining a size relationship between Sw and Y;

If Sw is less than Y, step S403 is performed; if Sw is greater than Y, step S404 is performed; if Sw is equal to Y, the resolution of the video is considered to match the effective display screen size, and no scaling and filling are required;

Step S403: performing double-sided filling in the width direction on the area where the video content is not played on the effective display screen, and the current adjustment process ends;

When filling in step S403, first calculate the scaling scale=Y/Yv, and then calculate the one-side filling amount in the width direction offsetY=(Y-Yv scale)/2 according to the calculated scaling ratio, and there is no video content on the effective display screen. The width direction of the area to be played is filled with a black border having a width equal to offsetY=(Y-Yv scale)/2;

Step S404: Perform double-side filling in the length direction on the area where the video content is not played on the effective display screen, and the current adjustment process ends.

In step S404, when the filling is performed, the scaling scale=X/Xv may be first calculated, and the one-side filling amount offsetX=(X-Xv scale)/2 in the longitudinal direction is calculated according to the calculated scaling ratio, so that there is no video on the effective display screen. The length direction of the area in which the content is played is filled with a black border having a length equal to offsetX=(X-Xv scale)/2.

FIG. 5 is a schematic structural diagram of a video multi-resolution adaptive device in a virtual reality terminal according to a fifth embodiment of the present invention, which includes: an acquisition module 501 and a video scaling module 502 coupled to each other, wherein:

The obtaining module 501 is configured to obtain an effective display screen size of the display in the virtual reality terminal and obtain a target resolution of the video playing;

The video scaling module 502 is configured to scale the play size of the video to be played according to the target resolution of the video play and the effective display screen size of the display in the virtual reality terminal to adapt the target resolution of the video play to the effective Display screen size.

In this embodiment, or any other embodiment, the effective display screen size of the display in the virtual reality terminal is represented by a long and wide pixel value of the effective display screen of the display in the virtual reality terminal.

In this embodiment, or in any other embodiment, the acquiring module 501 is further configured to adaptively select a video resolution as the target resolution from the candidate multiple video resolutions according to the transmission speed and the available bandwidth.

In this embodiment, or any other embodiment, the obtaining module 501 is further configured to capture an operation event of the user, and select a fixed video resolution from the plurality of video resolutions as the target resolution according to the captured operation event. rate.

In this embodiment, or any other embodiment, the video scaling module is further used according to the Determining the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal, calculating the playback size scaling of the video and the playback size of the scaled video, and the one-side padding in the length direction relative to the effective display screen size a quantity or a one-side fill amount in the width direction; and scaling the play size of the video to be played according to the calculated zoom ratio, and playing no video content on the effective display screen according to the calculated one-side fill amount The area is double-sided in the length direction or double-sided in the width direction.

For detailed explanation of the further or specific functions of the obtaining module 501 and the video scaling module 502 in this embodiment, please refer to FIG. 1 to FIG. 4 above, and details are not described in detail.

6 is a schematic structural diagram of a virtual reality terminal according to a sixth embodiment of the present invention; as shown in FIG. 6, the display unit 601 includes a processor 602, and the processor 602 is configured to obtain an effective display screen size of the display in the virtual reality terminal and obtain a video. The target resolution of the playback, and the playback size of the video to be played is scaled according to the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal to adapt the target resolution of the video playback to the effective Display screen size.

Preferably, in any of the embodiments of the present invention, the effective display screen size of the display in the virtual reality terminal is characterized by a long and wide pixel value of an effective display screen of the display in the virtual reality terminal.

Preferably, in any embodiment of the present invention, the processor 602 is further configured to adaptively select a video resolution from the candidate multiple video resolutions as the target resolution according to the transmission speed and the available bandwidth.

Preferably, in any embodiment of the present invention, the processor 602 is further configured to capture an operation event of the user, and select a fixed video resolution from the plurality of video resolutions as the target resolution according to the captured operation event. rate.

Preferably, in any embodiment of the present invention, the processor 602 is further configured to calculate a scaling ratio of a play size of the video according to a target resolution of the video playing and an effective display screen size of the display in the virtual reality terminal. And a playback amount of the scaled video relative to the effective display screen size in a one-side fill amount in the length direction or a one-side fill amount in the width direction; and a playback scale of the video to be played according to the calculated zoom ratio Scaling, and one side based on calculation The filling amount is a double-side filling in the longitudinal direction or a double-side filling in the width direction on the area where the video content is not played on the effective display screen.

For the detailed explanation of the further or specific functions of the processor 602 in this embodiment, please refer to FIG. 1 to FIG. 4 above, and details are not described in detail.

In the above embodiment of the present invention, the virtual reality terminal may be a virtual reality helmet or a virtual reality glasses, and the virtual reality terminal may include one display or two displays.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, apparatus (device), or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The instruction device implements the functions specified in one or more blocks of the flowchart or in a flow or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the invention has been described, it will be understood that Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and It is apparent that those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and modifications of the invention

Claims

A method for adapting video resolution in a virtual reality terminal is applied to an electronic device, and is characterized in that:

Obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing;

The playback size of the video to be played is scaled to adapt the target resolution of the video playback to the effective display screen size based on the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal.
The method of claim 1 wherein the effective display screen size of the display in the virtual reality terminal is characterized by a long, wide pixel value of an effective display screen of the display in the virtual reality terminal.
The method of claim 1 wherein obtaining a target resolution for video playback comprises selecting a video resolution from the candidate plurality of video resolutions as the target resolution based on the transmission speed and the available bandwidth.
The method according to claim 1, wherein obtaining the target resolution of the video playback comprises: capturing an operation event of the user, and selecting a fixed video resolution from the plurality of video resolutions as the target according to the captured operation event. Resolution.
The method according to any one of claims 1 to 4, wherein the playback size of the video to be played is scaled according to the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal. The target resolution of the video playback is adapted to the effective display screen size including:

Calculating a zoom ratio of the play size of the video and a play size of the scaled video relative to the effective display screen size in the length direction according to the target resolution of the video play and the effective display screen size of the display in the virtual reality terminal Side fill amount or one-side fill amount in the width direction;

Scales the playback size of the video to be played based on the calculated scaling, and Calculating the obtained one-side filling amount, and performing double-side filling in the longitudinal direction or double-side filling in the width direction on the area on the effective display screen where no video content is played.
An apparatus for adapting video resolution in a virtual reality terminal, comprising:

Obtaining a module, obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing;

a video scaling module, configured to scale a play size of the video to be played according to a target resolution of the video play and an effective display screen size of the display in the virtual reality terminal, so that the target resolution of the video play is adapted to the effective display screen size.
The apparatus of claim 6 wherein the effective display screen size of the display in the virtual reality terminal is characterized by a long, wide pixel value of an effective display screen of the display in the virtual reality terminal.
The apparatus according to claim 6, wherein the obtaining module is further configured to adaptively select a video resolution from the candidate plurality of video resolutions as the target resolution according to the transmission speed and the available bandwidth.
The apparatus according to claim 6, wherein the obtaining module is further configured to capture an operation event of the user, and select a fixed video resolution from the plurality of video resolutions as the target resolution according to the captured operation event. .
The device according to any one of claims 6-9, wherein the video scaling module is further configured to calculate a video based on a target resolution of the video playback and an effective display screen size of the display in the virtual reality terminal. a playback size of the playback size and a playback size of the scaled video relative to the effective display screen size in a one-side fill amount in the length direction or a one-side fill amount in the width direction; and a video to be played according to the calculated zoom ratio The playback size is scaled, and according to the calculated one-side fill amount, the area on the effective display screen where no video content is played is subjected to double-side filling in the length direction or double-side filling in the width direction.
A virtual reality terminal, comprising: a display, a processor, the processor is configured to obtain an effective display screen size of the display in the virtual reality terminal, and obtain a target resolution of the video playing Rate, and according to the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal, the playback size of the video to be played is scaled to adapt the target resolution of the video playback to the effective display screen size.
A non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores program instructions for executing, when the terminal is in the program instruction:

Obtaining an effective display screen size of the display in the virtual reality terminal and obtaining a target resolution of the video playing;

The playback size of the video to be played is scaled to adapt the target resolution of the video playback to the effective display screen size based on the target resolution of the video playback and the effective display screen size of the display in the virtual reality terminal.
A computer program product, comprising: a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by the terminal, The terminal performs the method of any of claims 1-5.