WO2019080913A1 - 视频处理方法、计算机设备和存储介质 - Google Patents

视频处理方法、计算机设备和存储介质

Info

Publication number
WO2019080913A1
WO2019080913A1 PCT/CN2018/111971 CN2018111971W WO2019080913A1 WO 2019080913 A1 WO2019080913 A1 WO 2019080913A1 CN 2018111971 W CN2018111971 W CN 2018111971W WO 2019080913 A1 WO2019080913 A1 WO 2019080913A1
Authority
WO
WIPO (PCT)
Prior art keywords
video frame
initial video
transition
initial
color value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/111971
Other languages
English (en)
French (fr)
Inventor
周扬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2019080913A1 publication Critical patent/WO2019080913A1/zh
Priority to US16/590,460 priority Critical patent/US11032510B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/02—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed
    • G09G5/026—Control of mixing and/or overlay of colours in general
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H04N7/0137—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes dependent on presence/absence of motion, e.g. of motion zones
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/133—Equalising the characteristics of different image components, e.g. their average brightness or colour balance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/15—Processing image signals for colour aspects of image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/167—Synchronising or controlling image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/324—Colour aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/373—Image reproducers using viewer tracking for tracking forward-backward translational head movements, i.e. longitudinal movements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/376—Image reproducers using viewer tracking for tracking left-right translational head movements, i.e. lateral movements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213—Monitoring of end-user related data
    • H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/64—Circuits for processing colour signals
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0666—Adjustment of display parameters for control of colour parameters, e.g. colour temperature
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/16—Determination of a pixel data signal depending on the signal applied in the previous frame
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present application relates to the field of video technologies, and in particular, to a video processing method computer device and a storage medium.
  • the panoramic video is a video that can be taken in all directions as needed or a video produced by the image, and the video content can be presented 360 degrees.
  • the user can freely control the playback of the panoramic video up and down and left and right as needed.
  • Most of the panoramic video is used in scenes such as travel exhibitions, venues or city presentations.
  • the Virtual Reality (VR) device is one of the devices capable of playing panoramic video.
  • the VR device can be worn on the user's head, and the user can directly control the playback of the panoramic video by turning the head. Watch what the user wants.
  • a video processing method, computer device, and storage medium are provided.
  • the embodiment of the present application provides a video processing method, including:
  • the terminal determines a second initial video frame at a second time point, where the second time point is after the first time point;
  • a transition initial video frame from a first initial video frame at the first time point to a second initial video frame, where a color value of a pixel point of the target position in the transition initial video frame is located in a target interval
  • the target interval refers to a color value determined according to a color value of a pixel point of the target position in the first initial video frame and a color value of a pixel point of the target position in the second initial video frame.
  • the terminal performs playback control according to the transition initial video frame to complete playback of the panoramic video.
  • a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor such that the processor performs the following steps:
  • the target interval refers to a color value interval determined according to a color value of a pixel point of the target position in the first initial video frame and a color value of a pixel point of the target position in the second initial video frame.
  • One or more non-volatile storage media storing computer readable instructions, when executed by one or more processors, cause one or more processors to perform the following steps:
  • the target interval refers to a color value interval determined according to a color value of a pixel point of the target position in the first initial video frame and a color value of a pixel point of the target position in the second initial video frame.
  • FIG. 1 is a schematic diagram of a scene of panoramic video playback according to an embodiment of the present application
  • 1b is a schematic flowchart of determining a visual plane in panoramic video playback according to an embodiment of the present application
  • FIG. 2a is a schematic diagram of posture information in different directions according to an embodiment of the present application.
  • 2b is a schematic diagram of an image area in a panoramic video frame at different viewing angles of an embodiment according to an embodiment of the present application;
  • FIG. 3 is a schematic diagram of a play video frame played to a user during a panoramic video playing process according to an embodiment of the present application
  • FIG. 4 is a schematic flowchart of a method for determining color values of transition initial video frames in an embodiment of the present application
  • FIG. 5 is a schematic flowchart diagram of a video processing method according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart diagram of another video processing method according to an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an intelligent terminal according to an embodiment of the present application.
  • FIG. 9 is an internal structural diagram of an intelligent terminal according to an embodiment of the present application.
  • the playback control may be performed in multiple motion directions as needed, so as to adjust the video frame that can be played and displayed to the user in the panoramic video frame.
  • Panorama video can be played through a VR device.
  • the user can rotate the head in all directions of up, down, left, and right to view the video images at different orientations, as shown in FIG. 1a, displaying the image on the display interface of the first field of view toward the angle.
  • the image corresponding to the area 101 displays an image corresponding to the image area 102 on the display interface of the second field of view facing the angle.
  • the panoramic video may also be played by a dedicated panoramic video playback device.
  • the embodiment of the present application is described by taking a VR device as an example.
  • substantially all the light sources are emitted by the VR device, and the problem may be caused by the brightness change, in the embodiment of the present application, according to the played video.
  • the brightness of the picture in the frame controls the playback of the panoramic video, and after this control, the degree of change in brightness between the video frames played to the user is reduced.
  • the eyes of the person are completely immersed in the playing environment of the panoramic video.
  • Playing the panoramic video frame in the VR device may cause the brightness of the emitted light to change drastically, or in the panoramic video frame.
  • a part of the area black has a relatively high brightness, and in these cases, the above-mentioned problem of viewing discomfort due to a bright change occurs.
  • the control of the playback of the panoramic video is mainly performed by performing interpolation correction on the pixel points of the corresponding location area in the panoramic video frame that may be played, so that the brightness of the video pixel points seen by the user can be gradually changed. Try to have as much time as possible to allow the eye to adapt, which in turn reduces the stimuli of the VR device to the user's eyes.
  • the viewing position of the user may be predicted, for example, predicting the angle of view of the field of view after the user turns the head, and the playback control process of the panoramic video is performed according to the predicted viewing position.
  • the process of predicting mainly includes: directly predicting the user's movement in the next unit time, for example, 1/30 second, the user moves up, down, left, or right or keeps the head stationary. In this case, the viewing position in the five directions in the next unit time is predicted.
  • the video frame processing of the panoramic video is then performed based on the predicted result and the panoramic video frame that will be displayed to the user at the next unit time.
  • the video frame processing includes processing of the transition initial video frame.
  • the position (x, y, z) of the user in the three-dimensional space needs to be acquired in S101, and acquired in S102.
  • the user's field of view is oriented toward angle information ( ⁇ , ⁇ , ⁇ ), which is a coordinate point in a spherical coordinate, where ⁇ can be 1, and different fields of view are determined based on different ⁇ , ⁇ Orientation.
  • the panoramic video is played by the VR device
  • the VR device includes a Head Mount Display (HMD) device
  • the screen in the HMD device is used to provide a display interface to display the corresponding video frame in real time.
  • the user's eyeball can be tracked, real-time motion data of the eyeball can be acquired, and the visual field orientation angle information can be determined based on the motion data of the eyeball.
  • the VR device may further include a tracking system composed of a motion sensor capable of tracking position information and motion information of the user in a real three-dimensional space, and then determining the field of view orientation angle information based on the position information and the motion information.
  • the eyeball or the head may move, and the visual field orientation angle information at the corresponding time point may be comprehensively determined according to the movement trajectory of the eyeball and the movement trajectory of the head.
  • the VR device includes: a calculation processing unit capable of obtaining real-time position and motion information of the user from the tracking system and/or acquiring eye movement data obtained by tracking the eyeball, and calculating a three-dimensionality of the user's head in the virtual three-dimensional space. Coordinates, combined with the eye movement data obtained by tracking the eyeball, calculate the visual field orientation of the user in the virtual three-dimensional space, and the three-dimensional coordinates of the user's real-time fixation point.
  • the visual plane of the user on the panoramic video frame that needs to be played may be determined in S103, and the corresponding area of the visual plane is used as the image area, and the image can be displayed according to the image in the image area. Play video frames on the display interface.
  • an optimal line of sight d may be, for example, 2.5 m
  • a field of view indicating a field of view width field of view, FOV
  • experience value can be, for example, 90°
  • based on the user's head position x, y, z
  • the field of view orientation ⁇ , ⁇ , ⁇
  • the visual plane of the user's visual pause is obtained, and the visual plane can be obtained.
  • FIG. 2a predicts the next unit time point T+1 (which can be regarded as the second time point) when the current time point T (which can be considered as the first time point).
  • the schematic diagram of the attitude information at the time, the attitude information referred to in FIG. 2a mainly refers to the perspective information of the user's field of view.
  • the gesture information at the second time point in other embodiments may refer to displacement information of the user's head, such as a displacement to the left, a displacement to the right, and the like.
  • the attitude information at the first time point is the initial field of view orientation angle information of the user, which is ( ⁇ , ⁇ , ⁇ ).
  • the angle of view angle information is: ( ⁇ , ⁇ , ⁇ ), ( ⁇ 1, ⁇ 1, ⁇ 1), ( ⁇ 2, ⁇ 2, ⁇ 2), ( ⁇ 3, ⁇ 3, ⁇ 3), ( ⁇ 4, ⁇ 4) , ⁇ 4).
  • the rotational angular velocity of the user may be an empirical value, which may be considered as a rotational angular velocity when the user turns the head to view the panoramic video under normal conditions.
  • the field of view orientation in different directions can be actually obtained based on different ⁇ and ⁇ . For example, an obliquely upward field of view ( ⁇ 5, ⁇ 5, ⁇ 5) or the like can also be obtained.
  • the upward visual field orientation angle information ( ⁇ 1, ⁇ 1, ⁇ 1) at the second time point corresponds to the second initial video frame 201a and the second broadcast video frame 201b.
  • the downward field of view orientation angle information ( ⁇ 2, ⁇ 2, ⁇ 2) can correspond to the second initial video frame 202a and the second broadcast video frame 202b, to the left
  • the field of view orientation angle information ( ⁇ 3, ⁇ 3, ⁇ 3) can correspond to the second initial video frame 203a and the second play video frame 203b
  • the rightward view direction angle information ( ⁇ 4, ⁇ 4, ⁇ 4) can correspond to the second initial video frame 204a and The second play video frame 204b.
  • the current initial field of view orientation angle information at the first time corresponds to the first play video frame 200a and the first initial video frame 200b.
  • the embodiment of the present application needs to perform interpolation processing between the currently playing video frame and the four new video frames that may exist to obtain one or more transition initial video frames and a second initial video frame at the second time point. Therefore, after determining the actual rotation direction of the user, the transitional video frame and the second image that can be projected onto the display interface of the VR device to be played to the user can be obtained according to each transition initial video frame and the second initial video frame. Play a video frame.
  • the video frame obtained by the initial processing is referred to as an initial video frame
  • the video frame that can be finally projected to the display interface and played to the user is referred to as a play video frame
  • each initial video frame and the played video frame are The difference is that the frame height of the initial video frame is greater than the frame height of the playback video frame, and/or the frame width of the initial video frame is greater than the frame width of the playback video frame.
  • the frame width and the frame height of the initial video frame are correspondingly 1.4 times the frame width and frame height of the final played video frame.
  • each initial video frame is larger than the size of various playback video frames, so as to ensure that the range of video pixels that the user actually sees after turning is within the pixel range of the transition video frame, ensuring the final transition video that is finally played to the user.
  • the frame can play normally.
  • the size of the various initial video frames may also be the same as the size of the final played video frame.
  • a quantity is determined, where the number refers to a transition initial video between the first initial video frame and the second initial video frame.
  • the number of frames In one embodiment, if the video frame playing frequency of the panoramic video is 30 Hz and the VR display screen refresh frequency is 60 Hz, one transition initial video frame can be determined between every two video frames; if the VR display screen is refreshed The frequency is 90HZ, then two transition initial video frames can be determined between every two video frames; if the VR display screen refresh frequency is 120HZ, three transition initial video frames can be determined between every two video frames.
  • FIG. 3 is a schematic diagram of a playback video frame finally displayed to the user when the three initial initial video frames are corresponding, including the first playback video frame 301 and the three initial transition video frames 302 corresponding to the first initial video frame. And a second initial video frame 303.
  • interpolation between color values of pixel points at the same position between the first initial video frame and the second initial video frame may be obtained based on a preset interpolation rule, so as to be Interpolation determines the color values of the transition pixels and further generates each transition initial video frame.
  • the interpolation rule may be set as needed.
  • the color value of the pixel of the target position in the obtained initial video frame of the transition is located in the target interval, where the target interval is Refers to a color value interval determined according to a color value of a pixel point of a target position in the first initial video frame and a color value of a pixel point of a target position in the second initial video frame.
  • the transition initial video frame includes more than one, in addition to ensuring that the color value of the pixel of the target position in the transition initial video frame is within the target interval, it is further required to ensure two adjacent initial video frames.
  • the color value of the pixel at the target position is changed by less than a certain preset amplitude threshold to ensure a gradual transition of the brightness of the image.
  • FIG. 4 is a schematic flowchart of a method for determining a pixel point color value at a target position in each transition initial video frame according to an interpolation rule according to an embodiment of the present application, where the first initial video frame is The target position is the pixel position coordinate, and the target position in the second initial video frame is also the pixel position coordinate, and the two target positions are on different images but the image positions are the same.
  • Each pixel position in the first initial video frame and the second initial video frame can be processed as shown in FIG. 4 as a target position.
  • the first initial video frame at the first time point and the second initial video frame at the second time point are RGB images, and the component R1 of the RGB channel of the target position in the first initial video frame at the first time point is acquired in S401.
  • G1, B1, in S402 the components R2, G2, B2 of the RGB channel of the target position in the second initial video frame at the second time point are acquired.
  • n 1
  • n 1
  • n 1.
  • the color value components of the nth transition initial video frame obtained after the S404 are (RC1, GC2, BC3), and the color values of the pixel points of the target position in the nth transition initial video frame are obtained according to the color value component.
  • the target area may include: the target area in the first initial video frame is white, and the second initial video frame is in the second initial video frame.
  • the target area in the black is black, or the target area in the first initial video frame is black, and the target area in the second initial video frame is white.
  • the difference processing may be performed only on the black to white or white to black transition portions of the video frame, and the pixel points of the regions other than the one or more target regions are not subjected to the interpolation processing of the color values.
  • the color value of the pixel of the other area may be the same as the color value of the pixel of the corresponding position of the first initial video frame, or the color of the pixel of the corresponding position of the second initial video frame.
  • the target area may further mean that in the first initial video frame and the second initial video frame, the absolute value of the difference of the color values of each pixel in the target area is greater than the pre-predetermined value.
  • the difference threshold is set, and it can be considered that the difference in color values between the pixel points at the same position is large.
  • the color value of the pixel point outside the target area may be the same as the color value of the pixel point at the same position in the first initial video frame, or may be the color of the pixel point at the same position in the second initial video frame. The values are the same.
  • the RGB flux of black and white pixels can be customized.
  • a series of non-pure black blacks in different color values (0, 0, 0) to (1, 1, 1) in RGB can also be defined as black.
  • the specific definition values can be referred to Table 1 below.
  • the pixel point color value of the target position is #1C1C1C (grey11)
  • the target position of the second initial video frame of the second time point is
  • the pixel value of the pixel is #828282 (grey51)
  • the region composed of the target location and the other target location is the target region, and interpolation processing of the color values is performed based on the target region to form a transition initial video frame.
  • the target initial The video frame group includes the second initial video frame and the transition initial video frame in a certain direction obtained as described above. For example, if the user turns to the left to ( ⁇ 3, ⁇ 3, ⁇ 3), the second initial video frame to the left and the transition initial video frame corresponding to the second initial video frame to the left are selected.
  • the user's actual field of view orientation angle can be detected based on motion sensors on the VR device.
  • each initial video frame in the target initial video frame group needs to be projected onto the display interface of the VR device, and may be projected according to the field of view of the VR device.
  • a pixel range on the display interface of the VR device performing pixel coordinate matching according to the pixel range in each initial video frame in the target initial video frame group, selecting a pixel point to be projected, and forming an actual transition video frame and a second
  • the video frames are played and then played back to the VR display interface in chronological order.
  • the FOV of the VR device is an inherent parameter of the VR device.
  • the present application it is possible to predict and determine a second time point, and generate some transition video frames according to the video frame at the second time point and the video frame at the current first time point, so that the video played at the first time point can be guaranteed.
  • the degree of pixel color change between the frame and the video frame played at the second time point is small, which can effectively reduce the bad stimulation to the user's glasses when the panoramic video is played, and the whole process does not require the user to manually adjust the device such as VR. Brightness is played to meet the needs of users for automation and intelligence.
  • FIG. 5 is a schematic flowchart of a video processing method according to an embodiment of the present application.
  • the method in the embodiment of the present application may be performed by a terminal device for playing a panoramic video.
  • the method is used. It can be performed by a VR device capable of playing panoramic video.
  • the method of the embodiment of the present application includes the following steps.
  • S501 When playing the panoramic video, determining a second initial video frame at the second time point, the second time point being after the first time point.
  • the first time point and the second time point may differ by one unit time, and the unit time may be determined according to the video frame playing frequency during the panoramic video playing, for example, when the video frame playing frequency is 30 Hz, the unit time is 1/30. second.
  • the second initial video frame corresponding to the second time point may be one or more. If the user's motion direction is known in advance, the second initial video frame is the panoramic video played at the second time point. A video frame in the frame that is formed by a partial image of the angle of view corresponding to the direction of motion. For example, as shown in FIG. 2a, if it is known in advance that the user will turn to the left, it can be considered that in the panoramic video played at the second time point, the ( ⁇ 3, ⁇ 3, ⁇ 3) field of view corresponding to the leftward moving direction is oriented toward the angle.
  • the partial image that can be viewed can obtain a corresponding second initial video frame, and the second initial video frame is larger than the pixel range of the second play video frame that can be actually viewed.
  • the direction of motion of the user is not known, the case of leftward, rightward, upward, downward, and stationary as in Fig. 2a can be simultaneously used as the predicted direction of motion, and then five second initial video frames are determined.
  • S502 Generate a transition initial video frame from the first initial video frame at the first time point to the second initial video frame, where a color value of a pixel point of the target position in the transition initial video frame is located in a target interval
  • the target interval refers to a color value interval determined according to a color value of a pixel point of a target position in the first initial video frame and a color value of a pixel point of a target position in the second initial video frame.
  • a transition initial video frame is generated, and the video frames cause a small change in the pixel point at each target position, especially at the first time point and the first
  • the target position can be gradually blacked out between the first time point and the second time point. Turns white.
  • S503 Perform playback control according to the transition initial video frame to complete playback of the panoramic video.
  • Performing play control according to the transition initial video frame mainly includes processing the transition initial video frame and the second initial video frame to obtain a transition play video frame and a second play video frame that can be actually projected onto the display interface of the VR device. And play to the user.
  • the pixel range of the transition initial video frame and the second initial video frame may be determined based on the FOV of the VR device, and then the transition play video frame and the second play video frame are determined to be determined, and at the first time point and Each of the transitional video frames is played in chronological order between the second time and finally the second played video frame is played.
  • the transition initial video frames generated in the S502 may be transition initial video frame data, and the generated initial video frame data can be generated on the display interface for display by the user.
  • FIG. 6 is a schematic flowchart of another video processing method according to an embodiment of the present application
  • the method in the embodiment of the present application may be performed by a terminal device for playing a panoramic video.
  • the method can be performed by a VR device capable of playing panoramic video.
  • the method of the embodiment of the present application includes the following steps.
  • the motion velocity value can be an estimate that can be dynamically determined based on the angular velocity of the user's head rotation during a period of time during which the VR device is watching the panoramic video over a period of time, such as an average angular velocity over a period of time.
  • S602 Obtain second posture information at the second time point according to the first posture information at the first time point, and according to the motion speed value and the duration between the first time point and the second time point.
  • the amount of posture change from the first time point to the second time point such as the amount of angular change, can be calculated, thereby obtaining
  • the second attitude information because the motion speed value is an estimated value, the second attitude information is also an estimated value, and is not the posture information after the user actually rotates, in order to facilitate the subsequent calculation of the transition initial video frame. It is convenient to finally play the transition play video frame corresponding to each transition initial video frame before playing the second play video frame at the second time point.
  • S603 Acquire a panoramic video frame of the panoramic video to be played at the second time point. After the panoramic video is recorded, the panoramic video frame at each time point is known, and the panoramic video frame can be obtained based on the time point, so that the subsequent partial image is intercepted from the panoramic video frame to form the second initial video frame and can be The second play video frame displayed on the display interface is displayed.
  • the S601 to S604 correspond to S501 in the previous embodiment.
  • the second posture information is a field of view orientation angle information. According to the angle of view of the user, the partial image can be found and intercepted from the panoramic video frame to form a second initial video frame, which can be referred to FIG. 3 .
  • the selected frame width of the second initial video frame obtained from the panoramic video frame is greater than the frame width of the play video frame that can be actually played on the display interface, and/or from the panoramic video.
  • Obtaining a frame height of the second initial video frame in the frame is greater than a frame height of the play video frame that can be actually displayed on the display interface; and playing the displayed play video frame on the display interface means: in the second posture Under the information, a video frame composed of a part of the image projected onto the terminal display interface in the panoramic video frame.
  • the size of the second initial video frame is larger than the size of the second play video frame that can actually be displayed on the display interface, thus ensuring a transitional video frame from the first time point to the second time point. Can be accurately derived and played normally.
  • the motion velocity value includes rotational angular velocity values in at least two different directions
  • the second posture information refers to visual field orientation angle information
  • each initial direction is associated with an initial video frame group
  • each initial The video frame group includes a second initial video frame and a transition initial video frame obtained from the second initial video frame.
  • S605 Generate a transition initial video frame from the first initial video frame at the first time point to the second initial video frame, where a color value of a pixel point of the target position in the transition initial video frame is located in a target interval
  • the target interval refers to a color value interval determined according to a color value of a pixel point of a target position in the first initial video frame and a color value of a pixel point of a target position in the second initial video frame.
  • the S605 may include: acquiring a first color value of a pixel point of the target position in the first initial video frame at the first time point; acquiring the pixel point of the target position in the second initial video frame a second color value; performing interpolation processing according to the first color value and the second color value to obtain a color value of a transition pixel point at a target position of the transition initial video frame; after obtaining color values of the plurality of transition pixel points, according to the obtained The color values of all the transition pixels are generated to generate a transitional initial video frame.
  • the performing interpolation processing according to the first color value and the second color value to obtain a color value of a transition pixel point at a target position of the transition initial video frame including: a color channel in the first color value Performing difference calculation on the color channel components in the component and the second color value to obtain component difference values of the respective color channel components; acquiring the amount of change of each color channel component according to the number of transition initial video frames generated as needed; The color channel values of the color values, the number of transition initial video frames that need to be generated, and the amount of change of each color channel component are obtained as color values of transition pixel points of the target positions of the respective transition initial video frames.
  • the process of calculating the color values of the transition pixels may refer to the description of the embodiment corresponding to FIG. 4.
  • the pixel positions of all the pixels of the first initial video frame and the second initial video frame are respectively used as the target position; or the pixel points of the target position in the first initial video frame
  • the color value is black, and the color value of the pixel value of the target position in the second initial video frame is white; or, the color value of the pixel of the target position in the first initial video frame is white, And the color value of the pixel value of the target position in the second initial video frame is black.
  • the video frame playing frequency of the panoramic video is 30 Hz
  • the VR display screen refresh frequency is 60 Hz
  • one transition initial video frame can be determined between every two video frames
  • the VR display screen The refresh frequency is 90HZ
  • two transition initial video frames can be determined between every two video frames
  • the VR display screen refresh frequency is 120HZ
  • three transition initial video frames can be determined between every two video frames.
  • S606 Detecting the direction of motion.
  • the direction of motion of the VR device can be detected based on sensors on the VR device, such as by a gyroscope to detect whether the VR device is turning up, down, left, or right, or no motion.
  • S607 Select an initial video frame group associated with the motion direction to obtain a target initial video frame group. For example, when it is determined that the user's head is rotated to the left, the second play video frame 203a as shown in FIG. 2b and each transition initial video frame calculated based on the second play video frame 203a may be selected. Target initial video frame group.
  • S608 Perform playback control according to the target initial video frame group.
  • the second play video frame and each of the transition play video frames that can be displayed on the VR device display interface may be generated according to the size of the second initial video frame and the respective transition initial video frames in the initial video frame group. And playing each second playing video frame and each transition playing video frame on the display interface.
  • the S608 may specifically include: acquiring a viewing angle parameter of the terminal; determining, according to the viewing angle parameter, a range of pixels projected onto the display interface of the terminal; and in the initial video frame group of the target In each initial video frame, the pixel coordinates are matched according to the pixel range, and the pixel points that need to be projected are selected to form a transition play video frame and a second play video frame; and the transition play video frame and the second play video frame are sequentially played in chronological order.
  • the present application it is possible to predict and determine a second time point, and generate some transition video frames according to the video frame at the second time point and the video frame at the current first time point, so that the video played at the first time point can be guaranteed.
  • the degree of pixel color change between the frame and the video frame played at the second time point is small, which can effectively reduce the bad stimulation to the user's glasses when the panoramic video is played, and the whole process does not require the user to manually adjust the device such as VR. Brightness is played to meet the needs of users for automation and intelligence.
  • an intelligent terminal is further provided.
  • the internal structure of the smart terminal is as shown in FIG. 9.
  • the smart terminal includes a video processing device, and the video processing device includes various modules, and each module may be all or part of It is implemented by software, hardware or a combination thereof.
  • a video processing device and an intelligent terminal according to an embodiment of the present application are described in detail below.
  • FIG. 7 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present application.
  • the apparatus in the embodiment of the present application may be disposed on a terminal that performs panoramic playback, such as a VR terminal.
  • the device includes the following modules.
  • a determining module 701 configured to determine a second initial video frame at a second time point when the panoramic video is played, where the second time point is after the first time point;
  • a generating module 702 configured to generate a transition initial video frame from a first initial video frame at the first time point to a second initial video frame, where a color of a pixel of a target position in the initial video frame is converted The value is located in a target interval, where the target interval is a color value of a pixel point according to the target position in the first initial video frame and a color value of a pixel point in the target position in the second initial video frame.
  • the determined color value interval is a color value of a pixel point according to the target position in the first initial video frame and a color value of a pixel point in the target position in the second initial video frame.
  • the control module 703 is configured to perform playback control according to the transition initial video frame to complete playback of the panoramic video.
  • the determining module 701 is configured to acquire a motion speed value, according to the first posture information of the first time point, and according to the motion speed value, and the first time point and the second time point. a duration of time, obtaining second posture information at a second time point; acquiring a panoramic video frame of the panoramic video to be played at the second time point; acquiring from the panoramic video frame according to the second posture information The second initial video frame.
  • the frame width of the second initial video frame obtained from the panoramic video frame is greater than the frame width of the play video frame that can be actually played on the display interface, and/or from the panoramic video frame.
  • Obtaining a frame height of the second initial video frame is greater than a frame height of the play video frame that can be actually displayed on the display interface; and the playing video frame that can be actually displayed on the display interface refers to: projecting to the terminal in the panoramic video frame A video frame composed of partial images on the interface is displayed.
  • the motion velocity value includes a rotational angular velocity value in at least two different motion directions
  • the second posture information refers to a visual field orientation angle information
  • each initial direction is associated with an initial video frame group
  • each An initial video frame group includes a second initial video frame and a transition initial video frame obtained from the second initial video frame.
  • the generating module 702 is further configured to acquire a video frame refresh frequency of the terminal, acquire a video frame playing frequency when the panoramic video is played, and determine a transition according to the video frame refresh frequency and the video frame playing frequency.
  • the number of initial video frames; the number of generated transitional initial video frames is the same as the determined number.
  • the generating module 702 is configured to acquire, when used to generate a transition initial video frame from the first initial video frame at the first time point to the second initial video frame. a first color value of a pixel point of the target position in the first initial video frame at the first time point; acquiring a second color value of the pixel point of the target position in the second initial video frame; according to the first color value and the first color value
  • the two color values are subjected to interpolation processing to obtain color values of transition pixel points at the target position of the transition initial video frame; after obtaining color values of the plurality of transition pixel points, generating color values according to all obtained transition pixel points Transition the initial video frame.
  • the generating module 702 is configured to perform interpolation processing according to the first color value and the second color value to obtain a color value of a transition pixel of the target position of the transition initial video frame. Performing a difference calculation on the color channel component in the first color value and the color channel component in the second color value to obtain component difference values of the respective color channel components; acquiring each of the number of transition initial video frames generated according to the need The amount of change of the color channel component; the transition pixel of the target position of each transition initial video frame is obtained according to each color channel component of the first color value, the number of transition initial video frames to be generated, and the amount of change of each color channel component The color value.
  • the pixel positions of all the pixels of the first initial video frame and the second initial video frame are respectively used as the target position; or, in the first initial video frame, the target position The color value of the pixel is black, and the color value of the pixel value of the target position in the second initial video frame is white; or, the pixel of the target position in the first initial video frame The color value is white, and the color value of the pixel value of the target position in the second initial video frame is black.
  • control module 703 is configured to detect a motion direction, select an initial video frame group associated with the motion direction, obtain a target initial video frame group, and perform playback control according to the target initial video frame group.
  • control module 703 is configured to acquire a viewing angle parameter of the terminal when performing playback control according to the target initial video frame group, and determine to project the projection according to the viewing angle parameter.
  • the terminal displays a pixel range on the interface; and each initial video frame in the target initial video frame group performs pixel coordinate matching according to the pixel range to form a transition play video frame and a second play video frame; and sequentially play the transition play video frame in time sequence. And the second play video frame.
  • each functional module in the device may refer to the description of related content in the foregoing various embodiments.
  • the present application it is possible to predict and determine a second time point, and generate some transition video frames according to the video frame at the second time point and the video frame at the current first time point, so that the video played at the first time point can be guaranteed.
  • the degree of pixel color change between the frame and the video frame played at the second time point is small, which can effectively reduce the bad stimulation to the user's glasses when playing the panoramic video, and the whole process does not require the user to manually adjust the playback of the VR and other devices. Brightness meets the needs of users for automation and intelligence
  • FIG. 8 is a schematic structural diagram of an intelligent terminal according to an embodiment of the present application.
  • the terminal in the embodiment of the present application may be a terminal capable of playing panoramic video, for example, may be a VR device.
  • the smart terminal includes a power supply module and the like, and further includes a processor 801, a storage device 802, and a data interface 803 in the embodiment of the present application.
  • a data connection can be established between the processor 801, the storage device 802, and the data interface 803 by way of a bus or the like.
  • the data interface 803 is configured to receive external data and send the external data to the processor 801 or to the storage device 802.
  • the storage device 802 may include a volatile memory, such as a random-access memory (RAM); the storage device 802 may also include a non-volatile memory, such as a fast A flash memory, a solid-state drive (SSD) or the like; the storage device 802 may further include a combination of the above types of memories.
  • RAM random-access memory
  • SSD solid-state drive
  • the processor 801 can be a central processing unit (CPU).
  • the processor 801 may further include a hardware chip.
  • the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or the like.
  • the PLD may be a field-programmable gate array (FPGA), a general array logic (GAL), or the like.
  • the storage device 802 is further configured to store program instructions.
  • the processor 801 can invoke the program instructions to implement the method mentioned above in the embodiments of the present application.
  • the processor 801 is configured to invoke the program instruction to determine a second initial video frame at a second time point after playing the panoramic video, the second time point being after the first time point Generating a transition initial video frame from the first initial video frame at the first time point to the second initial video frame, wherein a color value of a pixel point of the target position in the transition initial video frame is located in the target interval
  • the target interval refers to a color value determined according to a color value of a pixel point of the target position in the first initial video frame and a color value of a pixel point of the target position in the second initial video frame. Interval; performing playback control according to the transition initial video frame to complete playback of the panoramic video.
  • the processor 801 is configured to acquire a motion speed value when determining a second initial video frame at a second time point; according to the first attitude information at the first time point, and according to the a motion velocity value and a duration between the first time point and the second time point, obtaining second posture information at the second time point; acquiring a panoramic video frame of the panoramic video to be played at the second time point And acquiring, according to the second posture information, a second initial video frame from the panoramic video frame.
  • the frame width of the second initial video frame obtained from the panoramic video frame is greater than the frame width of the play video frame that can be actually played on the display interface, and/or from the panoramic video frame.
  • Obtaining a frame height of the second initial video frame is greater than a frame height of the play video frame that can be actually displayed on the display interface; and the playing video frame that can be actually displayed on the display interface refers to: projecting to the terminal in the panoramic video frame A video frame composed of partial images on the interface is displayed.
  • the motion velocity value includes a rotational angular velocity value in at least two different motion directions
  • the second posture information refers to a visual field orientation angle information
  • each initial direction is associated with an initial video frame group
  • each An initial video frame group includes a second initial video frame and a transition initial video frame obtained from the second initial video frame.
  • the processor 801 is further configured to acquire a video frame refresh frequency of the terminal, acquire a video frame playing frequency when the panoramic video is played, and determine a transition according to the video frame refresh frequency and the video frame playing frequency.
  • the number of initial video frames; the number of generated transitional initial video frames is the same as the determined number.
  • the processor 801 when used to generate a transition initial video frame from the first initial video frame to the second initial video frame at the first time point, is used to acquire a first color value of a pixel point of the target position in the first initial video frame at the first time point; acquiring a second color value of the pixel point of the target position in the second initial video frame; according to the first color value and the first color value
  • the two color values are subjected to interpolation processing to obtain color values of transition pixel points at the target position of the transition initial video frame; after obtaining color values of the plurality of transition pixel points, generating color values according to all obtained transition pixel points Transition the initial video frame.
  • the processor 801 is configured to perform interpolation processing according to the first color value and the second color value to obtain a color value of a transition pixel of the target position of the transition initial video frame. Performing a difference calculation on the color channel component in the first color value and the color channel component in the second color value to obtain component difference values of the respective color channel components; acquiring each of the number of transition initial video frames generated according to the need The amount of change of the color channel component; the transition pixel of the target position of each transition initial video frame is obtained according to each color channel component of the first color value, the number of transition initial video frames to be generated, and the amount of change of each color channel component The color value.
  • the pixel positions of all the pixels of the first initial video frame and the second initial video frame are respectively used as the target position; or, in the first initial video frame, the target position The color value of the pixel is black, and the color value of the pixel value of the target position in the second initial video frame is white; or, the pixel of the target position in the first initial video frame The color value is white, and the color value of the pixel value of the target position in the second initial video frame is black.
  • the processor 801 is configured to detect a motion direction when performing play control according to the transition initial video frame, and select an initial video frame group associated with the motion direction to obtain a target initial video frame. a group; playing control according to the target initial video frame group.
  • the processor 801 is configured to acquire a viewing angle parameter of the terminal when performing playback control according to the target initial video frame group, and determine to project the projection according to the viewing angle parameter.
  • the terminal displays a pixel range on the interface; and each initial video frame in the target initial video frame group performs pixel coordinate matching according to the pixel range to form a transition play video frame and a second play video frame; and sequentially play the transition play video frame in time sequence. And the second play video frame.
  • each function of the processor 801 may refer to the description of related content in the foregoing various embodiments.
  • the present application it is possible to predict and determine a second time point, and generate some transition video frames according to the video frame at the second time point and the video frame at the current first time point, so that the video played at the first time point can be guaranteed.
  • the degree of pixel color change between the frame and the video frame played at the second time point is small, which can effectively reduce the bad stimulation to the user's glasses when playing the panoramic video, and the whole process does not require the user to manually adjust the playback of the VR and other devices. Brightness meets the needs of users for automation and intelligence.
  • an internal structure diagram of an intelligent terminal includes a processor, a memory, a network interface, an input device, and a display screen connected by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device stores an operating system and can also store computer readable instructions that, when executed by the processor, cause the processor to implement a video processing method.
  • the internal memory can also store computer readable instructions that, when executed by the processor, cause the processor to perform a video processing method.
  • the display screen of the computer device may be a liquid crystal display or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touchpad provided on the computer device casing, and It can be an external keyboard, trackpad or mouse.
  • FIG. 9 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied.
  • the specific server may include a ratio. More or fewer components are shown in the figures, or some components are combined, or have different component arrangements.
  • the various steps in the various embodiments of the present application are not necessarily performed in the order indicated by the steps. Except as explicitly stated herein, the execution of these steps is not strictly limited, and the steps may be performed in other orders. Moreover, at least some of the steps in the embodiments may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be executed at different times, and the execution of these sub-steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of the other steps.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Hardware Design (AREA)
  • Databases & Information Systems (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请实施例公开了一种视频处理方法、计算机设备和存储介质，其中，方法包括：终端在播放全景视频时，确定第二时间点的第二初始视频帧，第二时间点在第一时间点之后；生成从第一时间点的第一初始视频帧到第二初始视频帧之间的过渡初始视频帧，过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，目标区间是指根据第一初始视频帧中目标位置的像素点的颜色值与第二初始视频帧中目标位置的像素点的颜色值所确定的颜色值区间；根据过渡初始视频帧进行播放控制，以完成全景视频的播放。

Description

视频处理方法、计算机设备和存储介质
本申请要求于2017年10月27日提交中国专利局，申请号为2017110218980，申请名称为“一种视频处理方法、装置及智能终端、存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频技术领域，尤其涉及一种视频处理方法计算机设备和存储介质。
背景技术
全景视频是一种可以根据需要进行全方位拍摄得到的视频或者通过图像制作得到的视频，可以360度呈现视频内容。用户在观看全景视频时，可以根据需要随意对全景视频进行上下左右的播放控制。全景视频大多应用在旅游展览，会场或者城市介绍等场景中。
而虚拟现实(Virtual Reality，VR)设备是其中一种能够播放全景视频的设备，VR设备可以佩戴在用户头部，用户可以直接通过转动头部，来实现对全景视频进行上下左右的播放控制，观看用户想要的画面内容。
而如何对全景视频进行播放控制以满足用户需求成为研究的热点。
发明内容
根据本申请提供的各种实施例，提供一种视频处理方法、计算机设备和存储介质。
一方面，本申请实施例提供了一种视频处理方法，包括：
终端在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
终端生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的 颜色值区间；
终端根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如下步骤：
在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
附图说明
图1a是本申请实施例的全景视频播放的场景示意图；
图1b是本申请实施例的全景视频播放中确定视觉平面的流程示意图；
图2a是本申请实施例的在不同方向上的姿态信息的示意图；
图2b是本申请实施例的一种在不同的视野朝向角度下全景视频帧中的图像区域示意图；
图3是本申请实施例的全景视频播放过程中播放给用户的播放视频帧的示意图；
图4是本申请实施例的确定各过渡初始视频帧颜色值的方法流程示意图；
图5是本申请实施例的一种视频处理方法的流程示意图；
图6是本申请实施例的另一种视频处理方法的流程示意图；
图7是本申请实施例的一种视频处理装置的结构示意图；
图8是本申请实施例的一种智能终端的结构示意图；
图9是本申请实施例的一种智能终端的内部结构图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例中在播放全景视频时，可以根据需要在多个运动方向上进行播放控制，以便于在全景视频帧中调节得到可以播放显示给用户的视频帧。全景视频可以通过VR设备播放。通过VR设备播放全景视频时，用户可以上、下、左、右各个方向转动头部，来观看不同方位处的视频画面，如图1a所示，在第一视野朝向角度的显示界面上显示图像区域101对应的图像，在第二视野朝向角度的显示界面上显示图像区域102对应的图像。在其他实施例中，全景视频也可以有专用的全景视频播放设备进行播放，本申请实施例以VR设备为例来进行说明。
在用户戴上VR设备播放全景视频的过程中，基本上所有的光源都是VR设备发出的，针对可能出现由于明亮变化引起的观看不适的问题，在本申请实施例中，可以根据播放的视频帧中画面的明亮度来对全景视频的播放进行控制，经过该控制后使得播放给用户的视频帧之间的明亮度的变化程度降低。在观看VR设备播放的全景视频时，人的双眼是完全沉浸在全景视频的播放环境中，VR设备中播放全景视频帧可能会导致发出的光线明亮度发生剧烈变化，或者在全景视频帧中大面积黑色中有一部分区域亮度相对较高，这些情况下都会出现上 述的由于明亮变化引起的观看不适的问题。
在本申请实施例中，对全景视频的播放进行控制主要是对可能会播放的全景视频帧中的相应位置区域的像素点进行插值修正，使得用户看到的视频像素点的亮度能够渐变过渡，尽可能有更长的时间来让眼球来适应，进而降低VR设备对用户的眼睛的刺激。在一个实施例中，可以对用户的观看位置进行预测，例如预测用户转动头部后的视野朝向角度，根据预测后的观看位置来进行全景视频的播放控制处理。在一个实施例中，预测的过程主要包括：直接预测用户在下一个单位时间内，例如1/30秒的单位时间内，用户向上、向下、向左、向右的运动或保持头部静止的情况下，来预测下一单位时间内5个方向上的观看位置。再根据预测的结果和在下一个单位时间会展示给用户的全景视频帧来进行全景视频的视频帧处理。该视频帧处理包括过渡初始视频帧的处理。
如图1b所示，为了在显示界面上显示全景图像的图像区域101和图像区域102的画面，需要在S101中获取用户在三维空间中的位置(x，y，z)，并在S102中获取用户的视野朝向角度信息(α，β，γ)，所述(α，β，γ)为一个球坐标下的坐标点，其中的γ可以为1，基于不同的α，β来确定不同的视野朝向。在本申请实施例中，通过VR设备播放全景视频，VR设备包括头戴式显示器(Head Mount Display，HMD)装置，HMD装置中的屏幕用于提供显示界面以实时显示相应的视频帧。另外，在HMD装置中还能够对用户的眼球进行追踪，能够获取到眼球的实时运动数据，进而根据眼球的运动数据来确定视野朝向角度信息。VR设备还可以包括一个由运动传感器构成的跟踪系统，该跟踪系统能够追踪用户在真实三维空间的位置信息与运动信息，进而根据这些位置信息与运动信息来确定视野朝向角度信息。在一个实施例中，眼球或头部都可能移动，可以根据眼球的移动轨迹和头部的移动轨迹，来综合确定在相应的时间点下视野朝向角度信息。所述VR设备包括：计算处理单元，能够从跟踪系统中获得用户的实时位置与运动信息和/或获取对眼球进行跟踪得到的眼球运动数据，并计算出用户头部在虚拟三维空间中的三维坐标，同时结合对眼球进行跟踪得到的眼球运动数据，计算得到用户在虚拟三维空间中的视野朝向，以及用户实时的注视点三维坐标。在确定了用户的视野朝向角度信息后，可以在S103中确定用户在需要播放的全景视频帧上的视觉平面，将该视觉平面对应区域作为图像区域，并根据该图像区域内的图像生成能够显示在显示界面上的播放视频帧。在一个实 施例中，在S103确定视觉平面的过程中，可以基于一系列经验数据，例如最佳视距d(经验值例如可为2.5m)、表示视野宽度的视场角(field of view，FOV)(经验值例如可为90°)，基于用户头部位置(x,y,z)，视野朝向(α,β,γ)，得到用户的视觉停留的视觉平面，具体可以得到该视觉平面的长与宽以及该视觉平面上每个点的空间三维坐标。
在本申请实施例中，如图2a所示，图2a是在当前时间点T(可以认为是第一时间点)时，预测下一个单位时间点T+1(可以认为是第二时间点)时的姿态信息的示意图，在图2a中涉及的姿态信息主要是指用户的视野朝向角度信息。在其他实施例中第二时间点的姿态信息可以是指用户头部的位移信息，例如向左移动的位移，向右移动的位移等等。在图2a中，第一时间点的姿态信息是用户的初始视野朝向角度信息，为(α，β，γ)。根据用户的旋转角速度*单位时间，得到用户可能的视角有向上(α1，β1，γ1)、向下(α2，β2，γ2)、向左(α3，β3，γ3)、向右(α4，β4，γ4)，当然也有可能视野朝向不变仍为(α，β，γ)。因此，第二时间点时，视角朝向角度信息为：(α，β，γ)、(α1，β1，γ1)、(α2，β2，γ2)、(α3，β3，γ3)、(α4，β4，γ4)。其中，所述用户的旋转角速度可以为一个经验值，可以认为是正常情况下，用户转动头部观看全景视频时的旋转角速度。在其他实施例中，基于不同的α和β实际可以得到不同方向上的视野朝向，例如，还可以得到斜向上的视野朝向(α5，β5，γ5)等。
在图2b中，不同的视野朝向角度下，能够看到不同的全景视频帧中的部分图像，在本申请实施例中，第二时间点下，向上的视野朝向角度信息(α1，β1，γ1)能够对应第二初始视频帧201a和第二播放视频帧201b，向下的视野朝向角度信息(α2，β2，γ2)能够对应第二初始视频帧202a和第二播放视频帧202b，向左的视野朝向角度信息(α3，β3，γ3)能够对应第二初始视频帧203a和第二播放视频帧203b，向右的视野朝向角度信息(α4，β4，γ4)能够对应第二初始视频帧204a和第二播放视频帧204b。在一个实施例中，如果用户的头部不转动，中间的图像区域在第二时间点也会出现新的图像，构成新的播放视频帧。而第一时间当前的初始视野朝向角度信息则对应第一播放视频帧200a和第一初始视频帧200b。
本申请实施例需要在当前播放的播放视频帧到可能存在的四个新的播放视频帧之间进行插值过渡处理，得到一个或者多个过渡初始视频帧和第二时间点 的第二初始视频帧，以便于后续在确定了用户的实际转动方向后，可以根据各个过渡初始视频帧和第二初始视频帧，得到能够投影到VR设备的显示界面上以播放给用户的过渡播放视频帧和第二播放视频帧。
在本申请实施例中，初始处理得到的视频帧称之为初始视频帧，而可供最终投影到显示界面播放给用户的视频帧称之为播放视频帧，各个初始视频帧与播放视频帧之间的差别在于：初始视频帧的帧高大于播放视频帧的帧高，和/或，初始视频帧的帧宽大于播放视频帧的帧宽。在一个实施例中，初始视频帧的帧宽、帧高均对应地为最终的播放视频帧的帧宽、帧高的1.4倍。各种初始视频帧的尺寸大于各种播放视频帧的尺寸，才能保证用户实际转向后看到的视频像素点范围是在过渡视频帧的像素点范围之内，确保最终播放给用户的各个过渡视频帧能够正常播放。当然，在其他实施例中，各种初始视频帧的尺寸也可以与最终的播放视频帧的尺寸相同。
在一个实施例中，根据全景视频正常播放的视频帧播放频率和VR设备的视频帧刷新频率，确定出一个数量，该数量是指第一初始视频帧与第二初始视频帧之间过渡初始视频帧的数量。在一个实施例中，如果全景视频播放的视频帧播放频率为30Hz，而VR显示屏幕刷新频率是60HZ，则每两个视频帧之间可以确定出1个过渡初始视频帧；如果VR显示屏幕刷新频率是90HZ，则每两个视频帧之间可以确定出2个过渡初始视频帧；如果VR显示屏幕刷新频率是120HZ，则每两个视频帧之间可以确定出3个过渡初始视频帧。其中，图3示出了对应3个过渡初始视频帧时，最终显示给用户的播放视频帧的示意图，包括了第一初始视频帧对应的第一播放视频帧301、三个过渡初始视频帧302、以及第二初始视频帧303。
在确定了过渡初始视频帧的数量后，可基于预设的插值规则，得到第一初始视频帧和第二初始视频帧之间相同位置上的像素点的颜色值之间的插值，以便于根据插值确定过渡像素点的颜色值并进一步生成各个过渡初始视频帧。所述插值规则可以根据需要进行设定，在一个实施例中，根据设定后的插值规则，得到的过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中目标位置的像素点的颜色值与所述第二初始视频帧中目标位置的像素点的颜色值所确定的颜色值区间。在一个实施例中，如果过渡初始视频帧包括多个时，除了保证过渡初始视频帧中目标位置的 像素点的颜色值位于目标区间内，还进一步地需要保证相邻的两个过渡初始视频帧中目标位置处的像素点的颜色值的变化幅度小于某个预设的幅度阈值，以保证图像的明亮程度能够渐变过渡。
在一个实施例中，如图4所示，图4是本申请实施例的按照插值规则确定各过渡初始视频帧中目标位置处像素点颜色值的方法流程示意图，所述第一初始视频帧的目标位置为像素位置坐标，所述第二初始视频帧中的目标位置也为像素位置坐标，两个目标位置在不同图像上但图像位置是相同的。所述第一初始视频帧和第二初始视频帧中每一个像素位置均可以作为目标位置进行如图4所示的处理。在第一时间点的第一初始视频帧和第二时间点的第二初始视频帧为RGB图像，在S401中获取第一时间点的第一初始视频帧中目标位置的RGB通道的分量R1、G1、B1，在S402中获取第二时间点的第二初始视频帧中目标位置的RGB通道的分量R2、G2、B2。在S403中计算通道的分量差值，计算方式可以为DR＝R2-R1，DG＝G2-G1，DB＝B2-B1，所述的分量差值为绝对值。在S404中根据上述确定的过渡初始视频帧的数量，得到每个过渡初始视频帧的目标位置处像素点颜色通道分量的变化量，计算方式可以为DTR＝DR/N，DTG＝DG/N，DTB＝DB/N。在S405中，根据第一初始视频帧目标位置处的像素点的各个颜色通道分量、每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值，计算方式可以为：RC1＝R1+n*DTR，GC2＝G2+n*DTG，BC3＝B3+n*DTB，其中，n是指过渡初始视频帧的序号，如果过渡初始视频帧有三帧，则n依次等于1、2、3，如果有两帧则n依次等于1、2，如果仅有一帧则n＝1。经过所述S404后得到的第n个过渡初始视频帧的各个颜色值分量为(RC1、GC2、BC3)，根据颜色值分量得到第n个过渡初始视频帧中目标位置的像素点的颜色值。
上述是根据第一初始视频帧和第二初始视频帧中所有位置的像素点的颜色值来确定中间的过渡初始视频帧，在一个实施例中，还可以仅将第一初始视频帧和第二初始视频帧中目标区域中的像素点的位置分别作为目标位置，在一个实施例中，该目标区域可以包括：在第一初始视频帧中的该目标区域为白色，而在第二初始视频帧中的该目标区域为黑色，或者，在第一初始视频帧中的该目标区域为黑色，而在第二初始视频帧中的该目标区域为白色。也就是说，可以仅对视频帧中黑色到白色，或者白色到黑色的跳变部分进行上述的差值处理， 除一个或者多个目标区域以外的区域的像素点不做颜色值的插值处理，其他区域的像素点的颜色值可以与第一初始视频帧相应位置的像素点的颜色值相同、或者与第二初始视频帧相应位置的像素点的颜色值相同。在一个实施例中，所述的目标区域还可以是指：在第一初始视频帧和第二初始视频帧中，该目标区域内的各个像素点的颜色值的差值的绝对值均大于预设的差值阈值，可以认为相同位置处的像素点之间颜色值差值较大。同样，在所述目标区域以外的像素点的颜色值可以与第一初始视频帧中相同位置处的像素点的颜色值相同，或者可以与第二初始视频帧中相同位置处的像素点的颜色值相同。
黑色和白色的像素点的RGB通量可以自定义。在一个实施例中，可以将RGB中不同的颜色值(0，0，0)到(1，1，1)中一系列的非纯黑的黑色也都定义为黑色。具体的定义值可参考下述的表1所示。根据表1所示，当检测到第一时间点的第一初始视频帧中目标位置的像素点颜色值为#1C1C1C(grey11)，而第二时间点的第二初始视频帧中所述目标位置处的像素点的像素值为#828282(grey51)时，可以认为两者的颜色值的差值较大，而如果与该目标位置距离较近的其他目标位置也存在颜色值差值较大的情况时，则可以认为由目标位置和所述其他目标位置构成的区域为目标区域，需要基于该目标区域进行颜色值的插值处理，以形成过渡初始视频帧。
表1
标识 颜色分量 颜色值
grey11 28 28 28 #1C1C1C
grey21 54 54 54 #363636
grey31 79 79 79 #4F4F4F
grey41 105 105 105 #696969
grey51 130 130 130 #828282
grey61 156 156 156 #9C9C9C
grey71 181 181 181 #B5B5B5
grey81 207 207 207 #CFCFCF
grey91 232 232 232 #E8E8E8
在得到各个目标位置处的颜色值后或者检测到用户向某个方向运动的情况下，获取用户的实际视野朝向角度，并根据实际视野朝向角度来确定出对应的 目标初始视频帧组，目标初始视频帧组包括上述预估得到的某个方向上的第二初始视频帧以及过渡初始视频帧。例如，如果用户向左转动至(α3，β3，γ3)，则选择向左的第二初始视频帧以及根据该向左的第二初始视频帧所对应得到的过渡初始视频帧。用户的实际视野朝向角度可以基于VR设备上的运动传感器来检测得到。
在得到了目标初始视频帧组后，需要将该目标初始视频帧组中各个初始视频帧投影到VR设备的显示界面上，可以根据VR设备的视场角(field of view，FOV获得会投影到VR设备的显示界面上的像素范围，对所述目标初始视频帧组中各个初始视频帧中根据像素范围进行像素坐标匹配，选出需要投影的像素点，构成实际的过渡播放视频帧和第二播放视频帧，再按照时间顺序依次播放到VR显示界面上。VR设备的FOV为VR设备的固有参数。
本申请实施例中，能够预测确定一个第二时间点，并根据第二时间点的视频帧和当前第一时间点的视频帧来生成一些过渡视频帧，这样可以保证第一时间点播放的视频帧与第二时间点播放的视频帧之间的像素颜色变化程度较小，能够有效地降低播放全景视频时对用户的眼镜造成的不良刺激，并且整个过程并不需要用户手动调节VR等设备的播放明亮度，满足了用户自动化、智能化的需求。
再请参见图5，是本申请实施例的一种视频处理方法的流程示意图，本申请实施例的所述方法可以由用于播放全景视频的终端设备来执行，在一个实施例中，该方法可以由一个能够播放全景视频的VR设备来执行。本申请实施例的所述方法包括如下步骤。
S501：在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后。第一时间点和第二时间点之间可以相差一个单位时间，该单位时间可以根据全景视频播放时的视频帧播放频率来确定，例如当视频帧播放频率为30Hz时，单位时间为1/30秒。
在一个实施例中，第二时间点时所对应的第二初始视频帧可以为一个或者多个，如果预先知道用户的运动方向，第二初始视频帧则为在第二时间点播放的全景视频帧中以该运动方向所对应视野朝向角度下部分图像构成的视频帧。例如，如图2a所示，如果预先知道用户会向左转动，则可以认为在第二时间点播放的全景视频中，根据向左运动方向对应的(α3，β3，γ3)视野朝向角度下能够 观看到的部分图像可以得到对应的第二初始视频帧，第二初始视频帧比实际能够观看到的第二播放视频帧的像素范围更大。同样，如果不知道用户的运动方向，则可以同时将如图2a中向左、向右、向上、向下以及静止不变的情况作为预测的运动方向，然后确定5个第二初始视频帧。
S502：生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中目标位置的像素点的颜色值与所述第二初始视频帧中目标位置的像素点的颜色值所确定的颜色值区间。也就是说，在第一时间点和第二时间点之间，生成过渡初始视频帧，这些视频帧使得每一个目标位置处的像素点的变化幅度较小，特别是在第一时间点和第二时间点之间，某个目标位置处的像素点由黑跳变为白色时，基于过渡初始视频帧能够使得在第一时间点和第二时间点之间，该目标位置处能够由黑色逐渐变为白色。
S503：根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。根据所述过渡初始视频帧进行播放控制主要包括对过渡初始视频帧和第二初始视频帧进行处理，得到实际上能够投影到VR设备的显示界面上的过渡播放视频帧和第二播放视频帧，并播放给用户。在一个实施例中可以基于VR设备的FOV，确定出过渡初始视频帧和第二初始视频帧的像素范围，然后提取确定出过渡播放视频帧和第二播放视频帧，并在第一时间点和第二时间之间按照时间顺序播放各个过渡播放视频帧并最终播放第二播放视频帧。
在一个实施例中，在所述S502中生成的过渡初始视频帧可以为过渡初始视频帧数据，通过对这些过渡初始视频帧数据能够生成图像显示在显示界面上，以便于用户观看。
本申请实施例的所述方法中各个步骤的实现可参考上述内容的描述。
再请参见图6，是本申请实施例的另一种视频处理方法的流程示意图，本申请实施例的所述方法可以由用于播放全景视频的终端设备来执行，在一个实施例中，该方法可以由一个能够播放全景视频的VR设备来执行。本申请实施例的所述方法包括如下步骤。
S601：获取运动速度值。该运动速度值可以是一个估计值，可以动态地根据在一段时间内，佩戴VR设备观看全景视频的过程中，用户头部转动的角速度 来确定该运动速度值，例如一段时间内的平均角速度。
S602：根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息。在一个实施例中，根据运动速度值和第一时间点到第二时间点之间的时长，可以计算得到从第一时间点到第二时间点的姿态变化量，例如角度变化量，进而得到第二姿态信息，由于所述运动速度值为一个估计值，因此，该第二姿态信息也是一个估算值，并不是用户实际转动后的姿态信息，是为了方便后续预先计算出过渡初始视频帧，方便最终在播放第二时间点的第二播放视频帧之前播放各个过渡初始视频帧对应的过渡播放视频帧。
S603：获取在第二时间点时所要播放的所述全景视频的全景视频帧。全景视频在录制完成后，每个时间点下的全景视频帧即为已知，基于时间点能够得到全景视频帧，方便后续从全景视频帧中截取出部分图像构成第二初始视频帧和能够在显示界面上播放显示的第二播放视频帧。
S604：根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。所述S601至S604对应于上一实施例中的S501。所述第二姿态信息为一个视野朝向角度信息，根据用户的视野朝向角度，可以从全景视频帧中找到并截取出部分图像构成第二初始视频帧，可参考图3所示。
在一个实施例中，选取的从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽，和/或，从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高；所述能够在显示界面上播放显示的播放视频帧是指：在所述第二姿态信息下，全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。同样参考图2b所示，第二初始视频帧的尺寸大于实际能够在显示界面上显示的第二播放视频帧的尺寸，这样可以确保从第一时间点到第二时间点中间的过渡播放视频帧能够准确得出，并正常播放。
在一个实施例中，所述运动速度值包括至少两个不同方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个方向上关联一个初始视频帧组，每一个初始视频帧组包括一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
S605：生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之 间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中目标位置的像素点的颜色值与所述第二初始视频帧中目标位置的像素点的颜色值所确定的颜色值区间。
在一个实施例中，所述S605可以包括：获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；获取在第二初始视频帧中目标位置的像素点的第二颜色值；根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的目标位置的过渡像素点的颜色值；在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
在一个实施例中，所述根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的目标位置的过渡像素点的颜色值，包括：对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；根据需要生成的过渡初始视频帧的数量，获取每个颜色通道分量的变化量；根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值。在一个实施例中，过渡像素点的颜色值的计算过程可参考图4所对应实施例的描述。
在一个实施例中，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置；或者，在所述第一初始视频帧中目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中该目标位置的像素值的颜色值为白色；或者，在所述第一初始视频帧中目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中该目标位置的像素值的颜色值为黑色。
在一个实施例中，可以确定过渡初始视频帧的数量，以便于基于数量来进行插值处理。确定在第一时间点和第二时间点之间的过渡初始视频帧的数量包括：获取终端的视频帧刷新频率；获取所述全景视频播放时的视频帧播放频率；根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；所述生成的过渡初始视频帧的数量与所述确定的数量相同。在一个实施例中，如果全景视频播放的视频帧播放频率为30Hz，如果如果VR显示屏幕刷新频率是60HZ，则每两个视频帧之间可以确定出1个过渡初始视频帧；如果VR显示屏幕刷新频率是90HZ，则每两个视频帧之间可以确定出2个过渡初始视频帧；如 果VR显示屏幕刷新频率是120HZ，则每两个视频帧之间可以确定出3个过渡初始视频帧。
S606：检测运动方向。可以根据VR设备上的传感器来检测VR设备的运动方向，例如通过陀螺仪来检测VR设备是否是向上、向下、向左、向右的方向转动或者没有运动。
S607：选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组。例如，在确定了用户头部是向左转动时，则可以选择如图2b所示的第二播放视频帧203a和基于所述第二播放视频帧203a计算得到的各个过渡初始视频帧，得到目标初始视频帧组。
S608：根据所述目标初始视频帧组进行播放控制。在一个实施例中，可以按照初始视频帧组中的第二初始视频帧和各个过渡初始视频帧的尺寸，生成能够在VR设备显示界面上显示的第二播放视频帧和各个过渡播放视频帧，并在显示界面上播放各个第二播放视频帧和各个过渡播放视频帧即可。
在一个实施例中，所述S608具体可以包括：获取终端的视场角参数；根据所述视场角参数确定投影到所述终端显示界面上的像素范围；对所述目标初始视频帧组中各个初始视频帧中根据像素范围进行像素坐标匹配，选出需要投影的像素点构成过渡播放视频帧和第二播放视频帧；按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
本申请实施例所述方法的各个步骤的具体实现可参考上述实施例中相关内容的描述。
本申请实施例中，能够预测确定一个第二时间点，并根据第二时间点的视频帧和当前第一时间点的视频帧来生成一些过渡视频帧，这样可以保证第一时间点播放的视频帧与第二时间点播放的视频帧之间的像素颜色变化程度较小，能够有效地降低播放全景视频时对用户的眼镜造成的不良刺激，并且整个过程并不需要用户手动调节VR等设备的播放明亮度，满足了用户自动化、智能化的需求。
在一个实施例中，还提供了一种智能终端，该智能终端的内部结构可如图9所示，该智能终端包括视频处理装置，视频处理装置中包括各个模块，每个模块可全部或部分通过软件、硬件或其组合来实现。
下面在对本申请实施例的一种视频处理装置及智能终端进行详细描述。
请参见图7，是本申请实施例的一种视频处理装置的结构示意图，本申请实施例的所述装置可以设置在进行全景播放的终端上，例如VR终端。所述装置包括如下模块。
确定模块701，用于在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
生成模块702，用于生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
控制模块703，用于根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
在一个实施例中，所述确定模块701，用于获取运动速度值；根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息；获取在第二时间点时所要播放的所述全景视频的全景视频帧；根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。
在一个实施例中，从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽，和/或，从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高；所述能够实际在显示界面上播放显示的播放视频帧是指：全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。
在一个实施例中，所述运动速度值包括至少两个不同运动方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个运动方向上关联一个初始视频帧组，每一个初始视频帧组包括一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
在一个实施例中，所述生成模块702，还用于获取终端的视频帧刷新频率；获取所述全景视频播放时的视频帧播放频率；根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；所述生成的过渡初始视频帧的数量与所述确定的数量相同。
在一个实施例中，所述生成模块702，在用于生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧时，用于获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；获取在第二初始视频帧中所述目标位置的像素点的第二颜色值；根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值；在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
在一个实施例中，所述生成模块702，在用于根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值时，用于对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；根据需要生成的过渡初始视频帧的数量，获取每个颜色通道分量的变化量；根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值。
在一个实施例中，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置；或者，在所述第一初始视频帧中所述目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为白色；或者，在所述第一初始视频帧中所述目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为黑色。
在一个实施例中，所述控制模块703，用于检测运动方向；选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组；根据所述目标初始视频帧组进行播放控制。
在一个实施例中，所述控制模块703，在用于根据所述目标初始视频帧组进行播放控制时，用于获取终端的视场角参数；根据所述视场角参数确定投影到所述终端显示界面上的像素范围；对所述目标初始视频帧组中各个初始视频帧根据像素范围进行像素坐标匹配，构成过渡播放视频帧和第二播放视频帧；按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
本申请实施例中，所述装置中各个功能模块的实现可参考上述各个实施例中相关内容的描述。
本申请实施例中，能够预测确定一个第二时间点，并根据第二时间点的视频帧和当前第一时间点的视频帧来生成一些过渡视频帧，这样可以保证第一时间点播放的视频帧与第二时间点播放的视频帧之间的像素颜色变化程度较小，可有效降低播放全景视频时对用户的眼镜造成的不良刺激，并且整个过程并不需要用户手动调节VR等设备的播放明亮度，满足了用户自动化、智能化的需求
再请参见图8，是本申请实施例的一种智能终端的结构示意图，本申请实施例的所述终端可以为能够播放全景视频的终端，例如可以为VR设备。所述智能终端包括供电模块等结构，在本申请实施例中还包括处理器801、存储装置802以及数据接口803。所述处理器801、存储装置802以及数据接口803之间可以通过总线等方式建立数据连接。
所述数据接口803用于接收外部数据，并将外部数据发送给处理器801或者存储到所述存储装置802中。所述存储装置802可以包括易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)；存储装置802也可以包括非易失性存储器(non-volatile memory)，例如快闪存储器(flash memory)，固态硬盘(solid-state drive，SSD)等；存储装置802还可以包括上述种类的存储器的组合。
所述处理器801可以是中央处理器801(central processing unit，CPU)。所述处理器801还可以进一步包括硬件芯片。所述硬件芯片可以是专用集成电路(application-specific integrated circuit，ASIC)，可编程逻辑器件(programmable logic device，PLD)等。上述PLD可以是现场可编程逻辑门阵列(field-programmable gate array，FPGA)，通用阵列逻辑(generic array logic，GAL)等。
可选地，所述存储装置802还用于存储程序指令。所述处理器801可以调用所述程序指令，实现本申请实施例中上述提及的方法。
在一个实施例中，所述处理器801，调用所述程序指令，用于在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区 间；根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
在一个实施例中，所述处理器801，在用于确定第二时间点的第二初始视频帧时，用于获取运动速度值；根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息；获取在第二时间点时所要播放的所述全景视频的全景视频帧；根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。
在一个实施例中，从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽，和/或，从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高；所述能够实际在显示界面上播放显示的播放视频帧是指：全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。
在一个实施例中，所述运动速度值包括至少两个不同运动方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个运动方向上关联一个初始视频帧组，每一个初始视频帧组包括一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
在一个实施例中，所述处理器801，还用于获取终端的视频帧刷新频率；获取所述全景视频播放时的视频帧播放频率；根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；所述生成的过渡初始视频帧的数量与所述确定的数量相同。
在一个实施例中，所述处理器801，在用于生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧时，用于获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；获取在第二初始视频帧中所述目标位置的像素点的第二颜色值；根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值；在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
在一个实施例中，所述处理器801，在用于根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值时，用于对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；根据需要生成的过渡初始视频帧的 数量，获取每个颜色通道分量的变化量；根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值。
在一个实施例中，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置；或者，在所述第一初始视频帧中所述目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为白色；或者，在所述第一初始视频帧中所述目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为黑色。
在一个实施例中，所述处理器801，在用于根据所述过渡初始视频帧进行播放控制时，用于检测运动方向；选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组；根据所述目标初始视频帧组进行播放控制。
在一个实施例中，所述处理器801，在用于根据所述目标初始视频帧组进行播放控制时，用于获取终端的视场角参数；根据所述视场角参数确定投影到所述终端显示界面上的像素范围；对所述目标初始视频帧组中各个初始视频帧根据像素范围进行像素坐标匹配，构成过渡播放视频帧和第二播放视频帧；按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
本申请实施例中，所述处理器801各个功能的实现可参考上述各个实施例中相关内容的描述。
本申请实施例中，能够预测确定一个第二时间点，并根据第二时间点的视频帧和当前第一时间点的视频帧来生成一些过渡视频帧，这样可以保证第一时间点播放的视频帧与第二时间点播放的视频帧之间的像素颜色变化程度较小，可有效降低播放全景视频时对用户的眼镜造成的不良刺激，并且整个过程并不需要用户手动调节VR等设备的播放明亮度，满足了用户自动化、智能化的需求。
在一个实施例中，如图9所示，提供了一种智能终端的内部结构图。该终端包括通过系统总线连接的处理器、存储器、网络接口、输入装置和显示屏。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质存储有操作系统，还可存储有计算机可读指令，该计算机可读指令被处理器执行时，可使得处理器实现视频处理方法。该内存储器中也可储存有计算 机可读指令，该计算机可读指令被处理器执行时，可使得处理器执行视频处理方法。计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。本领域技术人员可以理解，图9中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的服务器的限定，具体的服务器可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
应该理解的是，虽然本申请各实施例中的各个步骤并不是必然按照步骤标号指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，各实施例中至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上所揭露的仅为本申请的部分实施例而已，当然不能以此来限定本申请之权利范围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本申请权利要求所作的等同变化，仍属于申请所涵盖的范围。

Claims (39)

  1. 一种视频处理方法，其特征在于，包括：
    终端在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
    所述终端生成从第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
    所述终端根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
  2. 根据权利要求1所述的方法，其特征在于，所述确定第二时间点的第二初始视频帧，包括：
    所述终端获取运动速度值；
    所述终端根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息；
    所述终端获取在第二时间点时所要播放的所述全景视频的全景视频帧；
    所述终端根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。
  3. 根据权利要求2所述的方法，其特征在于，从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽；
    所述能够实际在显示界面上播放显示的播放视频帧是指：全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高。
  5. 根据权利要求2所述的方法，其特征在于，所述运动速度值包括至少两个不同运动方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个运动方向上关联一个初始视频帧组，每一个初始视频帧组包括一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
  6. 根据权利要求1所述的方法，其特征在于，所述生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧之前，还包括：
    所述终端获取终端的视频帧刷新频率；
    所述终端获取所述全景视频播放时的视频帧播放频率；
    所述终端根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；
    所述生成的过渡初始视频帧的数量与所述确定的数量相同。
  7. 根据权利要求1所述的方法，其特征在于，所述生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，包括：
    所述终端获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；
    所述终端获取在第二初始视频帧中所述目标位置的像素点的第二颜色值；
    所述终端根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值；
    所述终端在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
  8. 根据权利要求7所述的方法，其特征在于，所述根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值，包括：
    所述终端对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；
    所述终端根据需要生成的过渡初始视频帧的数量，获取每个颜色通道分量 的变化量；
    所述终端根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值。
  9. 根据权利要求7或8所述的方法，其特征在于，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置。
  10. 根据权利要求7或8所述的方法，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为白色。
  11. 根据权利要求7或8所述的方法，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为黑色。
  12. 根据权利要求5所述的方法，其特征在于，所述根据所述过渡初始视频帧进行播放控制，包括：
    所述终端检测运动方向；
    所述终端选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组；
    所述终端根据所述目标初始视频帧组进行播放控制。
  13. 根据权利要求12所述的方法，其特征在于，所述根据所述目标初始视频帧组进行播放控制，包括：
    所述终端获取终端的视场角参数；
    所述终端根据所述视场角参数确定投影到所述终端显示界面上的像素范围；
    所述终端对所述目标初始视频帧组中各个初始视频帧根据像素范围进行像素坐标匹配，构成过渡播放视频帧和第二播放视频帧；
    所述终端按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
  14. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可 读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行如下步骤：
    在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
    生成从第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
    根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
  15. 根据权利要求14所述的计算机设备，其特征在于，所述处理器执行的确定第二时间点的第二初始视频帧，包括：
    获取运动速度值；
    根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息；
    获取在第二时间点时所要播放的所述全景视频的全景视频帧；
    根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。
  16. 根据权利要求15所述的计算机设备，其特征在于，所述处理器执行的从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽；
    所述能够实际在显示界面上播放显示的播放视频帧是指：全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。
  17. 根据权利要求16所述的计算机设备，其特征在于，所述计算机可读指令还使得处理器执行以下步骤：从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高。
  18. 根据权利要求15所述的计算机设备，其特征在于，所述运动速度值包 括至少两个不同运动方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个运动方向上关联一个初始视频帧组，每一个初始视频帧组包括一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
  19. 根据权利要求14所述的计算机设备，其特征在于，所述处理器执行的生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧之前，所述计算机可读指令还使得所述处理器执行如下步骤：
    获取终端的视频帧刷新频率；
    获取所述全景视频播放时的视频帧播放频率；
    根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；
    所述生成的过渡初始视频帧的数量与所述确定的数量相同。
  20. 根据权利要求14所述的计算机设备，其特征在于，所述处理器执行的生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，包括：
    获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；
    获取在第二初始视频帧中所述目标位置的像素点的第二颜色值；
    根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值；
    在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
  21. 根据权利要求20所述的计算机设备，其特征在于，所述处理器执行的根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值，包括：
    对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；
    根据需要生成的过渡初始视频帧的数量，获取每个颜色通道分量的变化量；
    根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡 像素点的颜色值。
  22. 根据权利要求20或21所述的计算机设备，其特征在于，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置。
  23. 根据权利要求20或21所述的计算机设备，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为白色。
  24. 根据权利要求20或21所述的计算机设备，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为黑色。
  25. 根据权利要求18所述的计算机设备，其特征在于，所述处理器执行的根据所述过渡初始视频帧进行播放控制，包括：
    检测运动方向；
    选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组；
    根据所述目标初始视频帧组进行播放控制。
  26. 根据权利要求25所述的计算机设备，其特征在于，所述处理器执行的根据所述目标初始视频帧组进行播放控制，包括：
    获取终端的视场角参数；
    根据所述视场角参数确定投影到所述终端显示界面上的像素范围；
    对所述目标初始视频帧组中各个初始视频帧根据像素范围进行像素坐标匹配，构成过渡播放视频帧和第二播放视频帧；
    按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
  27. 一个或多个存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行如下步骤：
    在播放全景视频时，确定第二时间点的第二初始视频帧，所述第二时间点在第一时间点之后；
    生成从第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，所述过渡初始视频帧中目标位置的像素点的颜色值位于目标区间内，所述目标区间是指根据所述第一初始视频帧中所述目标位置的像素点的颜色值与所述第二初始视频帧中所述目标位置的像素点的颜色值所确定的颜色值区间；
    根据所述过渡初始视频帧进行播放控制，以完成所述全景视频的播放。
  28. 根据权利要求27所述的存储介质，其特征在于，所述处理器执行的确定第二时间点的第二初始视频帧，包括：
    获取运动速度值；
    根据第一时间点的第一姿态信息，并根据所述运动速度值和所述第一时间点与第二时间点之间的时长，得到在第二时间点的第二姿态信息；
    获取在第二时间点时所要播放的所述全景视频的全景视频帧；
    根据所述第二姿态信息从所述全景视频帧中获取第二初始视频帧。
  29. 根据权利要求28所述的存储介质，其特征在于，所述处理器执行的从所述全景视频帧中获取第二初始视频帧的帧宽大于能够实际在显示界面上播放显示的播放视频帧的帧宽；
    所述能够实际在显示界面上播放显示的播放视频帧是指：全景视频帧中投影到终端显示界面上的部分图像构成的视频帧。
  30. 根据权利要求29所述的存储介质，其特征在于，所述计算机可读指令还使得处理器执行以下步骤：从所述全景视频帧中获取第二初始视频帧的帧高大于能够实际在显示界面上播放显示的播放视频帧的帧高。
  31. 根据权利要求28所述的存储介质，其特征在于，所述运动速度值包括至少两个不同运动方向上的转动角速度值，所述第二姿态信息是指视野朝向角度信息；每一个运动方向上关联一个初始视频帧组，每一个初始视频帧组包括 一个第二初始视频帧及根据该第二初始视频帧得到的过渡初始视频帧。
  32. 根据权利要求27所述的存储介质，其特征在于，所述处理器执行的生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧之前，所述计算机可读指令还使得所述处理器执行如下步骤：
    获取终端的视频帧刷新频率；
    获取所述全景视频播放时的视频帧播放频率；
    根据所述视频帧刷新频率和视频帧播放频率确定过渡初始视频帧的数量；
    所述生成的过渡初始视频帧的数量与所述确定的数量相同。
  33. 根据权利要求27所述的存储介质，其特征在于，所述处理器执行的生成从所述第一时间点的第一初始视频帧到所述第二初始视频帧之间的过渡初始视频帧，包括：
    获取在第一时间点的第一初始视频帧中目标位置的像素点的第一颜色值；
    获取在第二初始视频帧中所述目标位置的像素点的第二颜色值；
    根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值；
    在得到多个过渡像素点的颜色值后，根据得到的所有过渡像素点的颜色值，生成过渡初始视频帧。
  34. 根据权利要求33所述的存储介质，其特征在于，所述处理器执行的根据第一颜色值和第二颜色值进行插值处理，得到在过渡初始视频帧的所述目标位置的过渡像素点的颜色值，包括：
    对第一颜色值中的颜色通道分量和第二颜色值中的颜色通道分量进行差值计算，得到各个颜色通道分量的分量差值；
    根据需要生成的过渡初始视频帧的数量，获取每个颜色通道分量的变化量；
    根据第一颜色值的各个颜色通道分量、需要生成的过渡初始视频帧的数量以及每个颜色通道分量的变化量，得到各个过渡初始视频帧的目标位置的过渡像素点的颜色值。
  35. 根据权利要求33或34所述的存储介质，其特征在于，所述第一初始视频帧和第二初始视频帧的所有像素点的像素位置均分别作为所述目标位置。
  36. 根据权利要求33或34所述的存储介质，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为黑色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为白色。
  37. 根据权利要求33或34所述的存储介质，其特征在于，在所述第一初始视频帧中所述目标位置的像素点的颜色值为白色、且在所述第二初始视频帧中所述目标位置的像素值的颜色值为黑色。
  38. 根据权利要求31所述的存储介质，其特征在于，所述处理器执行的根据所述过渡初始视频帧进行播放控制，包括：
    检测运动方向；
    选择与该运动方向关联的初始视频帧组，得到目标初始视频帧组；
    根据所述目标初始视频帧组进行播放控制。
  39. 根据权利要求38所述的存储介质，其特征在于，所述处理器执行的根据所述目标初始视频帧组进行播放控制，包括：
    获取终端的视场角参数；
    根据所述视场角参数确定投影到所述终端显示界面上的像素范围；
    对所述目标初始视频帧组中各个初始视频帧根据像素范围进行像素坐标匹配，构成过渡播放视频帧和第二播放视频帧；
    按照时间顺序依次播放过渡播放视频帧和第二播放视频帧。
PCT/CN2018/111971 2017-10-27 2018-10-25 视频处理方法、计算机设备和存储介质 Ceased WO2019080913A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/590,460 US11032510B2 (en) 2017-10-27 2019-10-02 Video processing method, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711021898.0 2017-10-27
CN201711021898.0A CN109729365B (zh) 2017-10-27 2017-10-27 一种视频处理方法、装置及智能终端、存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/590,460 Continuation US11032510B2 (en) 2017-10-27 2019-10-02 Video processing method, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2019080913A1 true WO2019080913A1 (zh) 2019-05-02

Family

ID=66246786

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/111971 Ceased WO2019080913A1 (zh) 2017-10-27 2018-10-25 视频处理方法、计算机设备和存储介质

Country Status (3)

Country Link
US (1) US11032510B2 (zh)
CN (1) CN109729365B (zh)
WO (1) WO2019080913A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112333513A (zh) * 2020-07-09 2021-02-05 深圳Tcl新技术有限公司 竖屏视频库的创建方法、设备及存储介质
CN115348398A (zh) * 2021-05-13 2022-11-15 影石创新科技股份有限公司 视频处理方法、装置、计算机设备和存储介质
WO2023065832A1 (zh) * 2021-10-18 2023-04-27 华为技术有限公司 视频的制作方法和电子设备
CN112866669B (zh) * 2021-01-15 2023-09-15 聚好看科技股份有限公司 一种数据切换时间确定方法及设备

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11461942B2 (en) * 2018-12-21 2022-10-04 Koninklijke Kpn N.V. Generating and signaling transition between panoramic images
US12086221B2 (en) * 2019-02-01 2024-09-10 Sony Group Corporation Multi-factor authentication for virtual reality
US20220174258A1 (en) * 2019-03-13 2022-06-02 Sony Group Corporation Information processing device, information processing method, and program
US11481879B2 (en) * 2019-06-26 2022-10-25 Dell Products L.P. Method for reducing visual fatigue and system therefor
CN110290360B (zh) * 2019-08-01 2021-02-23 浙江开奇科技有限公司 用于全景视频影像的影像拼接方法及终端设备
US11301035B1 (en) * 2019-09-27 2022-04-12 Apple Inc. Method and device for video presentation
CN110913278B (zh) * 2019-12-06 2022-04-08 深圳创维新世界科技有限公司 视频播放方法、显示终端及存储介质
CN111080801A (zh) * 2019-12-26 2020-04-28 北京像素软件科技股份有限公司 虚拟模型显示方法及装置
CN112312161A (zh) * 2020-06-29 2021-02-02 北京沃东天骏信息技术有限公司 用于生成视频的方法、装置、电子设备及可读存储介质
CN113242465B (zh) * 2021-04-27 2022-08-16 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备及可读存储介质
US12348702B2 (en) 2022-02-18 2025-07-01 Canon Kabushiki Kaisha Electronic device and method for controlling electronic device
CN114390222B (zh) * 2022-03-24 2022-07-08 北京唱吧科技股份有限公司 适用于180度全景视频的切换方法、装置及存储介质
CN120803280B (zh) * 2025-09-15 2025-11-25 成都市技师学院(成都工贸职业技术学院、成都市高级技工学校、成都铁路工程学校) 一种基于人工智能的场景vr交互方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080291326A1 (en) * 2007-05-21 2008-11-27 Victor Company Of Japan, Limited Apparatus for and method of displaying video signals
CN102932580A (zh) * 2012-11-19 2013-02-13 天津大学 针对逐行扫描摄录方式的视频图像校准方法及装置
CN105184738A (zh) * 2015-09-08 2015-12-23 郑州普天信息技术有限公司 一种三维虚拟展示装置与方法
CN106375748A (zh) * 2016-09-07 2017-02-01 深圳超多维科技有限公司 立体虚拟现实全景视图拼接方法、装置及电子设备

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100530223B1 (ko) * 2003-05-13 2005-11-22 삼성전자주식회사 프레임 레이트 변환시의 프레임 보간 방법 및 그 장치
JP4181598B2 (ja) * 2006-12-22 2008-11-19 シャープ株式会社 画像表示装置及び方法、画像処理装置及び方法
US9042682B2 (en) * 2012-05-23 2015-05-26 Dolby Laboratories Licensing Corporation Content creation using interpolation between content versions
US9645395B2 (en) * 2013-03-15 2017-05-09 Mark Bolas Dynamic field of view throttling as a means of improving user experience in head mounted virtual environments
US10726560B2 (en) * 2014-10-31 2020-07-28 Fyusion, Inc. Real-time mobile device capture and generation of art-styled AR/VR content
US10262426B2 (en) * 2014-10-31 2019-04-16 Fyusion, Inc. System and method for infinite smoothing of image sequences
US9615187B2 (en) * 2014-12-18 2017-04-04 Ross Video Limited Audio signal verification for video/audio production equipment
US9473758B1 (en) * 2015-12-06 2016-10-18 Sliver VR Technologies, Inc. Methods and systems for game video recording and virtual reality replay
CN105898138A (zh) * 2015-12-18 2016-08-24 乐视致新电子科技（天津）有限公司 全景视频播放方法及装置
US10003768B2 (en) * 2016-09-28 2018-06-19 Gopro, Inc. Apparatus and methods for frame interpolation based on spatial considerations
US20180158246A1 (en) * 2016-12-07 2018-06-07 Intel IP Corporation Method and system of providing user facial displays in virtual or augmented reality for face occluding head mounted displays

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080291326A1 (en) * 2007-05-21 2008-11-27 Victor Company Of Japan, Limited Apparatus for and method of displaying video signals
CN102932580A (zh) * 2012-11-19 2013-02-13 天津大学 针对逐行扫描摄录方式的视频图像校准方法及装置
CN105184738A (zh) * 2015-09-08 2015-12-23 郑州普天信息技术有限公司 一种三维虚拟展示装置与方法
CN106375748A (zh) * 2016-09-07 2017-02-01 深圳超多维科技有限公司 立体虚拟现实全景视图拼接方法、装置及电子设备

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112333513A (zh) * 2020-07-09 2021-02-05 深圳Tcl新技术有限公司 竖屏视频库的创建方法、设备及存储介质
CN112333513B (zh) * 2020-07-09 2024-02-06 深圳Tcl新技术有限公司 竖屏视频库的创建方法、设备及存储介质
CN112866669B (zh) * 2021-01-15 2023-09-15 聚好看科技股份有限公司 一种数据切换时间确定方法及设备
CN115348398A (zh) * 2021-05-13 2022-11-15 影石创新科技股份有限公司 视频处理方法、装置、计算机设备和存储介质
WO2023065832A1 (zh) * 2021-10-18 2023-04-27 华为技术有限公司 视频的制作方法和电子设备

Also Published As

Publication number Publication date
CN109729365A (zh) 2019-05-07
CN109729365B (zh) 2021-03-26
US20200036937A1 (en) 2020-01-30
US11032510B2 (en) 2021-06-08

Similar Documents

Publication Publication Date Title
WO2019080913A1 (zh) 视频处理方法、计算机设备和存储介质
US8330793B2 (en) Video conference
US10694146B2 (en) Video capture systems and methods
US8711198B2 (en) Video conference
US20200111256A1 (en) Real-world anchor in a virtual-reality environment
US9928655B1 (en) Predictive rendering of augmented reality content to overlay physical structures
CN110166758B (zh) 图像处理方法、装置、终端设备及存储介质
US9892565B2 (en) Reprojection OLED display for augmented reality experiences
US12586294B2 (en) Systems and methods for generating stabilized images of a real environment in artificial reality
CN104349155B (zh) 模拟立体图像显示方法及显示设备
US10755671B2 (en) Device, method, and program for controlling displaying of survey image
US11317072B2 (en) Display apparatus and server, and control methods thereof
US20170078570A1 (en) Image processing device, image processing method, and image processing program
EP3349095B1 (en) Method, device, and terminal for displaying panoramic visual content
US20230125961A1 (en) Systems and methods for displaying stereoscopic rendered image data captured from multiple perspectives
EP4451092A1 (en) Gaze based auto exposure control algorithm
JP2015231114A (ja) 映像表示装置
JP2016511587A (ja) ３ｄビジュアルコンテンツの自動評価のための技術
US12547244B2 (en) Gaze-driven autofocus camera for mixed-reality passthrough
CN114495827B (zh) 显示控制方法、显示装置及存储介质
US12423916B1 (en) Eye-tracking assisted passthrough rendering
CN119987565B (zh) 一种头戴显示设备的成像检测方法及系统
US20250247516A1 (en) Electronic device and information display system
CN118741314A (zh) 混合现实透传用的注视驱动的自动对焦摄像头

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18869529

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18869529

Country of ref document: EP

Kind code of ref document: A1