WO2020140784A1 - 视频处理方法、视频处理控制装置、显示控制装置 - Google Patents

视频处理方法、视频处理控制装置、显示控制装置 Download PDF

Info

Publication number
WO2020140784A1
WO2020140784A1 PCT/CN2019/127423 CN2019127423W WO2020140784A1 WO 2020140784 A1 WO2020140784 A1 WO 2020140784A1 CN 2019127423 W CN2019127423 W CN 2019127423W WO 2020140784 A1 WO2020140784 A1 WO 2020140784A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
motion
video
processing method
threshold
Prior art date
Application number
PCT/CN2019/127423
Other languages
English (en)
French (fr)
Inventor
王亚坤
林琳
孙剑
邵继洋
郭子强
訾峰
孙宾华
刘炳鑫
丁亚东
Original Assignee
京东方科技集团股份有限公司
北京京东方光电科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司, 北京京东方光电科技有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US16/973,299 priority Critical patent/US11317054B2/en
Publication of WO2020140784A1 publication Critical patent/WO2020140784A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G09G2340/0435—Change or adaptation of the frame rate of the video stream

Definitions

  • the present disclosure relates to the field of display technology, and in particular, to a video processing method, a video processing control device, a display control device, and a display device.
  • Virtual reality (Virtual Reality, VR) system is a computer simulation system that can create and experience a virtual world.
  • the image delay will affect the quality of the displayed image. For example, for a moving image (video), blurring may occur.
  • a video processing method including: receiving motion information of a display device; judging from the motion information of the display device and its motion threshold, whether the motion state of the display device is influencing whether the display device adopts The effect of processing the video displayed on the display device by a frame rate up-conversion processing method based on motion compensation; if so, stop using the frame rate up-conversion processing method based on motion compensation to process the video displayed by the display device; If not, continue to use the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device.
  • the motion information of the display device includes X-axis acceleration Accel_X, Y-axis acceleration Accel_Y, and Z-axis acceleration Accel_Z in a three-dimensional coordinate system of the display device;
  • the motion threshold includes an acceleration threshold Accel_Threshol;
  • the effects of video processing include:
  • the receiving motion information of the display device includes: receiving m pieces of motion information of the display device one by one within a first preset time period, m is an integer greater than or equal to 2; according to the Motion information and motion threshold of the display device to determine whether the motion state of the display device affects the effect of the display device using the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device ,Also includes:
  • each of the m motion information of the display device and the motion threshold determine whether each of the m motion states of the display device affects the display device based on Motion compensation frame rate up-conversion processing method for processing the video displayed by the display device; if so, confirm that the motion state of the display device affects the display device to use motion compensation-based frame rate up-conversion processing The effect of processing the video displayed by the display device in a manner; if not, confirm that the motion state of the display device does not affect the display device adopting a frame rate up conversion processing method based on motion compensation to the display device The processing effect of the displayed video.
  • the video processing method further includes determining whether the motion state of the display device is in accordance with the motion information of the display device and its motion threshold, whether it affects the display device to adopt a motion compensation-based Before the effect of processing the video displayed on the display device by the frame rate up conversion processing method,
  • the video includes multiple frames of image data; determine whether consecutive k frames of image data in the multiple frames of image data are consistent, k is an integer greater than or equal to 2; if so, stop Use the frame rate up-conversion processing method based on motion compensation to process the video displayed by the display device; if not, determine the motion state of the display device based on the motion information of the display device and its motion threshold, Whether it affects the effect that the display device uses the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device.
  • the video processing method further includes:
  • the motion information of the display device includes X-axis angular velocity Gyro_X, Y-axis angular velocity Gyro_Y, and Z-axis angular velocity Gyro_Z of the display device in a three-dimensional coordinate system;
  • the rest threshold includes an angular velocity threshold GYRO_Threshold;
  • the judging whether the display device is stationary or nearly stationary based on the motion information of the display device and its static threshold includes: determining whether the motion information of the display device and the static threshold meet:
  • the receiving motion information of the display device includes: receiving n pieces of motion information of the display device one by one within a second preset time period, where n is an integer greater than or equal to 2. Judging whether the display device is stationary or nearly stationary according to the motion information of the display device and its static threshold, and further includes: according to each of the n motion information of the display device and the static threshold, one by one Judging whether each of the n states in which the display device is located is still or nearly stationary; if so, confirm that the display device is stationary or nearly stationary; if not, confirm that the display device is in a moving state.
  • a video processing control device including: a receiving module, a processing module and a control module.
  • the receiving module is configured to receive motion information of the display device.
  • a processing module is coupled to the receiving module, and the processing module is configured to determine the motion state of the display device based on the motion information of the display device and its motion threshold, whether it affects the display device's use of motion-based
  • the compensated frame rate up-conversion processing method effects the processing of the video displayed by the display device, and outputs the judgment result.
  • the control module is coupled to the processing module, and the control module is configured to: according to the judgment result, the motion state of the display device affects the display device to adopt a frame rate up conversion processing method based on motion compensation In the case of the effect of processing the video displayed by the display device, output the first control signal that stops processing the video displayed by the display device using the frame rate up-conversion processing method based on motion compensation; When the motion state of the display device does not affect the effect of the display device using the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device, the output continues to use the frame rate based on motion compensation A second control signal that processes the video displayed by the display device in an up-conversion processing manner.
  • the motion information of the display device includes X-axis acceleration Accel_X, Y-axis acceleration Accel_Y, and Z-axis acceleration Accel_Z in a three-dimensional coordinate system of the display device; the motion threshold includes an acceleration threshold Accel_Threshol. d
  • the processing module is configured to determine whether the motion information of the display device and the motion threshold satisfy:
  • the receiving module is configured to receive m pieces of motion information of the display device one by one within a first preset time period, where m is an integer greater than or equal to 2.
  • the processing module is configured to determine whether each of the m motion states of the display device is one by one according to each of the m motion information of the display device and the motion threshold Both affect the display device's effect of processing the video displayed by the display device using a frame rate up conversion processing method based on motion compensation. If yes, confirm that the motion state of the display device affects the effect of the display device using the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device. If not, confirm that the motion state of the display device does not affect the display device's effect of processing the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation.
  • the receiving module is further configured to receive the video displayed by the display device, the video including multiple frames of image data.
  • the processing module is further configured to determine whether the motion state of the display device is in accordance with the motion information and the motion threshold of the display device, whether it affects the frame rate up conversion of the display device based on motion compensation Processing method Before processing the effect of the video displayed on the display device, determine whether the continuous k frames of image data in the multi-frame image data are all consistent, k is an integer greater than or equal to 2; in the continuous k If the frame image data does not remain consistent, determine whether the display device is in a motion state based on the motion information and the motion threshold of the display device, and whether it affects the display device to use frame rate up-conversion processing based on motion compensation The effect of processing the video displayed on the display device.
  • the control module is further configured to: when the continuous k-frame image data remains consistent, the output stops processing the video displayed on the display device using a frame rate up-conversion processing method based on motion compensation One control signal.
  • the processing module is further configured to, when the continuous k frames of image data do not remain consistent, execute the motion of the display device according to the motion information and the motion threshold of the display device Whether the state affects the display device's effect of processing the video displayed by the display device using the frame rate up-conversion processing method based on motion compensation, the display is determined based on the motion information of the display device and its still threshold Whether the device is stationary or nearly stationary; when the display device is in a motion state, performing a judgment based on the motion information and motion threshold of the display device to determine whether the motion state of the display device affects the display device is based on motion compensation The effect of processing the video displayed on the display device by the frame rate up conversion processing method.
  • the control module is further configured to: when the display device is stationary or relatively still, output the first control to stop processing the video displayed by the display device by using a frame rate up-conversion processing method based on motion compensation signal.
  • the motion information of the display device includes the X-axis angular velocity Gyro_X, the Y-axis angular velocity Gyro_Y, and the Z-axis angular velocity Gyro_Z in the three-dimensional coordinate system of the display device;
  • the relative rest threshold includes the angular velocity threshold GYRO_Threshold;
  • the processing module is configured to determine whether the motion information of the display device and the rest threshold are satisfied:
  • the receiving module is configured to receive n pieces of motion information of the display device one by one within a second preset time period, where n is an integer greater than or equal to 2.
  • the processing module is configured to determine whether each of the n states in which the display device is located is static or approximate according to each of the n motion information of the display device and a still threshold Still; if it is, confirm that the display device is still or nearly stationary; if not, confirm that the display device is in a moving state.
  • a display control device including: a video processor and a video processing control device as provided by the present disclosure.
  • the video processor is configured to process the video displayed by the display device using a frame rate up conversion processing method based on motion compensation.
  • the control module included in the video processing control device is coupled to the video processor and outputs a control signal to the video processor.
  • a display device including: a motion sensing unit and the display control device described above, the display control device being coupled to the motion sensing unit.
  • a computer-readable storage medium stores computer program instructions, which when executed on a processor, causes the processor to perform one or more steps in the video processing method described in any of the foregoing embodiments .
  • a computer program product includes computer program instructions.
  • the computer program instructions When the computer program instructions are executed on a computer, the computer program instructions cause the computer to perform one or more steps in the video processing method described in any one of the foregoing embodiments.
  • a computer program When the computer program is executed on a computer, the computer program causes the computer to perform one or more steps in the video processing method described in any of the above embodiments.
  • FIG. 1 is a flowchart of a video processing method according to some embodiments
  • FIG. 2 is another flowchart of a video processing method according to some embodiments.
  • FIG. 3 is another flowchart of a video processing method according to some embodiments.
  • FIG. 4 is a control timing diagram of a video processing method according to some embodiments.
  • FIG. 5 is another control timing diagram of the video processing method according to some embodiments.
  • FIG. 6 is another flowchart of a video processing method according to some embodiments.
  • FIG. 7 is another flowchart of a video processing method according to some embodiments.
  • FIG. 8 is a control timing diagram of still another video processing method according to some embodiments.
  • FIG. 9 is a structural diagram of a video processing control device according to some embodiments.
  • FIG. 10 is a structural diagram of a display device according to some embodiments.
  • FIG. 11 is another structural diagram of a display device according to some embodiments.
  • FIG. 12 is still another structural diagram of a display device according to some embodiments.
  • FIG. 13 is a structural diagram of a video processing terminal according to some embodiments.
  • Coupled and “connected” and their derivatives may be used.
  • some embodiments may be described using the term “connected” to indicate that two or more components are in direct physical or electrical contact with each other.
  • the term “coupled” may be used when describing some embodiments to indicate that two or more components have direct physical or electrical contact.
  • the term “coupled” or “communicatively coupled” may also mean that two or more components do not directly contact each other, but still cooperate or interact with each other.
  • the embodiments disclosed herein are not necessarily limited to the content herein.
  • the user as a participant can experience and interact with the virtual world through the virtual reality device. While the user is moving or performing actions, the virtual reality device will also perform corresponding movements.
  • the computer can immediately perform complex calculations and transmit accurate 3D world images back to produce a sense of presence.
  • the video displayed by the virtual reality device is related to the motion information status of the virtual reality device, and due to the image delay problem in the virtual reality system, the video frames displayed by the virtual reality device will be compared to the current virtual reality device. The motion state at the moment appears later, which affects the quality of the displayed video, for example, the displayed video will appear blurred.
  • the method of directly increasing the screen refresh rate is limited by the hardware of the display device, the image delay problem cannot be effectively solved.
  • the existing virtual reality system adopts the method of increasing the screen refresh rate to reduce the image delay, but the improvement of the screen refresh rate is limited by the rendering ability of the graphics processor, and it is difficult to effectively solve the problem of poor display quality caused by the image delay problem. .
  • the virtual reality device adopts a frame rate up conversion (FRUC) image processing method based on motion compensation to process video information to improve the video blurring caused by image delay, thereby effectively improving the displayed The quality of the video.
  • FRUC frame rate up conversion
  • the frame rate up-conversion image processing method based on motion compensation is to use a dynamic image system, adding a frame of motion compensation frame between the traditional two frames of images to achieve the purpose of improving the screen refresh rate, such as the refresh rate of the virtual reality device Increased from the original 50/60Hz to 100/120Hz.
  • the frame rate up-conversion image processing method based on motion compensation mainly includes a motion estimation (Motion Estimate, ME) step and a compensation difference (Motion Compensation, MC) step, where the motion estimation step includes estimating objects between adjacent two frames of images
  • the motion trajectory of the interpolation step includes obtaining a frame to be inserted between the two adjacent frames of images according to the information of the adjacent two frames of images and the estimated motion trajectory of the object between the two adjacent frames of images Image information.
  • the frame rate up-conversion image processing method based on motion compensation will fail, and if the frame rate up-conversion video processing based on motion compensation is still used Processing video in a way will not only increase the data processing burden of the video processor, but also cause the display of the virtual reality device to display video that is more serious than video blur due to the failure of the image processing method.
  • the displayed video does not correspond to the actual movement state of the user, causing physical discomfort, such as dizziness.
  • the inventors of the present disclosure have found through research that, because the video displayed by the virtual reality device corresponds to the motion information state of the virtual reality device, the motion trajectory of objects between two adjacent images in the video displayed by the virtual reality device cannot In the case of estimation, the motion state of the virtual reality device is described, which affects the effect of processing video using the frame rate up-conversion video processing method based on motion compensation, resulting in the failure of the image processing method.
  • the motion state of the virtual reality device is described, which affects the effect of processing video using the frame rate up-conversion video processing method based on motion compensation, resulting in the failure of the image processing method.
  • a virtual reality device as a VR headset for example, when the user wears the VR headset in a state of high-speed shaking, etc., the motion trajectory of objects between two adjacent frames of the video displayed in the VR headset cannot It is estimated that the frame rate up conversion video processing method based on motion compensation is invalid.
  • some embodiments of the present disclosure provide a video processing method, including:
  • the motion information of the display device may be motion information such as acceleration, angular velocity, and the like of the display device.
  • S400 Determine, based on the motion information of the display device and its motion threshold, whether the motion state of the display device affects the display device's effect of processing the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation .
  • the motion state that affects the processing effect of FRUC is called a complex motion state
  • the motion state that has no effect on the processing effect of FRUC is called a normal motion state.
  • S500 is executed to stop processing the video displayed by the display device using a frame rate up conversion processing method based on motion compensation.
  • the display device is in a normal motion state, such as moving at a constant speed.
  • S600 is executed, and the video displayed on the display device is processed using a frame rate up conversion processing method based on motion compensation.
  • the above-mentioned motion threshold may be set according to historical statistical results and pre-stored, and the pre-stored motion threshold may be called when used.
  • the effect of processing the video displayed on the display device (hereinafter referred to as the video processing effect) by using a frame rate up-conversion processing method based on motion compensation can be recorded by the display device in multiple motion states through multiple recordings,
  • the motion state of the display device is represented by the motion information of the display device, and the video processing effect depends on whether the video displayed on the display device exhibits a serious bad display phenomenon after adopting the frame rate up-conversion processing method based on motion compensation.
  • the video displayed on the display device has a serious bad display phenomenon, it indicates that the current motion state of the display device has an effect on the processing effect of the frame rate up conversion processing method based on motion compensation, and the frame rate up conversion processing method based on motion compensation is invalid ,
  • the video processing effect is invalid, if the video displayed by the display device does not show serious bad display phenomenon, and the video blur phenomenon is improved, it indicates that the current motion status of the display device handles the frame rate up conversion processing method based on motion compensation
  • the effect has no effect, the frame rate up conversion processing method based on motion compensation is effective, and the video processing effect is effective.
  • the complexity of the movement of the display device is gradually increased, for example, from gentle movement to violent shaking, and the video processing effect in each motion state is recorded to obtain the relationship between the motion information of the display device and the video processing effect.
  • the correspondence relationship the motion information of the display device corresponding to the critical state where the video processing effect changes from effective to invalid is determined as the critical motion information, and the motion threshold is determined according to the critical motion information.
  • the display device uses a video processor disposed therein to implement a frame rate up conversion processing method based on motion compensation to process the video displayed by the display device.
  • the method of stopping the processing of the video displayed on the display device by using the frame rate up-conversion processing method based on motion compensation is that the video processing method execution device performs video processing
  • the controller sends a first control signal to control the video processor to stop processing the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation.
  • the motion compensation-based frame rate up-conversion processing method is used to process the video displayed by the display device as follows: the video processing method execution device sends a second control signal to the video processor To control the video processor to continue to use the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device.
  • the motion state of the display device is influencing whether the display device adopts a frame rate up-conversion processing method based on motion compensation to the display device.
  • the processing effect of the displayed video so as to judge whether the result control frame rate up conversion processing method based on motion compensation should continue to be adopted or stopped. If the judgment result is yes, it means that the display device is in a complex motion state. In the complex motion state, the effect of processing the video displayed on the display device by using the frame rate up conversion processing method based on motion compensation will be affected. As a result, the image processing method becomes invalid.
  • the frame rate up conversion video processing method based on motion compensation is stopped to process the video displayed on the display device, which can reduce the data processing burden of the video processor and can also avoid
  • the failure of the image processing method leads to more serious display defects in the video displayed by the display device, which causes discomfort to the human body.
  • the judgment result is no, it means that the display device is in a normal motion state. In the normal motion state, the effect of processing the video displayed on the display device by the frame rate up conversion processing method based on motion compensation will not be affected
  • the display effect of the display device is improved.
  • the video processing method provided by the embodiments of the present disclosure can improve the video blur phenomenon and ensure the video quality when the display device is in a normal motion state, and can also reduce the data processing burden of the video processor when the display device is in a complex operating state , And to avoid more serious display defects in the displayed video.
  • the motion information of the display device includes X-axis acceleration Accel_X, Y-axis acceleration Accel_Y, and Z-axis acceleration Accel_Z in the three-dimensional coordinate system of the display device; wherein the X axis, Y axis, and Z axis are perpendicular to each other, X
  • the coordinate system formed by the axis, Y axis and Z axis meets the requirements of the world coordinate system.
  • the straight line of the Z axis is parallel to the straight line of the gravity direction
  • the straight line of the Z axis is perpendicular to the plane formed by the X axis and the Y axis.
  • the above motion threshold includes the acceleration threshold Accel_Threshold.
  • the acceleration threshold Accel_Threshold changes from valid to invalid according to the video processing effect.
  • the corresponding critical motion information of the display device is set.
  • the critical motion information of the display device includes the X axis Critical acceleration Accel_X0, Y-axis critical acceleration Accel_Y0 and Z-axis critical acceleration Accel_Z0.
  • the effects of S400 include:
  • S100 receiving motion information of the display device includes: receiving m motion information of the display device one by one within the first preset time period, m is An integer greater than or equal to 2.
  • the effect of S400 also includes:
  • each motion information and motion threshold value in the m motion information of the display device determine whether each of the m motion states in which the display device is located affects the display device to adopt the frame rate based on motion compensation The effect of conversion processing on the video displayed on the display device;
  • m is an integer greater than or equal to 2
  • each of the m motion states where the display device is in is determined one by one Whether the motion states all affect the display device's effect of processing the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation, so that by analyzing at least two motion states of the display device during the first preset time period, It can more accurately determine whether the motion state of the display device during the time period affects the display device's effect of processing the video displayed on the display device by using the frame rate up-conversion processing method based on motion compensation, thereby avoiding the display at a certain moment Misjudgment occurs in the motion state of the device and wrong instructions are output, which further ensures the accuracy of the video processing method.
  • the first preset time period T1 ⁇ 2/FPS, FPS is the video frame rate, which can ensure that the m motion information of the display device received within the first preset time period reflects the continuous refresh of the display device The motion state of at least two images, so as to determine more accurately whether the motion state of the display device within the first preset time period affects the display device using the frame rate up conversion processing method based on motion compensation The effect of video processing.
  • the first reception frequency is related to the refresh rate of the video displayed by the display device, for example, display
  • the first receiving frequency of the motion information of the device is greater than or equal to the video frame rate, and the value of m can be determined according to the first preset time period and the first receiving frequency.
  • the motion information of the display device can be collected by a motion sensing unit, the motion sensing unit collects the motion information of the display device at a fixed collection frequency, and sends the collected motion information of the display device to the above one by one Execution device of video processing method.
  • the fixed acquisition frequency is consistent with the fixed reception frequency. If the execution device of the above video processing method needs to receive m pieces of motion information of the display device one by one, the motion sensing unit will continuously collect the motion information of the display device collected m times.
  • the motion sensing unit collects the motion information of the display device at a collection frequency of 1000 Hz, for example, the motion information of the display device is the acceleration of the X-axis, Y-axis, and Z-axis, which is collectively referred to herein as acceleration information
  • motion The sensing unit collects the acceleration information of the display device every 1 ms, and sends the collected acceleration information of the display device to the execution device of the above-mentioned video processing method in an acceleration pulse manner of the display device.
  • the implementation of stopping the processing of the video displayed on the display device by using the frame rate up-conversion processing method based on motion compensation is that the execution device of the video processing method sends the first control signal to the video processor , S600 continues to adopt the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device in the case that the video processing method execution device sends a second control signal to the video processor ,
  • the first control signal and the second control signal are collectively referred to as the control signal, where the second control signal is a high-level signal, the high-level signal can control the video processor to continue to use the frame rate up conversion video processing method based on motion compensation.
  • the video is processed; the first control signal is a low-level signal, and the low-level signal can control the video processor to stop using the frame rate up-conversion image processing method based on motion compensation to process the video.
  • the control signal is changed from the second control signal (high level Signal) into a first control signal (low-level signal) to use the low-level signal to control the video processor to stop processing the image using the frame rate up-conversion image processing method based on motion compensation.
  • the low-level signal controls the video processor to stop using the frame rate up-conversion image processing method based on motion compensation to process the video, according to the m acceleration information and the motion threshold of the display device, it is determined None of the m states of the display device affect the display device’s effect of processing the video displayed on the display device using the frame rate up-conversion processing method based on motion compensation, then change the control signal from the first control signal (low level signal) ) Into a second control signal (high-level signal) to control the video processor to process the image using the FRUC image processing method based on motion compensation using the high-level signal.
  • the time period during which the motion sensing unit collects the acceleration information of the display device and the control signal controls whether the video processor uses a frame rate up-conversion image processing method based on motion compensation to perform video processing on the video displayed by the display device
  • the processing time is different, but because the acquisition frequency of the motion sensing unit is in the millisecond level, the time is very short, so the time difference between the acceleration information acquisition process of the motion sensing unit and the control process of the control signal to control the video processor can be approximated ignore.
  • the video processing method provided by the present disclosure further includes: determining whether the display device is in a motion state based on the motion information of the display device and its motion threshold Before affecting the effect of the display device adopting a frame rate up conversion processing method based on motion compensation to process the video displayed by the display device,
  • S001 Receive a video displayed by a display device; the video includes multiple frames of image data;
  • the video displayed by the S001 receiving display device and the motion information of the S100 receiving display device may be executed at the same time, or may be received when it is required to be applied to the video information or the motion information of the display device, depending on the actual situation.
  • S500 is executed to stop processing the video displayed by the display device using a frame rate up conversion processing method based on motion compensation.
  • the frame rate up-conversion video processing method based on motion compensation is to estimate the motion trajectory of the object between two adjacent frames of images, and obtain the adjacent two according to the image information of the two adjacent frames and the motion trajectory.
  • One frame of image information needs to be inserted between frame images to improve the refresh rate of the video and improve the video blur phenomenon. Therefore, when the continuous k frame image data in the multi-frame image data are consistent, the current time video is explained For still pictures, consecutive k-frame images are not updated. Therefore, it is not necessary to simultaneously use the frame rate up-conversion video processing method based on motion compensation to process the video at the current time.
  • the display device by receiving the video displayed by the display device, it is determined whether the consecutive k-frame image data in the multi-frame image data are consistent, so that when the judgment result is yes, the video at the current time is Still pictures, no frame rate up-conversion video processing method based on motion compensation is required to process the video at the current moment, so S500 is executed, and the frame rate up-conversion processing method based on motion compensation is stopped to perform the video displayed on the display device Processing, this can reduce the data processing burden of the video processor. If the judgment result is no, it means that the video at the current moment is a dynamic picture, and the video at the current moment needs to be processed based on the frame rate up-conversion video processing method of motion compensation, so that S600 is executed.
  • the processing method also includes: between S200 and S400,
  • S300 According to the motion information of the display device and its static threshold, determine whether the display device is static or nearly static.
  • the display device is in a motion state.
  • the normal motion state will not affect the display device's frame rate up conversion processing based on motion compensation.
  • the effect of processing the video displayed by the device, and the complex motion state will affect the effect of the display device processing the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation.
  • S400 is executed, based on the motion information of the display device and its motion threshold, to determine whether the motion state of the display device is influencing whether the display device adopts a frame rate up conversion processing method based on motion compensation to the The effect of processing the video displayed on the display device.
  • the above display device is stationary or near stationary means that the display device does not move at all, or, within the error range of the motion information of the display device received by the execution device of the video processing method, For small movements, the above rest threshold can be set according to the movement range of the display device.
  • the motion information of the display device includes the X-axis angular velocity Gyro_X, the Y-axis angular velocity Gyro_Y, and the Z-axis angular velocity Gyro_Z in the three-dimensional coordinate system of the display device; wherein, the definitions of the X-axis, Y-axis, and Z-axis Refer to the previous description for the method.
  • the above-mentioned static threshold includes the angular velocity threshold GYRO_Threshold, and the specific size can be set according to the movement range of the display device.
  • determining whether the display device is stationary or nearly stationary S300 includes:
  • S100 of receiving motion information of the display device includes: within the second preset time period T2, receiving n motion information of the display device one by one, n is an integer greater than or equal to 2, and the size of n can be based on The actual situation is set.
  • S300 to determine whether the display device is stationary or nearly stationary further includes:
  • S310b According to each of the n pieces of motion information of the display device and the static threshold, determine whether each of the n states in which the display device is located is static or nearly static one by one;
  • n is an integer greater than or equal to 2
  • each of the n states in which the display device is in the state is determined one by one Whether they are all stationary or nearly stationary, so by analyzing at least two states of the display device during the second preset time period, it can be more accurately determined whether the display device is stationary or nearly stationary during the time period, thereby avoiding a certain moment Misjudgment of the state of the display device and output of wrong instructions further ensure the accuracy of the video processing method.
  • the second preset time period T2 ⁇ 2/FPS, FPS is the video frame rate, which can ensure that the n motion information of the display device received within the second preset time period reflects the continuous refresh of the display device The state of the at least two images, so as to determine more accurately whether the display device is stationary or nearly stationary within the second preset time period.
  • n pieces of motion information of the display device are received one by one at the second reception frequency, and the second reception frequency is related to the refresh rate of the video displayed by the display device, for example, display
  • the first receiving frequency of the motion information of the device is less than or equal to the video frame rate, and the value of n can be determined according to the second preset time period and the second receiving frequency.
  • the motion information of the above display device can be collected by a motion sensing unit, the motion sensing unit collects the motion information of the above display device at a fixed collection frequency, and sends the collected motion information of the display device one by one to An execution device of the above video processing method.
  • the fixed acquisition frequency is consistent with the fixed reception frequency. If the execution device of the above video processing method needs to receive n pieces of motion information of the display device, the motion sensing unit will continuously collect the motion information of the display device collected n times.
  • the motion sensing unit collects the motion information of the above display device at a collection frequency of 1000 Hz, for example, the motion information of the display device is the angular velocity of the X-axis, Y-axis, and Z-axis, which is collectively referred to herein as angular velocity information
  • motion The sensing unit collects the angular velocity information of the display device every 1 ms, and sends the collected angular velocity information of the display device to the execution device of the above-mentioned video processing method in a pulse manner of the angular velocity of the display device.
  • the implementation of stopping the processing of the video displayed on the display device by using the frame rate up-conversion processing method based on motion compensation is that the execution device of the video processing method sends the first control signal to the video processor , S600 continues to adopt the frame rate up conversion processing method based on motion compensation to process the video displayed by the display device in the case that the video processing method execution device sends a second control signal to the video processor ,
  • the first control signal and the second control signal are collectively referred to as the control signal, where the second control signal is a high-level signal, the high-level signal can control the video processor to use the frame rate based on motion compensation up-conversion image processing method for video Processing; when the first control signal is a low-level signal, the low-level signal can control the video processor to stop using the frame rate up-conversion image processing method based on motion compensation to process the image.
  • the high-level signal controls the video processor to process the image using the frame rate up-conversion image processing method based on motion compensation
  • n angular velocity information of the display device and the relative still threshold are used according to the n angular velocity information of the display device
  • the second control signal high level signal
  • the first control signal low level signal
  • the level signal can control the video processor to stop using the frame rate up-conversion image processing method based on motion compensation to process the image.
  • the time period during which the motion sensing unit collects the angular velocity information of the display device and the control signal controls the video processor to use a frame rate up-conversion image processing method based on motion compensation to process the video displayed by the display device The time period is different, but because the acquisition frequency of the motion sensing unit is in the millisecond level, the time is very short, so the time difference between the acceleration information acquisition process of the motion sensing unit and the control process of the control signal to control the video processor can be approximately ignored .
  • each step of the above video processing method may be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software.
  • the steps of the video processing method provided in conjunction with the embodiments of the present disclosure may be directly embodied and executed by a hardware processor, or may be executed and completed by a combination of hardware and software modules in the processor.
  • the software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, and a register.
  • the storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. In order to avoid repetition, they are not described in detail here.
  • the above mainly introduces the video processing method provided by the embodiment of the present disclosure.
  • a video processing control device 01 that implements the above-described video processing method is also provided.
  • the video processing control device 01 will be exemplarily introduced below.
  • the video processing control device 01 provided by some embodiments of the present disclosure includes: a receiving module 110, a processing module 120 and a control module 130.
  • the receiving module 110 is configured to receive motion information of the display device.
  • the receiving module 110 is a transceiver.
  • the processing module 120 is coupled to the receiving module 110.
  • the processing module 120 is configured to determine the motion state of the display device based on the motion information of the display device and its motion threshold, and whether it affects the display device's use of frame rate up conversion processing based on motion compensation The effect of processing the video displayed on the display device by the method and output the judgment result.
  • the processing module 120 is a processor, and the processor may be a central processing unit (Central Processing Unit, CPU for short), or may be other general-purpose processors, digital signal processors (DSPs), and application specific integrated circuits (ASICs). ), field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may be any conventional processor.
  • the control module 130 is coupled to the processing module 120.
  • the control module 130 is configured to: according to the judgment result, the motion state of the display device affects the display device to adopt a frame rate up-conversion processing method based on motion compensation to display video on the display device
  • the first control signal that stops processing the video displayed on the display device by using the frame rate up conversion processing method based on motion compensation to control the video processor 210 to stop using the frame rate based on motion compensation
  • the up-conversion processing method processes the video displayed by the display device.
  • the output continues to use the frame rate up conversion processing based on motion compensation
  • a second control signal that processes the video displayed by the display device to control the video processor 210 to continue to process the video displayed by the display device using a frame rate up-conversion processing method based on motion compensation.
  • control module 130 is a microcontroller unit (Microcontroller Unit; MCU).
  • MCU Microcontroller Unit
  • the video processing control device 01 provided by the embodiment of the present disclosure can improve the video blur phenomenon and ensure the video quality when the display device is in a normal motion state, and can also reduce the data processing burden of the video processor when the display device is in a complex operation state , And to avoid more serious display defects in the displayed video.
  • beneficial effects please refer to the relevant content in the aforementioned video processing method, which will not be repeated here.
  • the motion information of the display device includes X-axis acceleration Accel_X, Y-axis acceleration Accel_Y and Z-axis acceleration Accel_Z in a three-dimensional coordinate system; the motion threshold Including acceleration threshold Accel_Threshold.
  • the processing module 120 is configured to determine whether the motion information of the display device and the motion threshold are satisfied: Sqrt(Accel_X 2 +Accel_Y 2 +Accel_Z 2 ) ⁇ Accel_Threshold.
  • the receiving module 110 is configured to receive m pieces of motion information of the display device one by one within a first preset time period T1, where m is equal to or greater than 2. Integer.
  • the above processing module 120 is configured to determine whether each of the m motion states in which the display device is located affects the display device based on each of the m motion information and the motion threshold of the display device The effect of motion-compensated frame rate up conversion processing on the video displayed by the display device.
  • the device uses a frame rate up-conversion processing method based on motion compensation to process the video displayed by the display device.
  • the receiving module 110 is further configured to receive the video displayed by the display device; the video includes multiple frames of image data.
  • the processing module 120 is further configured to determine whether the display device is in a motion state based on the motion information and the motion threshold of the display device, and whether it affects the display device to display the display device using a frame rate up conversion processing method based on motion compensation Before processing the effect of the video, determine whether the consecutive k frames of image data in the multi-frame image data are consistent, k is an integer greater than or equal to 2. In the case that the image data of consecutive k frames is not consistent, it is performed to determine the motion state of the display device according to the motion information of the display device and the motion threshold, whether it affects the display device to adopt the frame rate up conversion processing method based on motion compensation. The effect of processing the video displayed on the display device.
  • the above control module 130 is further configured to output a first control signal that stops processing the video displayed by the display device by using a frame rate up-conversion processing method based on motion compensation when the image data of consecutive k frames remains consistent To control the video processor 210 to stop processing the video using the frame rate up-conversion image processing method based on motion compensation.
  • the processing module 120 is further configured to execute the motion information and the motion threshold according to the display device in the case where the image data of consecutive k frames does not remain consistent, Before determining whether the motion state of the display device affects the display device's effect of processing the video displayed on the display device using the frame rate up conversion processing method based on motion compensation, determine the display device based on the motion information of the display device and its static threshold Whether it is stationary or nearly stationary. When the display device is in a motion state, determine whether the motion state of the display device is affected by the display device's motion information and motion threshold. The display device uses the frame rate up conversion processing method based on motion compensation to display the display device. The effect of video processing.
  • the above control module 130 is further configured to: when the display device is still or relatively still, output the first control to stop processing the video displayed on the display device by using a frame rate up conversion processing method based on motion compensation Signal to control the video processor 210 to process the video using a frame rate up-conversion image processing method based on motion compensation.
  • the motion information of the display device includes the X-axis angular velocity Gyro_X, the Y-axis angular velocity Gyro_Y and the Z-axis angular velocity Gyro_Z in the three-dimensional coordinate system of the display device; the rest threshold Including the angular velocity threshold GYRO_Threshold.
  • the above processing module 120 is configured to determine whether the motion information and the static threshold of the display device satisfy: Sqrt(Gyro_X 2 +Gyro_Y 2 +Gyro_Z 2 ) ⁇ GYRO_Threshold.
  • the above-mentioned receiving module 110 is configured to receive n pieces of motion information of the display device one by one within a second preset time period T2, where n is an integer greater than or equal to 2.
  • the processing module 120 is configured to determine whether each of the n states in which the display device is located is static or approximately static according to each of the n motion information of the display device and the static threshold. If yes, confirm that the display device is stationary or nearly stationary; if not, confirm that the display device is in motion.
  • module (receiving module, processing module, control module) is used to represent a computer-related entity, hardware, firmware, a combination of hardware and software, software, or software in execution.
  • a module may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and/or a computer.
  • both the application running on the computing device and the computing device can be modules.
  • the functional modules in some embodiments of the present disclosure may be integrated into one processing unit, or each module may exist alone physically, or two or more modules may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or software function module.
  • some embodiments of the present disclosure also provide a display control device 02.
  • the display control device 02 includes a video processor 210 and the above-mentioned video processing control device 01, wherein the video processor 210 is configured to use The frame rate up conversion processing method based on motion compensation processes the video displayed by the display device.
  • the video processing control device 01 is configured to control whether the video processor 210 uses a frame rate up conversion processing method based on motion compensation to process the video displayed by the display device.
  • the control module 130 included in the video processing control device 01 is coupled to the video processor 210, and outputs a control signal to the video processor 210, and controls the video processor 210 to use the frame rate based on motion compensation to up-convert the image through the control signal
  • the processing method processes the video.
  • the beneficial effects of the display control device 02 provided by the present disclosure are the same as the beneficial effects of the above-mentioned video processing method, which will not be repeated here.
  • the display control device 02 further includes a video data interface 220, and the receiving module 110 included in the video processing control device 01 is also coupled to the video data interface 220 to receive the display device’s Video information.
  • some embodiments of the present disclosure also provide a display device 03.
  • the display device 03 includes a motion sensing unit 300 and the above display control device 02.
  • the motion sensing unit 300 and the video processing control device The receiving module 110 included in 01 is connected.
  • the motion sensing unit 300 can provide the receiving module 110 with the motion information of the display device 03.
  • the display device 03 may be a virtual reality device, an augmented reality device, or the like.
  • the display device 03 provided by the present disclosure can improve the blurring phenomenon of the displayed video under the condition of normal motion, ensure the quality of the video, and reduce the video processor in the display device under the condition of complex operation The burden of data processing, and to avoid the display of more serious display defects.
  • the motion sensing unit 300 can be set in the body of the display device 03, and the collection frequency of the motion sensing unit 300 for collecting motion information can be set according to actual conditions.
  • the collection frequency of the motion sensing unit 300 is less than the video frame rate FPS to ensure that the motion information of the display device can be collected at least twice within a frame of image time.
  • the motion sensing unit 300 is an integrated structure.
  • the motion sensing unit 300 is an inertial measurement device 310, and the inertial measurement device 310 can measure the angular velocity information and acceleration of the display device 03 information.
  • Inertial measurement unit 310 also known as inertial measurement unit (Inertial measurement unit, IMU for short) is a device for measuring the three-axis attitude angle (ie angular rate) and acceleration of an object, which contains three single-axis accelerometers and three single-axis gyros .
  • the motion sensing unit 300 is a split structure.
  • the motion sensing unit 300 includes a three-axis angular velocity sensor 321 and a three-axis acceleration sensor 322; the three-axis angular velocity sensor 321 is configured To collect the angular velocity information of the display device 03, the triaxial acceleration sensor 322 is configured to collect the acceleration information of the display device 03.
  • Some embodiments of the present disclosure also provide a computer-readable storage medium that stores computer program instructions, which when executed on a processor, causes the processor to execute as described in the present disclosure One or more steps in the provided video processing method.
  • the program may be stored in a computer-readable storage medium. When executed, it may include the processes of the foregoing method embodiments.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random storage memory (Random Access Memory, RAM), etc.
  • Some embodiments of the present disclosure also provide a computer program product.
  • the computer program product includes computer program instructions. When the computer program instructions are executed on a computer, the computer program instructions cause the computer to perform one or more steps in the video processing method provided by the present disclosure.
  • a computer program When the computer program is executed on a computer, the computer program causes the computer to perform one or more steps in the video processing method provided by the present disclosure.
  • some embodiments of the present disclosure also provide a video processing terminal 400, including a processor 430, a transceiver 410, a memory 420, and a bus 440.
  • the processor 430, the transceiver 410, and the memory 420 pass through the bus 440 Communicate with each other.
  • the memory 420 is configured to store multiple instructions to implement the above video processing method
  • the processor 430 is configured to execute the multiple instructions to implement the above video processing method.
  • the processor 430 described in some embodiments of the present disclosure may be a single processor or a collective term for multiple processing elements.
  • the processor 430 may be a central processing unit (Central Processing Unit, CPU for short) or a specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), or one or more configured to implement the embodiments of the present disclosure
  • An integrated circuit for example: one or more microprocessors (digital signal processor, DSP for short), or one or more field programmable gate arrays (Field Programmable Gate Array, FPGA for short).
  • the processor 430 is an ARM (Advanced RISC Machine) processor, an AP processor, and a microcontroller unit (MCU).
  • the ARM processor itself is a 32-bit design, but it is also equipped with a 16-bit instruction set, which generally saves up to 35% over the equivalent 32-bit code, but retains all the advantages of the 32-bit system
  • the AP processor is also known as a baseband chip Microprocessor control unit, also known as single chip microcomputer (Single Chip Microcomputer) or single chip microcomputer, is to appropriately reduce the frequency and specifications of the central processor (Central Process Unit; CPU), and the memory (memory), counter ( Peripheral interfaces such as Timer), USB, A/D conversion, UART, PLC, DMA, and even LCD driver circuits are integrated on a single chip to form a chip-level computer, which can be combined and controlled for different applications.
  • CPU Central Process Unit
  • memory memory
  • Peripheral interfaces such as Timer
  • USB A/D conversion
  • the memory 420 may be a storage device, or a collective term for multiple storage elements, and is used to store executable program code and the like.
  • the memory 420 may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as a disk memory, a flash memory (Flash), and so on.
  • RAM random access memory
  • non-volatile memory non-volatile memory
  • flash flash
  • the bus 440 may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnection (Peripheral Component, PCI) bus, or an extended industry standard architecture (Extended Industry Standard, Architecture, EISA) bus, or the like.
  • the bus 440 can be divided into an address bus, a data bus, and a control bus. For ease of representation, only a thick line is used in FIG. 13, but it does not mean that there is only one bus or one type of bus.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Television Systems (AREA)

Abstract

一种视频处理方法，包括：接收显示装置的运动信息；根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；若否，继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。

Description

视频处理方法、视频处理控制装置、显示控制装置
本申请要求于2019年01月02日提交的、申请号为201910002289.3的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及显示技术领域，尤其涉及一种视频处理方法及视频处理控制装置、显示控制装置和显示装置。
背景技术
虚拟现实(Virtual Reality，VR)系统是一种可以创建和体验虚拟世界的计算机仿真系统。在虚拟现实系统的应用中，图像延时会影响所显示的图像的质量，例如对于运动图像(视频)来说，可能会出现模糊现象。
发明内容
一方面，提供一种视频处理方法，包括：接收显示装置的运动信息；根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；若否，继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
在一些实施例中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度Accel_Z；所述运动阈值包括加速度阈值Accel_Threshol；d
所述根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，包括：
判断所述显示装置的运动信息与所述运动阈值是否满足：
Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold；
若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若否，确认所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
在一些实施例中，所述接收显示装置的运动信息，包括：在第一预 设时间段内，逐一接收所述显示装置的m个运动信息，m为大于或等于2的整数；根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，还包括：
根据所述显示装置的m个运动信息中的每个运动信息与所述运动阈值，逐一判断所述显示装置所处的m个运动状态中的每个运动状态是否均影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若否，确认所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
在一些实施例中，视频处理方法还包括，在所述根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果前，
接收所述显示装置所显示的视频；所述视频包括多帧图像数据；判断所述多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数；若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；若否，根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
在一些实施例中，若所述连续k帧图像数据未保持一致，在根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果之前，所述视频处理方法还包括：
根据所述显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止；若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；若否，根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示 装置所显示的视频进行处理的效果。
在一些实施例中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴角速度Gyro_X、Y轴角速度Gyro_Y和Z轴角速度Gyro_Z；所述静止阈值包括角速度阈值GYRO_Threshold；
所述根据显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止，包括：判断所述显示装置的运动信息和所述静止阈值是否满足：
Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold；若是，确认所述显示装置静止或近似静止；若否，确认所述显示装置处于运动状态。
在一些实施例中，所述接收显示装置的运动信息包括：在第二预设时间段内，逐一接收所述显示装置的n个运动信息，n为大于或等于2的整数。所述根据显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止，还包括：根据所述显示装置的n个运动信息中的每个运动信息与所述静止阈值，逐一判断所述显示装置所处的n个状态中的每个状态是否均静止或近似静止；若是，确认所述显示装置静止或近似静止；若否，确认所述显示装置处于运动状态。
另一方面，提供一种视频处理控制装置，包括：接收模块、处理模块和控制模块。接收模块被配置为接收显示装置的运动信息。处理模块与所述接收模块耦接，所述处理模块被配置为根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，并输出判断结果。控制模块与所述处理模块耦接，所述控制模块被配置为：根据所述判断结果，在所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号；在所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果的情况下，输出继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第二控制信号。
在一些实施例中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度 Accel_Z；所述运动阈值包括加速度阈值Accel_Threshol。d
所述处理模块被配置为：判断所述显示装置的运动信息与所述运动阈值是否满足：
Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold；
若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；若否，确认所述显示装置所处的运动状态没有影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对对所述显示装置所显示的视频进行处理的效果。
在一些实施例中，所述接收模块被配置为在第一预设时间段内，逐一接收所述显示装置的m个运动信息，m为大于或等于2的整数。
所述处理模块被配置为：根据所述显示装置的m个运动信息中的每个运动信息与所述运动阈值，逐一判断所述显示装置所处的m个运动状态中的每个运动状态是否均影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。若否，确认所述显示装置所处的运动状态没有影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
在一些实施例中，所述接收模块还被配置为接收所述显示装置所显示的视频，所述视频包括多帧图像数据。所述处理模块还被配置为：在所述根据所述显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果前，判断所述多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数；在所述连续k帧图像数据未保持一致的情况下，执行根据所述显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。所述控制模块还被配置为：在所述连续k帧图像数据均保持一致的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的 第一控制信号。
在一些实施例中，所述处理模块还被配置为，在所述连续k帧图像数据未保持一致的情况下，执行根据显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果之前，根据所述显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止；在所述显示装置处于运动状态的情况下，执行根据显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。所述控制模块还被配置为：在所述显示装置静止或相对静止的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号。
在一些实施例中，所述显示装置的运动信息包括所述显示装置在三维坐标系中X轴角速度Gyro_X、Y轴角速度Gyro_Y和Z轴角速度Gyro_Z；所述相对静止阈值包括角速度阈值GYRO_Threshold；
所述处理模块被配置为：判断所述显示装置的运动信息和所述静止阈值是否满足：
Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold；若是，确认所述显示装置静止或近似静止；若否，确认所述显示装置处于运动状态。
在一些实施例中，所述接收模块被配置为：在第二预设时间段内，逐一接收所述显示装置的n个运动信息，n为大于等于2的整数。所述处理模块被配置为：根据所述显示装置的n个运动信息中的每个运动信息与静止阈值，逐一判断所述显示装置所处的n个状态中的每个状态是否均静止或近似静止；若是，确认所述显示装置静止或近似静止；若否，确认所述显示装置处于运动状态。
再一方面，提供一种显示控制装置，包括：视频处理器和如本公开所提供的视频处理控制装置。视频处理器被配置为采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。所述视频处理控制装置所包括的控制模块与所述视频处理器耦接，并向所述视频处理器输出控制信号。
又一方面，提供一种显示装置，包括：运动感应单元和如上所述的显示控制装置，所述显示控制装置与所述运动感应单元耦接。
又一方面，提供一种计算机可读存储介质。所述计算机可读存储介质存储有计算机程序指令，所述计算机程序指令在处理器上运行时，使得所述处理器执行如上述任一实施例所述的视频处理方法中的一个或多个步骤。
又一方面，提供一种计算机程序产品。所述计算机程序产品包括计算机程序指令，在计算机上执行所述计算机程序指令时，所述计算机程序指令使计算机执行如上述任一实施例所述的视频处理方法中的一个或多个步骤。
又一方面，提供一种计算机程序。当所述计算机程序在计算机上执行时，所述计算机程序使计算机执行如上述任一实施例所述的视频处理方法中的一个或多个步骤。
附图说明
为了更清楚地说明本公开中的技术方案，下面将对本公开一些实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例的附图，对于本领域普通技术人员来讲，还可以根据这些附图获得其他的附图。此外，以下描述中的附图可以视作示意图，并非对本公开实施例所涉及的产品的实际尺寸、方法的实际流程、信号的实际时序等的限制。
图1为根据一些实施例的视频处理方法的一种流程图；
图2为根据一些实施例的视频处理方法的另一种流程图；
图3为根据一些实施例的视频处理方法的再一种流程图；
图4为根据一些实施例的视频处理方法的一种控制时序图；
图5为根据一些实施例的视频处理方法的另一种控制时序图；
图6为根据一些实施例的视频处理方法的又一种流程图；
图7为根据一些实施例的视频处理方法的又一种流程图；
图8为根据一些实施例的视频处理方法的再一种的控制时序图；
图9为根据一些实施例的视频处理控制装置的结构图；
图10为根据一些实施例的显示装置的一种结构图；
图11为根据一些实施例的显示装置的另一种结构图；
图12为根据一些实施例的显示装置的再一种结构图；
图13为根据一些实施例的视频处理终端的结构图。
具体实施方式
下面将结合附图，对本公开一些实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。基于本公开所提供的实施例，本领域普通技术人员所获得的所有其他 实施例，都属于本公开保护的范围。
除非上下文另有要求，否则，在整个说明书和权利要求书中，术语“包括(comprise)”及其其他形式例如第三人称单数形式“包括(comprises)”和现在分词形式“包括(comprising)”被解释为开放、包含的意思，即为“包含，但不限于”。在说明书的描述中，术语“一个实施例(one embodiment)”、“一些实施例(some embodiments)”、“示例性实施例(exemplary embodiments)”、“示例(example)”、“特定示例(specific example)”或“一些示例(some examples)”等旨在表明与该实施例或示例相关的特定特征、结构、材料或特性包括在本公开的至少一个实施例或示例中。上述术语的示意性表示不一定是指同一实施例或示例。此外，所述的特定特征、结构、材料或特点可以以任何适当方式包括在任何一个或多个实施例或示例中。
在描述一些实施例时，可能使用了“耦接”和“连接”及其衍伸的表达。例如，描述一些实施例时可能使用了术语“连接”以表明两个或两个以上部件彼此间有直接物理接触或电接触。又如，描述一些实施例时可能使用了术语“耦接”以表明两个或两个以上部件有直接物理接触或电接触。然而，术语“耦接”或“通信耦合(communicatively coupled)”也可能指两个或两个以上部件彼此间并无直接接触，但仍彼此协作或相互作用。这里所公开的实施例并不必然限制于本文内容。
在虚拟现实系统中，用户作为参与者，通过虚拟现实设备，可以对虚拟世界进行体验和交互作用。用户在进行移动、或者发生动作的同时，虚拟现实设备也会进行相应的运动，计算机可以立即进行复杂的运算，将精确的3D世界影像传回产生临场感。也就是说，虚拟现实设备所显示的视频，与虚拟现实设备的运动信息状态有关，而由于虚拟现实系统中存在图像延时问题，会导致虚拟现实设备所显示的视频帧相比虚拟现实设备当前时刻的运动状态延后出现，影响所显示的视频的质量，例如所显示的视频会出现模糊现象。而由于直接采用提升屏幕刷新率的方式会受到显示装置的硬件限制，无法有效解决图像延时问题。现有虚拟现实系统采用提升屏幕刷新率的方式，降低图像延时，但屏幕刷新率的提升受到图形处理器的渲染能力的限制，难以有效解决图像延时问题所造成的显示质量不佳的问题。
相关技术中，虚拟现实设备采用基于运动补偿的帧速率上转换(Frame rate up conversion，FRUC)图像处理方式处理视频信息，以改善图像延时所导致的视频模糊现象，从而有效地提高所显示的视频的质量。
基于运动补偿的帧速率上转换图像处理方式是采用动态映像系统，在传 统的两帧图像之间加插一帧运动补偿帧，以达到提高屏幕刷新率的目的，例如将虚拟现实设备的刷新率由原本的50/60Hz提升至100/120Hz。基于运动补偿的帧速率上转换图像处理方式主要包括运动估计(Motion Estimate，ME)步骤和补偿差值(Motion Compensation，MC)步骤，其中，运动估计步骤包括预估相邻两帧图像之间物体的运动轨迹，插值步骤包括根据根据相邻两帧图像信息，及所预估得到的该相邻两帧图像之间的物体的运动轨迹，获得相邻两帧图像之间所需要插入的一帧图像信息。但是，在相邻两帧图像之间物体的运动轨迹无法估计的情况下，基于运动补偿的帧速率上转换图像处理方式就会失效，此时若依旧采用基于运动补偿的帧速率上转换视频处理方式对视频进行处理，不仅会加重视频处理器的数据处理负担，还会因该图像处理方式失效，而导致虚拟现实设备所显示的视频出现比视频模糊现象更为严重的显示不良现象，例如所显示的视频与用户实际的运动状态不对应，从而引起人体不适，例如头晕目眩。
本公开的发明人经研究发现，由于虚拟现实设备所显示的视频与虚拟现实设备的运动信息状态相对应，因此在虚拟现实设备所显示的视频中相邻两帧图像之间物体的运动轨迹无法估计的情况下，说明该虚拟现实设备所处的运动状态，对采用基于运动补偿的帧速率上转换视频处理方式对视频进行处理的效果造成了影响，导致该图像处理方式失效。示例性地，以虚拟现实设备为VR头显为例，在用户佩戴VR头显处于高速晃动等状态时，VR头显中所显示的视频的相邻两帧图像之间的物体的运动轨迹无法预估，基于运动补偿的帧速率上转换视频处理方式失效。
基于此，如图1所示，本公开的一些实施例提供了一种视频处理方法，包括：
S100、接收显示装置的运动信息。示例性地，显示装置的运动信息可以为该显示装置的加速度、角度速度等运动信息。
S400、根据显示装置的运动信息及其运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。以下称对FRUC的处理效果产生影响的运动状态为复杂运动状态，称对FRUC的处理效果没有产生影响的运动状态为正常运动状态。
若是，则说明显示装置处于复杂运动状态，例如高速晃动等。此时执行S500、停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
若否，则说明显示装置处于正常运动状态，例如匀速移动等。此时执行S600、继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
在一些实施例中，上述运动阈值可根据历史统计结果设定，并进行预存，在使用时调用预存的运动阈值即可。例如：可通过多次记录显示装置在多个运动状态下，显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果(以下简称视频处理效果)，显示装置的运动状态用显示装置的运动信息表示，视频处理效果根据采用基于运动补偿的帧速率上转换处理方式后，显示装置所显示的视频是否出现严重不良显示现象而定。若显示装置所显示的视频出现严重不良显示现象，则表明显示装置当前时刻的运动状态对基于运动补偿的帧速率上转换处理方式的处理效果产生影响，基于运动补偿的帧速率上转换处理方式失效，视频处理效果为无效，若显示装置所显示的视频没有出现严重不良显示现象，且改善了视频模糊现象，则表明显示装置当前时刻的运动状态对基于运动补偿的帧速率上转换处理方式的处理效果无影响，基于运动补偿的帧速率上转换处理方式有效，视频处理效果为有效。
示例性地，逐渐增大显示装置的运动的复杂性，例如由平缓移动逐渐变为剧烈晃动，记录在每个运动状态下的视频处理效果，得到显示装置的运动信息与视频处理效果之间的对应关系，根据该对应关系，确定视频处理效果由有效转变为无效的临界状态下所对应的显示装置的运动信息，视为临界运动信息，根据该临界运动信息确定运动阈值。
可以理解的是，不同显示装置对应的运动阈值有所不同，对每个显示装置均应该进行事先设定运动阈值的操作。
在一些实施例中，显示装置利用设置于其内部的视频处理器实现基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
在上述实施例中，示例性地，S500中停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第一控制信号，以控制视频处理器停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
示例性地，S600中继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第二控制信号，以控制视频处理器继续采用基于运动补 偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
本公开所提供的视频处理方式中，根据显示装置的运动信息与运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，从而判断结果控制基于运动补偿的帧速率上转换处理方式应该继续采用还是停止。在判断结果为是的情况下，则说明显示装置处于复杂运动状态，在复杂运动状态下，采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果会受到影响，导致该图像处理方式失效，此时停止采用基于运动补偿的帧速率上转换视频处理方式对所述显示装置所显示的视频进行处理，这样能够降低视频处理器的数据处理负担，还能够避免因该图像处理方式失效，而导致显示装置所显示的视频出现更加严重的显示不良现象，而引起人体不适。而在判断结果为否的情况下，说明显示装置处于正常运动状态，在正常运动状态下，采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果不会受到影响，此时继续采用相关技术中的基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理，以保证显示装置所显示的视频不会因图像延时问题而出现视频模糊现象，从而提高显示装置的显示效果。
因此，本公开实施例提供的视频处理方法，能够在显示装置处于正常运动状态下，改善视频模糊现象，保证视频质量，还能够在显示装置处于复杂运行状态下，降低视频处理器的数据处理负担，及避免所显示的视频出现更加严重的显示不良现象。
在一些实施例中，上述显示装置的运动信息包括显示装置在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度Accel_Z；其中，X轴、Y轴、Z轴相互垂直，X轴、Y轴和Z轴所构成的坐标系符合世界坐标系的要求。例如：在显示装置的显示面与地平面垂直，而与重力方向所在直线平行时，Z轴所在直线与重力方向所在直线平行，Z轴所在直线垂直于X轴和Y轴所形成的平面。
上述运动阈值包括加速度阈值Accel_Threshold，在一些实施例中，加速度阈值Accel_Threshold根据视频处理效果由有效转变为无效的时刻，所对应的显示装置的临界运动信息设定，显示装置的临界运动信息包括X轴临界加速度Accel_X0、Y轴临界加速度Accel_Y0和Z轴临界加速度Accel_Z0。
如图2所示，根据显示装置的运动信息及其运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方 式对显示装置所显示的视频进行处理的效果的S400，包括：
S410a、判断显示装置的运动信息与运动阈值是否满足：
Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold，
若是，执行S420、确认显示装置所处的运动状态影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果；
若否，执行S430、确认显示装置所处的运动状态不影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
在一些实施例中，为了避免对显示装置所处状态的误判，接收显示装置的运动信息的S100，包括：在第一预设时间段内，逐一接收显示装置的m个运动信息，m为大于等于2的整数。
如图3所示，根据显示装置的运动信息及其运动阈值，判断显示装置所处的运动状态是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果的S400，还包括：
S410a、根据显示装置的m个运动信息中的每个运动信息与运动阈值，逐一判断显示装置所处的m个运动状态中的每个运动状态是否均影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果；
若是，执行S420、确认显示装置所处的运动状态影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果；
若否，执行S430、确认显示装置所处的运动状态不影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
在上述实施例中，通过在第一预设时间段内，逐一接收显示装置的m个运动信息，m为大于等于2的整数，进而逐一判断显示装置所处的m个运动状态中的每个运动状态是否均影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，这样通过分析第一预设时间段内，显示装置的至少两个运动状态，能够更加准确地确定该时间段内显示装置的运动状态是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，从而避免了在某一时刻对显示装置的运动状态出现误判而输出错误指令，进一步保证了该视频处理 方法的准确性。
在一些示例中，第一预设时间段T1≥2/FPS，FPS为视频帧率，这样可保证在第一预设时间段内所接收的显示装置的m个运动信息反映了显示装置连续刷新至少两幅图像时所处的运动状态，从而更加准确地确定该第一预设时间段内显示装置的运动状态是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。作为一种可能的设计，在第一预设时间段内，以第一接收频率逐一接收显示装置的m个运动信息，第一接收频率与显示装置所显示的视频的刷新率有关，例如，显示装置的运动信息的第一接收频率大于或等于视频帧率，根据第一预设时间段和第一接收频率，可以确定m的值。
在一种可实现的方式中，可通过运动感应单元采集上述显示装置的运动信息，运动感应单元以固定采集频率采集显示装置的运动信息，并逐一将所采集的显示装置的运动信息发送至上述视频处理方法的执行装置。在一些示例中，固定采集频率与固定接收频率一致。若上述视频处理方法的执行装置需要逐一接收显示装置的m个运动信息，那么运动感应单元会连续采集m次采集的显示装置的运动信息。
示例性的，在运动感应单元以1000Hz的采集频率采集上述显示装置的运动信息的情况下，例如显示装置的运动信息为X轴、Y轴和Z轴的加速度，此处统称为加速度信息，运动感应单元每1ms采集一次显示装置的加速度信息，并将所采集的显示装置的加速度信息以显示装置的加速度脉冲方式发送给上述视频处理方法的执行装置。
进一步的，在S500中停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第一控制信号，S600中继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第二控制信号的情况下，第一控制信号和第二控制信号统称为控制信号，其中，第二控制信号为高电平信号，高电平信号可控制视频处理器继续采用基于运动补偿的帧速率上转换视频处理方式对视频进行处理；第一控制信号为低电平信号，低电平信号可控制视频处理器停止采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理。
基于此，如图4所示，当高电平信号控制视频处理器采用基于运动补偿的帧速率上转换图像处理方式对图像进行处理的时候，若根据显示装置的m 个加速度信息与运动阈值，判断出显示装置所处的m个状态均影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，那么将控制信号从第二控制信号(高电平信号)转为第一控制信号(低电平信号)，以利用低电平信号控制视频处理器停止采用基于运动补偿的帧速率上转换图像处理方式对图像进行处理。
如图5所示，当低电平信号控制视频处理器停止采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理的时候，若根据显示装置的m个加速度信息与运动阈值，判断出显示装置所处的m个状态没有均影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，那么将控制信号从第一控制信号(低电平信号)转为第二控制信号(高电平信号)，以利用高电平信号控制视频处理器采用基于运动补偿的FRUC图像处理方式对图像进行处理。
可以理解的是，上述运动感应单元采集上述显示装置的加速度信息所处的时间段与上述控制信号控制视频处理器是否采用基于运动补偿的帧速率上转换图像处理方式对显示装置所显示的视频进行处理所处的时间段不同，但由于运动感应单元的采集频率在毫秒级别，时间非常短，所以，上述运动感应单元的加速度信息采集过程与控制信号控制视频处理器的控制过程的时间差异可以近似忽略。
在一些实施例中，如图1所示，本公开提供的视频处理方法还包括：在所述根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果前，
S001、接收显示装置所显示的视频；所述视频包括多帧图像数据；
在一些示例中，S001接收显示装置所显示的视频与S100接收显示装置的运动信息可在同一时间执行，也可以在需要应用到视频信息或显示装置的运动信息时接收，具体根据实际情况决定。
S200、判断所述多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数。
若是，则执行S500，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
若否，则执行S400，根据显示装置的运动信息及其运动阈值，判断显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
由于在本公开中，基于运动补偿的帧速率上转换视频处理方式是通过预估相邻两帧图像之间物体的运动轨迹，根据相邻两帧图像信息及所述运动轨迹，获得相邻两帧图像之间所需要插入的一帧图像信息，以提升视频的刷新率，改善视频模糊现象，因此，在多帧图像数据中的连续k帧图像数据均保持一致的情况下，说明当前时刻视频为静止画面，连续k帧图像没有进行更新，因此，不需要同时也无法采用基于运动补偿的帧速率上转换视频处理方式对当前时刻的视频进行处理。
在上述实施例中，通过接收所述显示装置所显示的视频，判断多帧图像数据中的连续k帧图像数据是否均保持一致，这样在判断结果为是的情况下，说明当前时刻的视频为静止画面，不需要基于运动补偿的帧速率上转换视频处理方式对当前时刻的视频进行处理，从而执行S500，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理，这样可以降低视频处理器的数据处理负担。在判断结果为是否的情况下，说明当前时刻的视频为动态画面，需要基于运动补偿的帧速率上转换视频处理方式对当前时刻的视频进行处理，从而执行S600。
在显示装置静止或近似静止的情况下，显示装置所显示的视频并不会受到图像延时的影响而出现模糊现象，基于此，如图1所示，本公开的一些实施例所提供的视频处理方法还包括：在S200与S400之间，
S300：根据显示装置的运动信息及其静止阈值，判断显示装置是否静止或近似静止。
若是，则说明显示装置所处的状态并不会使显示装置所显示的视频受到图像延时的影响而出现模糊现象，因此不需要采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理，执行S500，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
若否，说明显示装置所处的状态为运动状态，则需要判断该运动状态为正常运动状态还是复杂运动状态，正常运动状态不会影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，而复杂运动状态会影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。基于此，执行S400，根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
需要说明的是，上述显示装置静止或近似静止指的是显示装置完全不发生运动，或者，在视频处理方法的执行装置的所接收到的显示装置的运动信息的误差范围内，显示装置发生较小的移动，上述静止阈值可依据显示装置的移动幅度设定。
在一种具体实现方式中，上述显示装置的运动信息包括显示装置在三维坐标系中的X轴角速度Gyro_X、Y轴角速度Gyro_Y和Z轴角速度Gyro_Z；其中，X轴、Y轴、Z轴的定义方式参照前文描述。
上述静止阈值包括角速度阈值GYRO_Threshold，具体大小可根据显示装置的移动幅度设定。
示例性的，如图6所示，根据显示装置的运动信息及其静止阈值，判断显示装置是否静止或者近似静止的S300，包括：
S310a、判断显示装置的运动信息和所述静止阈值是否满足：
Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold；
若是，执行S320、确认显示装置静止或近似静止。
若否，执行S330、确认显示装置处于运动状态。
在一些实施例中，接收显示装置的运动信息的S100，包括：在第二预设时间段T2内，逐一接收显示装置的n个运动信息，n为大于等于2的整数，n的大小可根据实际情况设定。
如图7所示，根据显示装置的运动信息及其静止阈值，判断显示装置是否静止或近似静止的S300，还包括：
S310b：根据显示装置的n个运动信息中的每个运动信息与静止阈值，逐一判断显示装置所处的n个状态中的每个状态是否均静止或近似静止；
若是，执行S320、确认显示装置静止或近似静止。
若否，执行S330、确认显示装置处于运动状态。
在上述实施例中，通过在第二预设时间段内，逐一接收显示装置的n个运动信息，n为大于等于2的整数，进而逐一判断显示装置所处的n个状态中的每个状态是否均静止或近似静止，这样通过分析第二预设时间段内，显示装置的至少两个状态，能够更加准确地确定该时间段内显示装置是否静止或近似静止，从而避免了在某一时刻对显示装置的状态出现误判而输出错误指令，进一步保证了该视频处理方法的准确性。
在一些示例中，第二预设时间段T2≥2/FPS，FPS为视频帧率，这样可保证在第二预设时间段内所接收的显示装置的n个运动信息反映了显示装置连 续刷新至少两幅图像时所处的状态，从而更加准确地确定该第二预设时间段内显示装置是否静止或近似静止。作为一种可能的设计，在第二预设时间段内，以第二接收频率逐一接收显示装置的n个运动信息，第二接收频率与显示装置所显示的视频的刷新率有关，例如，显示装置的运动信息的第一接收频率小于或等于视频帧率，根据第二预设时间段和第二接收频率，可以确定n的值。
在一种可实现的方式中，可通过运动感应单元采集上述显示装置的运动信息，运动感应单元以固定采集频率采集上述显示装置的运动信息，并逐一将所采集的显示装置的运动信息发送至上述视频处理方法的执行装置。在一些示例中，固定采集频率与固定接收频率一致。若上述视频处理方法的执行装置需要接收显示装置的n个运动信息，那么运动感应单元会连续采集n次采集的显示装置的运动信息。
示例性的，当运动感应单元以1000Hz的采集频率采集上述显示装置的运动信息的情况下，例如显示装置的运动信息为X轴、Y轴和Z轴的角速度，此处统称为角速度信息，运动感应单元每1ms采集一次显示装置的角速度信息，并将所采集的显示装置的角速度信息以显示装置的角速度脉冲方式发送给上述视频处理方法的执行装置。
进一步的，在S500中停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第一控制信号，S600中继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的实现方式为，该视频处理方法的执行装置向视频处理器发送第二控制信号的情况下，第一控制信号和第二控制信号统称为控制信号，其中，第二控制信号为高电平信号，高电平信号可控制视频处理器采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理；第一控制信号为低电平信号时，低电平信号可控制视频处理器停止采用基于运动补偿的帧速率上转换图像处理方式对图像进行处理。
基于此，如图8所示，当高电平信号控制视频处理器采用基于运动补偿的帧速率上转换图像处理方式对图像进行处理的时候，若根据显示装置的n个角速度信息与相对静止阈值，判断出显示装置所处的n个状态均为静止状态或近似静止状态时，那么将从第二控制信号(高电平信号)转为第一控制信号(低电平信号)，以利用低电平信号可控制视频处理器停止采用基于运动补偿的帧速率上转换图像处理方式对图像进行处理。
可以理解的是，上述运动感应单元采集上述显示装置的角速度信息所处的时间段与上述控制信号控制视频处理器采用基于运动补偿的帧速率上转换图像处理方式对显示装置所显示的视频进行处理所处的时间段不同，但由于运动感应单元的采集频率在毫秒级别，时间非常短，所以，上述运动感应单元的加速度信息采集过程与控制信号控制视频处理器的控制过程的时间差异可以近似忽略。
在实现过程中，上述视频处理方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本公开实施例所提供的视频处理方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
以上主要介绍了本公开实施例提供的视频处理方法。如图9所示，在本公开的一些实施例中，还提供了实现上述视频处理方法的视频处理控制装置01，下面将对视频处理控制装置01进行示例性的介绍。
如图1、图9和图10所示，本公开的一些实施例所提供的视频处理控制装置01包括：接收模块110、处理模块120和控制模块130。
接收模块110被配置为接收显示装置的运动信息。示例性地，接收模块110为收发器。
处理模块120与接收模块110耦接，处理模块120被配置为根据显示装置的运动信息及其运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果，并输出判断结果。
示例性地，上述处理模块120为处理器，该处理器可以是中央处理单元(Central Processing Unit，简称CPU)，还可以是其他通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
控制模块130与处理模块120耦接，控制模块130被配置为：根据判断结果，在显示装置所处的运动状态影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行 处理的第一控制信号，以控制视频处理器210停止采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理。在显示装置所处的运动状态不影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果的情况下，输出继续采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的第二控制信号，以控制视频处理器210继续采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理。
示例性地，上述控制模块130为微控制单元(Microcontroller Unit；MCU)。
本公开实施例提供的视频处理控制装置01，能够在显示装置处于正常运动状态下，改善视频模糊现象，保证视频质量，还能够在显示装置处于复杂运行状态下，降低视频处理器的数据处理负担，及避免所显示的视频出现更加严重的显示不良现象。有益效果的具体推理可参见前述视频处理方法中的相关内容，此处不再赘述。
在一些实施例中，如图2、图9和图10所示，上述显示装置的运动信息包括在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度Accel_Z；所述运动阈值包括加速度阈值Accel_Threshold。
处理模块120被配置为：判断所述显示装置的运动信息与所述运动阈值是否满足：Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold。
若是，确认显示装置所处的运动状态影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
若否，确认显示装置所处的运动状态没有影响显示装置采用基于运动补偿的帧速率上转换处理方式对对显示装置所显示的视频进行处理的效果。
在一些实施例中，如图3、图9和图10所示，上述接收模块110被配置为在第一预设时间段T1内逐一接收显示装置的m个运动信息，m为大于等于2的整数。
上述处理模块120被配置为：根据显示装置的m个运动信息中的每个运动信息与运动阈值，逐一判断显示装置所处的m个运动状态中的每个运动状态是否均影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
若是，确认显示装置所处的运动状态影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果；若否，确认显示装置所处的运动状态没有影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
在一些实施例中，如图1、图9和图10所示，所述接收模块110还被配置为接收显示装置所显示的视频；所述视频包括多帧图像数据。
上述处理模块120还被配置为在根据所述显示装置的运动信息与运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果前，判断多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数。在连续k帧图像数据未保持一致的情况下，执行根据显示装置的运动信息与运动阈值，判断显示装置所处的运动状态，是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
上述控制模块130还被配置为在连续k帧图像数据均保持一致的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号，以控制视频处理器210停止采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理。
在一些实施例中，如图1、图9和图10所示，处理模块120还被配置为，在连续k帧图像数据未保持一致的情况下，执行根据显示装置的运动信息与运动阈值，判断显示装置所处的运动状态是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果之前，根据显示装置的运动信息及其静止阈值，判断显示装置是否静止或近似静止。在显示装置处于运动状态的情况下，执行根据显示装置的运动信息与运动阈值，判断显示装置所处的运动状态是否影响显示装置采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理的效果。
上述控制模块130还被配置为：在所述显示装置静止或相对静止的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号，以控制视频处理器210采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理。
示例性的，如图6、图9和图11所示，上述显示装置的运动信息包括显示装置在三维坐标系中的X轴角速度Gyro_X、Y轴角速度Gyro_Y和Z轴角速度Gyro_Z；所述静止阈值包括角速度阈值GYRO_Threshold。
上述处理模块120被配置为判断显示装置的运动信息和静止阈值是否满足：Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold。
若是，确认显示装置静止或近似静止；若否，确认显示装置处于运动状态。
示例性的，如图7、图9和图11所示，上述接收模块110被配置为在第二预设时间段T2内逐一接收显示装置的n个运动信息，n为大于等于2的整数。
上述处理模块120被配置为根据显示装置的n个运动信息中的每个运动信息与静止阈值，逐一判断显示装置所处的n个状态中的每个状态是否均静止或近似静止。若是，确认显示装置静止或近似静止；若否，确认显示装置处于运动状态。
需要说明的是，上述“模块”(接收模块、处理模块、控制模块)用于表示计算机相关的实体、硬件、固件、硬件和软件的组合、软件、或执行中的软件。例如，模块可以是但不限于，在处理器上运行的进程、处理器、对象、可执行文件、执行线程、程序和/或计算机。通过图示，在计算设备上运行的应用和计算设备都可以是模块。
另外，在本公开一些实施例中的各功能模块可以集成在一个处理单元中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
如图10所示，本公开的一些实施例还提供了一种显示控制装置02，该显示控制装置02包括视频处理器210以及上述视频处理控制装置01，其中，视频处理器210被配置为采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理。视频处理控制装置01被配置为控制视频处理器210是否采用基于运动补偿的帧速率上转换处理方式对显示装置所显示的视频进行处理。
视频处理控制装置01所包括的控制模块130与视频处理器210耦接，并向视频处理器210输出控制信号，通过该控制信号，控制视频处理器210是否采用基于运动补偿的帧速率上转换图像处理方式对视频进行处理。
与相关技术相比，本公开所提供的显示控制装置02的有益效果与上述视频处理方法的有益效果相同，在此不做赘述。
在一些实施例中，如图10所示，上述显示控制装置02还包括视频数据接口220，视频处理控制装置01所包括的接收模块110还与视频数据接口220耦接，用以接收显示装置的视频信息。
如图10～图12所示，本公开的一些实施例还提供了一种显示装置03，该显示装置03包括运动感应单元300以及上述显示控制装置02，该运动感应单元300与视频处理控制装置01所包括的接收模块110连接。其中，运动感应 单元300可向接收模块110提供显示装置03的运动信息。
在一些示例中，显示装置03可以为虚拟现实装置、增强现实装置等。
本公开所提供的显示装置03，能够在处于正常运动状态的情况下，改善所显示的视频的模糊现象，保证视频质量，还能够在处于复杂运行状态的情况下，降低显示装置中视频处理器的数据处理负担，及避免所显示的视频出现更加严重的显示不良现象。
其中，上述运动感应单元300可设在显示装置03的机体内，运动感应单元300采集运动信息的采集频率大小可以根据实际情况设定，示例性地，运动感应单元300的采集频率小于视频帧率FPS，以确保一帧图像时间内可采集至少两次显示装置的运动信息。
在一些实施例中，如图11所示，上述运动感应单元300为一体式结构，示例性地，运动感应单元300为惯性测量器310，惯性测量器310可测量显示装置03的角速度信息和加速度信息。惯性测量器310又称惯性测量单元(Inertial measurement unit，简称IMU)是测量物体三轴姿态角(即角速率)以及加速度的装置，其中含有三个单轴的加速度计和三个单轴的陀螺。
在另一些实施例中，如图12所示，运动感应单元300为分体结构，示例性地，运动感应单元300包括三轴角速度传感器321和三轴加速度传感器322；三轴角速度传感器321被配置为采集显示装置03的角速度信息，三轴加速度传感器322被配置为采集显示装置03的加速度信息。
本公开的一些实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序指令，所述计算机程序指令在处理器上运行时，使得处理器执行如本公开所提供的视频处理方法中的一个或多个步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。
本公开的一些实施例还提供一种计算机程序产品。所述计算机程序产品包括计算机程序指令，在计算机上执行所述计算机程序指令时，所述计算机程序指令使计算机执行如本公开所提供的视频处理方法中的一个或多个步骤。
又一方面，提供一种计算机程序。当所述计算机程序在计算机上执行时，所述计算机程序使计算机执行如本公开所提供的视频处理方法中的一个或多 个步骤。
如图13所示，本公开的一些实施例还提供了一种视频处理终端400，包括处理器430、收发器410、存储器420和总线440，处理器430、收发器410和存储器420通过总线440彼此通信。
其中，存储器420被配置为存储多个指令以实现上述视频处理方法，处理器430被配置为执行所述多个指令以实现上述视频处理方法。
在一些示例中，本公开的一些实施例所述的处理器430可以是一个处理器，也可以是多个处理元件的统称。例如，该处理器430可以是中央处理器(Central Processing Unit，简称CPU)，也可以是特定集成电路(Application Specific Integrated Circuit，简称ASIC)，或者是被配置成实施本公开实施例的一个或多个集成电路，例如：一个或多个微处理器(digital signal processor，简称DSP)，或，一个或者多个现场可编程门阵列(Field Programmable Gate Array，简称FPGA)。
或者，上述处理器430为ARM(Advanced RISC Machine)处理器、AP处理器、微控制单元(Microcontroller Unit；MCU)。其中，ARM处理器本身是32位设计，但也配备16位指令集，一般来讲比等价32位代码节省达35％，却能保留32位系统的所有优势；AP处理器又称基带芯片的处理器；微控制单元又称单片微型计算机(Single Chip Microcomputer)或者单片机，是把中央处理器(Central Process Unit；CPU)的频率与规格做适当缩减，并将内存(memory)、计数器(Timer)、USB、A/D转换、UART、PLC、DMA等周边接口，甚至LCD驱动电路都整合在单一芯片上，形成芯片级的计算机，为不同的应用场合做不同组合控制。
存储器420可以是一个存储装置，也可以是多个存储元件的统称，且用于存储可执行程序代码等。且存储器420可以包括随机存储器(RAM)，也可以包括非易失性存储器(non-volatile memory)，例如磁盘存储器，闪存(Flash)等。
总线440可以是工业标准体系结构(Industry Standard Architecture，ISA)总线、外部设备互连(Peripheral Component，PCI)总线或扩展工业标准体系结构(Extended Industry Standard Architecture，EISA)总线等。该总线440可以分为地址总线、数据总线、控制总线等。为便于表示，图13中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同 之处。尤其，对于设备实施例而言，由于其基本相似于方法实施例，所以描述得比较简单，相关之处参见方法实施例的部分说明即可。
以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。

Claims (16)

  1. 一种视频处理方法，包括：
    接收显示装置的运动信息；
    根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；
    若否，继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理。
  2. 根据权利要求1所述的视频处理方法，其中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度Accel_Z；所述运动阈值包括加速度阈值Accel_Threshold；
    所述根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，包括：
    判断所述显示装置的运动信息与所述运动阈值是否满足：
    Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold；
    若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若否，确认所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
  3. 根据权利要求1或2所述的视频处理方法，其中，所述接收显示装置的运动信息，包括：在第一预设时间段内，逐一接收所述显示装置的m个运动信息，m为大于或等于2的整数；
    根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，还包括：
    根据所述显示装置的m个运动信息中的每个运动信息与所述运动阈值，逐一判断所述显示装置所处的m个运动状态中的每个运动状态是否均影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显 示的视频进行处理的效果；
    若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若否，确认所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
  4. 根据权利要求1～3任一项所述的视频处理方法，还包括，在所述根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果前，
    接收所述显示装置所显示的视频；所述视频包括多帧图像数据；
    判断所述多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数；
    若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；
    若否，根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
  5. 根据权利要求4所述的视频处理方法，其中，若所述连续k帧图像数据未保持一致，在根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果之前，所述视频处理方法还包括：
    根据所述显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止；
    若是，停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；
    若否，根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
  6. 根据权利要求5所述的视频处理方法，其中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴角速度Gyro_X、Y轴角速度 Gyro_Y和Z轴角速度Gyro_Z；所述静止阈值包括角速度阈值GYRO_Threshold；
    所述根据显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止，包括：
    判断所述显示装置的运动信息和所述静止阈值是否满足：
    Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold；
    若是，确认所述显示装置静止或近似静止；
    若否，确认所述显示装置处于运动状态。
  7. 根据权利要求5或6所述的视频处理方法，其中，所述接收显示装置的运动信息包括：在第二预设时间段内，逐一接收所述显示装置的n个运动信息，n为大于或等于2的整数；
    所述根据显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止，还包括：
    根据所述显示装置的n个运动信息中的每个运动信息与所述静止阈值，逐一判断所述显示装置所处的n个状态中的每个状态是否均静止或近似静止；
    若是，确认所述显示装置静止或近似静止；
    若否，确认所述显示装置处于运动状态。
  8. 一种视频处理控制装置，包括：
    接收模块，被配置为接收显示装置的运动信息；
    与所述接收模块耦接的处理模块，所述处理模块被配置为根据所述显示装置的运动信息及其运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果，并输出判断结果；和
    与所述处理模块耦接的控制模块，所述控制模块被配置为：根据所述判断结果，在所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号；在所述显示装置所处的运动状态不影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果的情况下，输出继续采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第二控制信号。
  9. 根据权利要求8所述的视频处理控制装置，其中，所述显示装置的运 动信息包括所述显示装置在三维坐标系中的X轴加速度Accel_X、Y轴加速度Accel_Y和Z轴加速度Accel_Z；所述运动阈值包括加速度阈值Accel_Threshold；
    所述处理模块被配置为：
    判断所述显示装置的运动信息与所述运动阈值是否满足：
    Sqrt(Accel_X 2+Accel_Y 2+Accel_Z 2)≥Accel_Threshold；
    若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若否，确认所述显示装置所处的运动状态没有影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对对所述显示装置所显示的视频进行处理的效果。
  10. 根据权利要求8或9所述的视频处理控制装置，其中，所述接收模块被配置为在第一预设时间段内，逐一接收所述显示装置的m个运动信息，m为大于或等于2的整数；
    所述处理模块被配置为：
    根据所述显示装置的m个运动信息中的每个运动信息与所述运动阈值，逐一判断所述显示装置所处的m个运动状态中的每个运动状态是否均影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若是，确认所述显示装置所处的运动状态影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    若否，确认所述显示装置所处的运动状态没有影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果。
  11. 根据权利要求8～10任一项所述的视频处理控制装置，其中，
    所述接收模块还被配置为接收所述显示装置所显示的视频，所述视频包括多帧图像数据；
    所述处理模块还被配置为：在所述根据所述显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果前，
    判断所述多帧图像数据中的连续k帧图像数据是否均保持一致，k为大于或等于2的整数；在所述连续k帧图像数据未保持一致的情况下，执行根据所述显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态，是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    所述控制模块还被配置为：在所述连续k帧图像数据均保持一致的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号。
  12. 根据权利要求11所述的视频处理控制装置，其中，
    所述处理模块还被配置为，在所述连续k帧图像数据未保持一致的情况下，执行根据显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果之前，根据所述显示装置的运动信息及其静止阈值，判断所述显示装置是否静止或近似静止；在所述显示装置处于运动状态的情况下，执行根据显示装置的运动信息与运动阈值，判断所述显示装置所处的运动状态是否影响所述显示装置采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的效果；
    所述控制模块还被配置为：在所述显示装置静止或相对静止的情况下，输出停止采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理的第一控制信号。
  13. 根据权利要求12所述的视频处理控制装置，其中，所述显示装置的运动信息包括所述显示装置在三维坐标系中的X轴角速度Gyro_X、Y轴角速度Gyro_Y和Z轴角速度Gyro_Z；所述相对静止阈值包括角速度阈值GYRO_Threshold；
    所述处理模块被配置为：
    判断所述显示装置的运动信息和所述静止阈值是否满足：
    Sqrt(Gyro_X 2+Gyro_Y 2+Gyro_Z 2)≤GYRO_Threshold；
    若是，确认所述显示装置静止或近似静止；
    若否，确认所述显示装置处于运动状态。
  14. 根据权利要求12或13所述的视频处理控制装置，其中，
    所述接收模块被配置为：在第二预设时间段内，逐一接收所述显示装置的n个运动信息，n为大于等于2的整数；
    所述处理模块被配置为：根据所述显示装置的n个运动信息中的每个运 动信息与静止阈值，逐一判断所述显示装置所处的n个状态中的每个状态是否均静止或近似静止；
    若是，确认所述显示装置静止或近似静止；
    若否，确认所述显示装置处于运动状态。
  15. 一种显示控制装置，包括：
    视频处理器，被配置为采用基于运动补偿的帧速率上转换处理方式对所述显示装置所显示的视频进行处理；
    如权利要求8～14中任一项所述视频处理控制装置，所述视频处理控制装置所包括的控制模块与所述视频处理器耦接，并向所述视频处理器输出控制信号。
  16. 一种显示装置，包括：
    运动感应单元；
    如权利要求15所述的显示控制装置，所述显示控制装置与所述运动感应单元耦接。
PCT/CN2019/127423 2019-01-02 2019-12-23 视频处理方法、视频处理控制装置、显示控制装置 WO2020140784A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/973,299 US11317054B2 (en) 2019-01-02 2019-12-23 Video processing method, video processing control apparatus and display control apparatus and display apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910002289.3A CN109725729B (zh) 2019-01-02 2019-01-02 图像处理方法及图像控制装置、显示控制装置和显示装置
CN201910002289.3 2019-01-02

Publications (1)

Publication Number Publication Date
WO2020140784A1 true WO2020140784A1 (zh) 2020-07-09

Family

ID=66298099

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127423 WO2020140784A1 (zh) 2019-01-02 2019-12-23 视频处理方法、视频处理控制装置、显示控制装置

Country Status (3)

Country Link
US (1) US11317054B2 (zh)
CN (1) CN109725729B (zh)
WO (1) WO2020140784A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109725729B (zh) * 2019-01-02 2021-02-09 京东方科技集团股份有限公司 图像处理方法及图像控制装置、显示控制装置和显示装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105979360A (zh) * 2015-12-04 2016-09-28 乐视致新电子科技（天津）有限公司 一种渲染图像的处理方法及装置
CN106598252A (zh) * 2016-12-23 2017-04-26 深圳超多维科技有限公司 一种图像显示调节方法、装置、存储介质及电子设备
CN107870666A (zh) * 2016-09-28 2018-04-03 腾讯科技（深圳）有限公司 一种终端控制方法及终端
CN109725729A (zh) * 2019-01-02 2019-05-07 京东方科技集团股份有限公司 图像处理方法及图像控制装置、显示控制装置和显示装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009207128A (ja) * 2008-01-29 2009-09-10 Sanyo Electric Co Ltd 表示装置及び表示方法
JP2010048976A (ja) * 2008-08-20 2010-03-04 Sony Corp 信号処理装置および信号処理方法
JP5372721B2 (ja) * 2009-12-11 2013-12-18 ルネサスエレクトロニクス株式会社 映像信号処理装置、方法及びプログラム
JP5698476B2 (ja) * 2010-08-06 2015-04-08 オリンパス株式会社 内視鏡システム、内視鏡システムの作動方法及び撮像装置
CN102665061A (zh) * 2012-04-27 2012-09-12 中山大学 一种基于运动矢量处理的帧率上转换方法及其装置
KR20170011870A (ko) * 2015-07-24 2017-02-02 삼성전자주식회사 콘텐트를 제공하는 전자 장치 및 방법
RU2632272C1 (ru) * 2016-04-04 2017-10-03 Общество С Ограниченной Ответственностью "Яндекс" Способ создания синтетического изображения
CN106874057B (zh) * 2017-04-11 2018-06-29 惠州学院 一种信息录入方法及其设备
CN108810649B (zh) * 2018-07-12 2021-12-21 深圳创维－Rgb电子有限公司 画质调节方法、智能电视机及存储介质
CN110956910B (zh) * 2018-09-26 2023-05-26 中强光电股份有限公司 投影装置及其投影方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105979360A (zh) * 2015-12-04 2016-09-28 乐视致新电子科技（天津）有限公司 一种渲染图像的处理方法及装置
CN107870666A (zh) * 2016-09-28 2018-04-03 腾讯科技（深圳）有限公司 一种终端控制方法及终端
CN106598252A (zh) * 2016-12-23 2017-04-26 深圳超多维科技有限公司 一种图像显示调节方法、装置、存储介质及电子设备
CN109725729A (zh) * 2019-01-02 2019-05-07 京东方科技集团股份有限公司 图像处理方法及图像控制装置、显示控制装置和显示装置

Also Published As

Publication number Publication date
CN109725729A (zh) 2019-05-07
US20210250544A1 (en) 2021-08-12
US11317054B2 (en) 2022-04-26
CN109725729B (zh) 2021-02-09

Similar Documents

Publication Publication Date Title
US11836289B2 (en) Use of eye tracking to adjust region-of-interest (ROI) for compressing images for transmission
US10775886B2 (en) Reducing rendering computation and power consumption by detecting saccades and blinks
US20190156794A1 (en) Real-time user adaptive foveated rendering
US10324523B2 (en) Rendering virtual images based on predicted head posture
US20170287112A1 (en) Selective peripheral vision filtering in a foveated rendering system
EP3447735A1 (en) Information processing device, information processing method, and program
US20170064157A1 (en) Camera-assisted display motion compensation
US11244427B2 (en) Image resolution processing method, system, and apparatus, storage medium, and device
WO2020003860A1 (ja) 情報処理装置、情報処理方法、及びプログラム
TWI622957B (zh) 改善圖像品質的方法及虛擬實境裝置
CN108241211B (zh) 一种头戴显示设备及图像渲染方法
WO2020140784A1 (zh) 视频处理方法、视频处理控制装置、显示控制装置
CN110728749B (zh) 虚拟三维图像显示系统及方法
WO2017005070A1 (zh) 显示控制方法及装置
TWI792535B (zh) 圖形處理方法和相關眼動追蹤系統
CN110209277A (zh) 一种仅基于瞳孔特征的头戴式视线跟踪方法
TWI495293B (zh) 用於圖像失真校正之方法及裝置
CN111103979A (zh) 一种基于视觉焦点的分区渲染方法及装置
CN207096928U (zh) 虚拟现实系统
US11232291B2 (en) Posture estimation system, posture estimation apparatus, error correction method, and error correction program
EP3662813B1 (en) Wearable device-compatible electrooculography data processing device, spectacle-type wearable device provided with same, and wearable device-compatible electrooculography data processing method
EP3363509A1 (en) Motion tracking apparatus and system
US11507185B1 (en) Electrooculography-based eye tracking using normalized electrode input
TWI632395B (zh) 虛擬實境裝置以及其畫面產生方法
EP4083926A1 (en) Information processing device, information processing method and information processing program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19906812

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19906812

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03.11.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19906812

Country of ref document: EP

Kind code of ref document: A1