WO2016086494A1 - Video presentation method for smart mobile terminal - Google Patents

Video presentation method for smart mobile terminal Download PDF

Info

Publication number
WO2016086494A1
WO2016086494A1 PCT/CN2015/000297 CN2015000297W WO2016086494A1 WO 2016086494 A1 WO2016086494 A1 WO 2016086494A1 CN 2015000297 W CN2015000297 W CN 2015000297W WO 2016086494 A1 WO2016086494 A1 WO 2016086494A1
Authority
WO
WIPO (PCT)
Prior art keywords
mobile terminal
displacement
smart mobile
video
during
Prior art date
Application number
PCT/CN2015/000297
Other languages
French (fr)
Chinese (zh)
Inventor
钱晓炯
龚宗耀
董泽
Original Assignee
钱晓炯
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 钱晓炯 filed Critical 钱晓炯
Publication of WO2016086494A1 publication Critical patent/WO2016086494A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4314Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand

Definitions

  • the invention relates to a video presentation method of an intelligent mobile terminal.
  • the mobile phone needs to keep the mobile phone moving in a horizontal or vertical direction at a stable and uniform speed.
  • the mobile phone splicing and synthesizing the photos through a software method, which has a relatively high requirement for the photographer, and a slight instability may result in the stitching not being caused. Traces of nature.
  • viewers view a panoramic photo because the panorama is often long or wide, the viewer needs to slide the photo to view the photo panorama.
  • the technical problem to be solved by the present invention is to provide a video presentation method for an intelligent mobile terminal that allows a viewer to have the same immersive experience as a photographer when viewing a video file.
  • the technical solution adopted by the present invention to solve the above technical problem is: a video presentation method for an intelligent mobile terminal, characterized in that: when the video is captured by the intelligent mobile terminal, the displacement or/and rotation of the smart mobile terminal during the shooting process is recorded. Information, and then the displacement or/and rotation information of the smart mobile terminal during the shooting process is saved together with the captured video file; when the smart mobile terminal plays the aforementioned video file, the displacement of the smart mobile terminal during viewing or / and rotation information, if the displacement or/and rotation information during viewing matches the displacement or/and rotation information during shooting, the aforementioned video file is played.
  • the data collected by the acceleration sensor and the record of the intelligent mobile terminal are opened by the acceleration sensor device that can detect the rotational acceleration or/and the movement acceleration of the intelligent mobile terminal. Time, obtain the displacement or/and rotation information of the smart mobile terminal during the shooting process; when the smart mobile terminal plays the aforementioned video file, turn on the rotation acceleration or/and the movement acceleration of the smart mobile terminal that is self-contained in the mobile terminal.
  • the acceleration sensor device obtains the displacement or/and rotation information during the viewing process of the smart mobile terminal by using the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal.
  • the displacement accuracy detected by the acceleration sensor device is low, and only the approximate motion direction and displacement can be judged.
  • the image information collected by the camera can be analyzed for auxiliary processing, that is, using an intelligent mobile terminal.
  • Displacement or / and the direction and relative distance of the rotation information and then obtain the displacement information of the intelligent mobile terminal during the shooting process by the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal; when the smart mobile terminal plays the aforementioned video file Open the front camera of the intelligent mobile terminal, take the front picture, and also detect the direction and relative distance of the displacement information of the positioning process by the frame to the front picture, and then collect the data collected by the acceleration sensor, and the smart mobile terminal At the time of recording, the displacement information of the intelligent mobile terminal during the viewing process is obtained.
  • the displacement or/and rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is more than the displacement or/and during the shooting.
  • the rotation speed is fast, and the aforementioned video file is quickly played; if the displacement or/and rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is higher than during the shooting If the displacement or/and the rotation speed is slow, the slow motion lens puts the aforementioned video file; if the viewer stops moving, the video file is stopped.
  • the displacement information of the smart mobile terminal during the viewing process or the shooting process can be obtained by:
  • the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the acceleration sensor device of the intelligent mobile terminal collecting data is ti, and the acceleration of the acceleration sensor device of the intelligent mobile terminal instantaneously collecting ai, then the intelligent mobile terminal Instantaneous speed Displacement information of intelligent mobile terminal
  • a low-pass filter may be applied to the acceleration of calculating the displacement information, that is, the displacement information of the smart mobile terminal during the viewing process or the shooting process is obtained by:
  • the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the data collected by the accelerometer device of the intelligent mobile terminal is t i , and the acceleration of the accelerometer device of the intelligent mobile terminal instantaneously collected is a i , the calculation is low
  • the present invention also provides a certain tolerance to the deviation of the displacement or/and the rotation information, that is, the displacement or/and the rotation information during the viewing and the deviation of the displacement or/and the rotation information during the shooting are preset. Fault tolerance Within the range, it is considered that the displacement or/and rotation information during the viewing matches the displacement or/and rotation information during the shooting.
  • the present invention uses a user to move (rotate) a mobile phone to watch a video.
  • the video file playing interface is simultaneously The smart mobile terminal is prompted to shift or/and rotate information during the shooting process, and the viewer moves or rotates the smart mobile terminal according to the displacement or/and rotation information prompted by the video file playing interface.
  • the invention has the advantages that: in addition to saving audio and video information, the present invention saves displacement (rotation) information, allowing viewers to experience video from another new media dimension, by communicating The displacement or / rotation information, the displacement or / and rotation of the viewer in synchronization with the photographer improves the immersion when watching the video.
  • This new way of playing video is not a traditional time-based variable to display video pictures in frames, but to display pictures in frames according to displacement (rotation) variables; the present invention can replace the panoramic photo shooting and browsing experience, reducing the photographer
  • the stability requirements require that the panoramic angle of the shooting is arbitrarily large (a 360 can be supported), the shooting direction is arbitrary (up, down, left, right, and beveled), and the viewer's viewing experience is more rich and interesting.
  • FIG. 1 is a figure of a data collected by a mobile acceleration sensor when acquiring displacement information by using a mobile phone as an example in the embodiment of the present invention
  • FIG. 2 is a mobile phone moving route diagram of a mobile phone as an example in the embodiment of the present invention.
  • Figure 3 is a vector diagram of the displacement vector straight line of Figure 2;
  • FIG. 4 is a schematic diagram of a photographing process taking a mobile phone as an example in the embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a viewing process using a mobile phone as an example in the embodiment of the present invention.
  • FIG. 6 is a schematic diagram showing the rotation of a shooting or viewing process using a mobile phone as an example in the embodiment of the present invention.
  • FIG. 7 is a schematic diagram showing a video frame displayed by a mobile phone as an example in the embodiment of the present invention.
  • FIG. 8 is a schematic diagram of fault tolerance of rotating information using a mobile phone as an example in the embodiment of the present invention.
  • FIG. 9 is a schematic diagram of a screen display prompting by using a mobile phone as an example in the embodiment of the present invention.
  • FIG. 10 is a schematic diagram of another screen display prompting by using a mobile phone as an example in the embodiment of the present invention.
  • the invention provides a video presentation method for an intelligent mobile terminal, wherein the main feature is: when the video is captured by the intelligent mobile terminal, the displacement or/and rotation information of the smart mobile terminal during the shooting process is recorded, and then the smart mobile terminal is The displacement or/and rotation information during the shooting is saved with the captured video file; when the smart mobile terminal plays the aforementioned video file, the displacement or/and rotation information during the viewing process of the smart mobile terminal is also recorded, such as If the displacement or/and rotation information during viewing matches the displacement or/and rotation information during shooting, the aforementioned video file is played.
  • the above video file is played quickly; if the displacement or/and rotation information during the viewing does not match the displacement or/and rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is shifted compared to during the shooting or / And the rotation speed is slow, the slow lens puts the aforementioned video file; if the viewer stops moving, the above video file is stopped.
  • the acceleration sensor can know the direction and size of the acceleration motion of the smart mobile terminal in space.
  • Different intelligent mobile terminals have different acceleration sensors, which can measure the moving acceleration and rotational acceleration of the intelligent mobile terminal respectively.
  • the acceleration sensor system interface and data precision are also different.
  • the mainstream includes moving acceleration sensors (for measuring moving acceleration) and gyro. Instrument (used to measure the rotational acceleration), we discuss the mobile acceleration sensor with higher popularity in this implementation. Other more advanced sensors can obtain more accurate and sensitive displacement information, but the principle of interaction is the same.
  • the smart mobile terminal can be obtained by turning on the mobile acceleration sensor that can sense the movement acceleration of the intelligent mobile terminal and the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal. Displacement information during shooting; when the smart mobile terminal plays the aforementioned video file, the mobile acceleration sensor that can sense the moving acceleration of the intelligent mobile terminal, the data collected by the moving acceleration sensor, and the intelligent movement are turned on again. The time recorded by the terminal obtains the displacement during the viewing process of the smart mobile terminal.
  • the motion acceleration in the x, y, and z directions is periodically collected, wherein the z value reflects the direction of motion perpendicular to the plane of the smartphone screen, which can be ignored, and x and y indicate horizontal and vertical directions.
  • the acceleration is moved, and x and y can be understood as the coordinate decomposition of the oblique motion.
  • the initial velocity is 0, the displacement distance is 0, the data acquisition period is ti, and the instantaneous acceleration is a i (this acceleration has been removed).
  • the above displacement, velocity and acceleration are all vectors, that is, the angle and distance information are included, and the moving distance can be a curve. See S in Fig. 2, we can simplify it to a straight line vector, see the displacement vector in the coordinates in Fig. 3. S1, so just store a point of information.
  • S1 we set the angle of vector S1 to alpha and length to d.
  • S1 On the line from the origin to S1, we uniformly take points s0, s1, s2, ..., sn, as shown in Figure 2.a.
  • Vector S1 ".
  • the displacement accuracy detected by the moving acceleration sensor is very low, and only the approximate motion direction and displacement can be judged.
  • the image information collected by the camera can be analyzed for auxiliary processing. That is, when the video is captured by the smart mobile terminal, the direction and relative distance of the displacement or/and the rotation information during the shooting process are detected by the video image of the video file by the frame, and then the data collected by the acceleration sensor and the intelligent movement are detected.
  • the time recorded by the terminal obtains the displacement or/and rotation information of the smart mobile terminal during the shooting process; when the smart mobile terminal plays the aforementioned video file, the front camera of the smart mobile terminal is turned on, and the front picture is taken, also by the frame pair
  • the front picture detects the displacement of the bit view process and/or the direction and relative distance of the rotation information, and then obtains the displacement of the smart mobile terminal during the viewing process by the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal. And rotate the information.
  • the direction and relative distance of the displacement can be detected by the frame according to the frame. As shown in Fig. 4, although we cannot obtain the precise moving distance, we can obtain the relative displacement. Size does not care about the distance of the scene.
  • the video when the video is playing, we open the front camera at the same time, take the front picture, and detect the face. If the detected face and background are removed, the relative displacement information can be obtained in the same way, as shown in Figure 5. At this time, the played video is displayed.
  • the information displayed by the camera is displayed in the figure. The face is used as the positioning reference, and is not used as the information for judging the displacement.
  • the front camera collects the video to judge the displacement, and the non-display video. .
  • the background image captured by the front camera is different from the scene of the video being played, the displacement speed and distance error may be caused, but this does not affect the processing speed of the system in the same environment.
  • the displacement is directly judged by the entire video frame; if all the faces are detected without the background image, the playback process is not performed or some interesting prompts are provided, for example, the screen can directly display the front camera collection. Picture.
  • the smart mobile terminal can be obtained by turning on the gyroscope which can be used to sense the rotational acceleration of the intelligent mobile terminal, the data collected by the gyroscope, and the time recorded by the intelligent mobile terminal.
  • Rotation information during the shooting process when the smart video terminal plays the aforementioned video file, the gyroscope which is built in the mobile terminal and can sense the rotational acceleration of the intelligent mobile terminal, the data collected by the gyroscope, and the intelligent mobile terminal record are turned on again. The time to get the rotation information during the viewing process of the smart mobile terminal.
  • the gyroscope can obtain the rotation angle of the mobile phone.
  • the phone is picked up and the an angle is rotated with the body as the axis.
  • the angle can be projected into the planes of x, y, and z.
  • the change value is taken as a discrete sample of a0, a1, a2, ... an.
  • the playback frame is discrete, and the subscript n should be used to represent a continuous to discrete meaningful range of values.
  • the sampling of the acceleration state information by the mobile phone is also discrete, and some displacement points may not be collected when the moving speed is fast.
  • the value of the displacement needs to be matched with the frame to ensure that the entire video frame sequence is played when S1 is shifted. If the instantaneous displacement ds is detected during playback, and the video has a total of m frames, the video to be played is m*ds/ S1 frame, taking the integer part, leaving the decimal part to the back. It can be seen that each time the displacement sensing sample may correspond to the number of frames that need to be played may be less than 1, or may be large, such as tens of frames.
  • the time variable t of the video is replaced by the angle rotation variable a, that is, the original video is displayed by the time points of t0, t1, t2, ..., tn for each frame F0, F1, F2. .., Fn, is converted to display each frame at each of the rotation angle points a0, a1, a2, ..., an.
  • the logic of the time function of the playback frame mapped to the angle function is consistent.
  • a low-pass filter to calculate the acceleration of the displacement variable.
  • the expression is beta*a i +(1-beta)a i-1 , beta is greater than 0 and less than 1, and the smaller the beta is, the smaller the filter is. The stronger the value, the recommended value is 0.3. It is difficult for the user to move at a constant speed, and even there is jitter when the playback is played. Therefore, a buffer Jitter buffer is set, and the buffer size is recommended to be 10 to 20 frames.
  • the buffer inputs the sequence of the video file to be played.
  • the number and play timestamp, after smoothing the timestamp, the output plays the aforementioned video file, thereby eliminating jitter.
  • the deviation of the moving speed should be tolerated, that is, as long as the displacement or/and the rotation information during the viewing and the displacement or/and the rotation information during the shooting are within the preset tolerance range, the viewing process is considered
  • the displacement or/and rotation information in the match matches the displacement or/and rotation information during the shooting. For example, if the speed of movement during playback is within v_theta (recommended value is 50%), it can be played at the original rate. The motion beyond the tolerance range is fast forward or slow. If the displacement speed is too fast, consider skipping the frame. Play the video. In order to improve usability, we allow some deviation between the viewer and the recorder's displacement; as long as the displacement angle of the two can match within 45 degrees, it can respond. As shown in Fig.
  • the 45° range of the same displacement direction is the play area, the opposite direction is 45 degrees for the reverse area, and the other areas are the non-response area; we call this 45 degree range the fault tolerance angle r_theta, we can Adjust the r_theta angle size to control the matching accuracy of the two.
  • angle rotation is used to drive the video, starting from the ease of use, it is possible to consider only the rotation angles of the two directions of the left and right rotation, that is, the same direction rotation is normal playback, and the reverse rotation is reverse rotation.
  • this method needs to watch the video by the user moving or/and rotating the mobile phone, and it is important to provide an intuitive and simple interactive prompt.
  • the video file playing interface prompts the displacement or/and rotation information of the smart mobile terminal during the shooting process, and the viewer moves according to the displacement or/and rotation information prompted by the video file playing interface or Rotate the smart mobile terminal.
  • the arrow on the screen of the mobile phone not only indicates the direction of displacement or/and rotation, but also reflects the progress of the current video playback.
  • Figure 10 is another design. The orientation of the sphere in the circle indicates the direction of displacement or / and rotation.
  • the audio should also be played at the same time. Since the speed of the viewer and the recorder's displacement (rotation) may not be consistent, the played audio needs to be processed.
  • the innovation of the present invention is:
  • the video In addition to saving audio and video information, the video also saves the displacement (rotation) information, allowing the viewer to experience the video in another new media dimension;

Abstract

A video presentation method for a smart mobile terminal. The method comprises: when the smart mobile terminal is used to shoot a video, recording displacement or/and rotation information about the smart mobile terminal in a shooting process, and then saving the displacement or/and the rotation information about the smart mobile terminal in the shooting process together with a shot video file; and when the smart mobile terminal plays the above-mentioned video file, also recording the displacement or/and the rotation information about the smart mobile terminal in a viewing process, and if the displacement or/and the rotation information in the viewing process match the displacement or/and the rotation information in the shooting process, playing the above-mentioned video file. According to the present invention, when a video is shot, in addition to saving audio and video information, the displacement (rotation) information is further saved, so that a viewer can experience the video from another new media dimension. By means of the transmitted displacement or/and the rotation information, the immersion in viewing the video is improved due to the displacement or/and rotation actions of the viewer synchronous with those of a photographer.

Description

一种智能移动终端的视频呈现方法Video presentation method of intelligent mobile terminal 技术领域Technical field
本发明涉及一种智能移动终端的视频呈现方法。The invention relates to a video presentation method of an intelligent mobile terminal.
背景技术Background technique
通常用户拍摄全景照片时需要保持手机稳定匀速的在水平或垂直方向移动,手机通过软件的方法将各幅照片进行拼接合成,这对拍摄者有比较高的要求,稍不稳定就会导致拼接不自然的痕迹。观看者查看全景照片时,由于全景照片往往很长或很宽,观看者需要滑动照片进行浏览照片全景。Usually, when the user takes a panoramic photo, the mobile phone needs to keep the mobile phone moving in a horizontal or vertical direction at a stable and uniform speed. The mobile phone splicing and synthesizing the photos through a software method, which has a relatively high requirement for the photographer, and a slight instability may result in the stitching not being caused. Traces of nature. When viewers view a panoramic photo, because the panorama is often long or wide, the viewer needs to slide the photo to view the photo panorama.
同样,如果拍摄者用视频拍摄全景,观看者直接观看视频时由于缺乏代入感,只是直接观看这个视频而已。Similarly, if the photographer takes a video to capture a panorama, the viewer can watch the video directly because of the lack of substitution, just watching the video directly.
发明内容Summary of the invention
本发明所要解决的技术问题是针对上述现有技术提供一种能让观着者在观看视频文件时与拍摄者具有相同的沉浸体验的智能移动终端的视频呈现方法。The technical problem to be solved by the present invention is to provide a video presentation method for an intelligent mobile terminal that allows a viewer to have the same immersive experience as a photographer when viewing a video file.
本发明解决上述技术问题所采用的技术方案为:一种智能移动终端的视频呈现方法,其特征在于:采用智能移动终端拍摄视频时,记录下智能移动终端在拍摄过程中的位移或/和旋转信息,然后将智能移动终端在拍摄过程中的位移或/和旋转信息与拍摄的视频文件保存在一起;当智能移动终端播放前述视频文件时,也记录下智能移动终端的观看过程中的位移或/和旋转信息,如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配,则播放前述视频文件。The technical solution adopted by the present invention to solve the above technical problem is: a video presentation method for an intelligent mobile terminal, characterized in that: when the video is captured by the intelligent mobile terminal, the displacement or/and rotation of the smart mobile terminal during the shooting process is recorded. Information, and then the displacement or/and rotation information of the smart mobile terminal during the shooting process is saved together with the captured video file; when the smart mobile terminal plays the aforementioned video file, the displacement of the smart mobile terminal during viewing or / and rotation information, if the displacement or/and rotation information during viewing matches the displacement or/and rotation information during shooting, the aforementioned video file is played.
作为改进,采用智能移动终端拍摄视频时,通过开启智能移动终端中自带的能感知智能移动终端的转动加速度或/和移动加速度的加速度传感器设备,通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移或/和旋转信息;在智能移动终端播放前述视频文件时,再次开启移动终端中自带的能感知智能移动终端的转动加速度或/和移动加速度的加速度传感器设备,通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端的观看过程中的位移或/和旋转信息。As an improvement, when the video is captured by the smart mobile terminal, the data collected by the acceleration sensor and the record of the intelligent mobile terminal are opened by the acceleration sensor device that can detect the rotational acceleration or/and the movement acceleration of the intelligent mobile terminal. Time, obtain the displacement or/and rotation information of the smart mobile terminal during the shooting process; when the smart mobile terminal plays the aforementioned video file, turn on the rotation acceleration or/and the movement acceleration of the smart mobile terminal that is self-contained in the mobile terminal. The acceleration sensor device obtains the displacement or/and rotation information during the viewing process of the smart mobile terminal by using the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal.
在实际应用中,通过加速度传感器设备检测到的位移精度较低,只能判断出大概的运动方向和位移,为了提高精度,可通过分析摄像头采集的图像信息进行辅助处理,即:采用智能移动终端拍摄视频时,先通过按帧对视频文件的视频画面检测出拍摄过程中的 位移或/和旋转信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移信息;在智能移动终端播放前述视频文件时,打开智能移动终端的前置摄像头,拍摄前置画面,同样通过按帧对前置画面检测出位观看过程的位移信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在观看过程中的位移信息。In practical applications, the displacement accuracy detected by the acceleration sensor device is low, and only the approximate motion direction and displacement can be judged. In order to improve the accuracy, the image information collected by the camera can be analyzed for auxiliary processing, that is, using an intelligent mobile terminal. When shooting a video, first detect the video during the shooting by pressing the frame on the video screen of the video file. Displacement or / and the direction and relative distance of the rotation information, and then obtain the displacement information of the intelligent mobile terminal during the shooting process by the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal; when the smart mobile terminal plays the aforementioned video file Open the front camera of the intelligent mobile terminal, take the front picture, and also detect the direction and relative distance of the displacement information of the positioning process by the frame to the front picture, and then collect the data collected by the acceleration sensor, and the smart mobile terminal At the time of recording, the displacement information of the intelligent mobile terminal during the viewing process is obtained.
作为改进技术方案,如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中的位移或/和旋转速度比拍摄过程中的位移或/和旋转速度快,则快速播放前述视频文件;如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中位移或/和旋转速度比拍摄过程中位移或/和旋转速度慢,则慢镜头放前述视频文件;如果观看者停止移动,则停止播放前述视频文件。As an improved technical solution, if the displacement or/and rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is more than the displacement or/and during the shooting. The rotation speed is fast, and the aforementioned video file is quickly played; if the displacement or/and rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is higher than during the shooting If the displacement or/and the rotation speed is slow, the slow motion lens puts the aforementioned video file; if the viewer stops moving, the video file is stopped.
可以通过以下方式获得智能移动终端在观看过程或拍摄过程中的位移信息:The displacement information of the smart mobile terminal during the viewing process or the shooting process can be obtained by:
设智能移动终端的起始的速度为0,位移距离为0,智能移动终端的加速度传感器设备采集数据的周期为ti,智能移动终端的加速度传感器设备瞬时采集到的加速度为ai,则智能移动终端瞬时速度
Figure PCTCN2015000297-appb-000001
智能移动终端的位移信息
Figure PCTCN2015000297-appb-000002
Let the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the acceleration sensor device of the intelligent mobile terminal collecting data is ti, and the acceleration of the acceleration sensor device of the intelligent mobile terminal instantaneously collecting ai, then the intelligent mobile terminal Instantaneous speed
Figure PCTCN2015000297-appb-000001
Displacement information of intelligent mobile terminal
Figure PCTCN2015000297-appb-000002
为了提高播放时的顺滑度,可以对计算位移信息的加速度施加一个低通滤波,即:通过以下方式获得智能移动终端在观看过程或拍摄过程中的位移信息:In order to improve the smoothness during playback, a low-pass filter may be applied to the acceleration of calculating the displacement information, that is, the displacement information of the smart mobile terminal during the viewing process or the shooting process is obtained by:
设智能移动终端的起始的速度为0,位移距离为0,智能移动终端的加速度传感器设备采集数据的周期为ti,智能移动终端的加速度传感器设备瞬时采集到的加速度为ai,计算低通滤波后的加速度将ai’,ai’=0.3*ai+0.7*ai-1,智能移动终端瞬时速度
Figure PCTCN2015000297-appb-000003
智能移动终端的位移信息
Figure PCTCN2015000297-appb-000004
Let the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the data collected by the accelerometer device of the intelligent mobile terminal is t i , and the acceleration of the accelerometer device of the intelligent mobile terminal instantaneously collected is a i , the calculation is low The filtered acceleration will be a i ', a i '=0.3*a i +0.7*a i-1 , the instantaneous speed of the intelligent mobile terminal
Figure PCTCN2015000297-appb-000003
Displacement information of intelligent mobile terminal
Figure PCTCN2015000297-appb-000004
用户拍摄或观看时很难做到匀速的移动,甚至位移播放的时候还会存在抖动,为了消除抖动,在智能移动终端播放前述视频文件时,设定一个缓冲区,缓冲区大小为10~20帧,该缓冲区输入需要播放的视频文件的帧序列编号,在智能移动终端播放前述视频文件时,设定一个缓冲区,缓冲区大小为10~20帧,该缓冲区输入需要播放的视频文件的帧序列编号,然后按帧序列顺序输出播放前述视频文件。It is difficult for the user to move at a constant speed when shooting or watching, and even when the playback is played, there is still jitter. To eliminate the jitter, when the smart mobile terminal plays the video file, set a buffer with a buffer size of 10-20. Frame, the buffer enters the frame sequence number of the video file to be played. When the smart mobile terminal plays the video file, set a buffer with a buffer size of 10 to 20 frames. The buffer inputs the video file to be played. The frame sequence number is then output in the order of the frame sequence to play the aforementioned video file.
为了使用更方便,本发明还对位移或/和旋转信息的偏差提供一定容忍度,即:观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息偏差在预设的容错 范围内,即认为观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配。In order to be more convenient to use, the present invention also provides a certain tolerance to the deviation of the displacement or/and the rotation information, that is, the displacement or/and the rotation information during the viewing and the deviation of the displacement or/and the rotation information during the shooting are preset. Fault tolerance Within the range, it is considered that the displacement or/and rotation information during the viewing matches the displacement or/and rotation information during the shooting.
不同于通常的播放方式,本发明通过用户移动(旋转)手机来观看视频,为了提供直观简洁的交互提示,作为改进的技术方案,当智能移动终端播放前述视频文件时,同时在视频文件播放界面提示智能移动终端在拍摄过程中的位移或/和旋转信息,观看者根据视频文件播放界面提示的位移或/和旋转信息来移动或旋转智能移动终端。Different from the usual playing mode, the present invention uses a user to move (rotate) a mobile phone to watch a video. In order to provide an intuitive and simple interactive prompt, as an improved technical solution, when the smart mobile terminal plays the aforementioned video file, the video file playing interface is simultaneously The smart mobile terminal is prompted to shift or/and rotate information during the shooting process, and the viewer moves or rotates the smart mobile terminal according to the displacement or/and rotation information prompted by the video file playing interface.
与现有技术相比,本发明的优点在于:本发明拍摄视频时除了保存音视频信息外,还保存了位移(旋转)信息,可让观看者从另一个新的媒体维度体验视频,通过传达的位移或/旋转信息,观看者跟拍摄者同步的位移或/和旋转动作提高了观看视频时的沉浸感。这种全新的播放视频方式,不是传统的基于时间变量来按帧显示视频画面,而是按照位移(旋转)变量来按帧显示画面;本发明可代替全景照片的拍摄和浏览体验,降低拍摄者的稳定性要求,拍摄全景角度任意大(一圈360都可以支持),拍摄方向任意(上下左右、斜角都可以),观看者的观看体验也更加丰富和有趣。Compared with the prior art, the invention has the advantages that: in addition to saving audio and video information, the present invention saves displacement (rotation) information, allowing viewers to experience video from another new media dimension, by communicating The displacement or / rotation information, the displacement or / and rotation of the viewer in synchronization with the photographer improves the immersion when watching the video. This new way of playing video is not a traditional time-based variable to display video pictures in frames, but to display pictures in frames according to displacement (rotation) variables; the present invention can replace the panoramic photo shooting and browsing experience, reducing the photographer The stability requirements require that the panoramic angle of the shooting is arbitrarily large (a 360 can be supported), the shooting direction is arbitrary (up, down, left, right, and beveled), and the viewer's viewing experience is more rich and interesting.
附图说明DRAWINGS
图1为本发明实施例中以手机为例的采集位移信息时移动加速度传感器采集数据的坐标;1 is a figure of a data collected by a mobile acceleration sensor when acquiring displacement information by using a mobile phone as an example in the embodiment of the present invention;
图2为本发明实施例中以手机为例的手机移动路线图;2 is a mobile phone moving route diagram of a mobile phone as an example in the embodiment of the present invention;
图3为图2中位移矢量直线向量图;Figure 3 is a vector diagram of the displacement vector straight line of Figure 2;
图4为本发明实施例中以手机为例的拍摄过程示意图;4 is a schematic diagram of a photographing process taking a mobile phone as an example in the embodiment of the present invention;
图5为本发明实施例中以手机为例的观看过程示意图;FIG. 5 is a schematic diagram of a viewing process using a mobile phone as an example in the embodiment of the present invention; FIG.
图6为本发明实施例中以手机为例的拍摄或观看过程旋转示意图;FIG. 6 is a schematic diagram showing the rotation of a shooting or viewing process using a mobile phone as an example in the embodiment of the present invention; FIG.
图7为本发明实施例中以手机为例的观看过程显示视频帧的示意图;FIG. 7 is a schematic diagram showing a video frame displayed by a mobile phone as an example in the embodiment of the present invention; FIG.
图8为本发明实施例中以手机为例的旋转信息容错示意图;FIG. 8 is a schematic diagram of fault tolerance of rotating information using a mobile phone as an example in the embodiment of the present invention; FIG.
图9为本发明实施例中以手机为例的屏幕显示提示示意图;FIG. 9 is a schematic diagram of a screen display prompting by using a mobile phone as an example in the embodiment of the present invention; FIG.
图10为本发明实施例中以手机为例的另一种屏幕显示提示示意图。FIG. 10 is a schematic diagram of another screen display prompting by using a mobile phone as an example in the embodiment of the present invention.
具体实施方式detailed description
以下结合附图实施例对本发明作进一步详细描述。The invention will be further described in detail below with reference to the embodiments of the drawings.
本发明提供了一种智能移动终端的视频呈现方法,其主要特点是:采用智能移动终端拍摄视频时,记录下智能移动终端在拍摄过程中的位移或/和旋转信息,然后将智能移动终端在拍摄过程中的位移或/和旋转信息与拍摄的视频文件保存在一起;当智能移动终端播放前述视频文件时,也记录下智能移动终端的观看过程中的位移或/和旋转信息,如 果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配,则播放前述视频文件。The invention provides a video presentation method for an intelligent mobile terminal, wherein the main feature is: when the video is captured by the intelligent mobile terminal, the displacement or/and rotation information of the smart mobile terminal during the shooting process is recorded, and then the smart mobile terminal is The displacement or/and rotation information during the shooting is saved with the captured video file; when the smart mobile terminal plays the aforementioned video file, the displacement or/and rotation information during the viewing process of the smart mobile terminal is also recorded, such as If the displacement or/and rotation information during viewing matches the displacement or/and rotation information during shooting, the aforementioned video file is played.
当然,如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中的位移或/和旋转速度比拍摄过程中的位移或/和旋转速度快,则快速播放前述视频文件;如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中位移或/和旋转速度比拍摄过程中位移或/和旋转速度慢,则慢镜头放前述视频文件;如果观看者停止移动,则停止播放前述视频文件。Of course, if the displacement or/and rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is faster than the displacement or/and the rotation speed during the shooting , the above video file is played quickly; if the displacement or/and rotation information during the viewing does not match the displacement or/and rotation information during the shooting, and the displacement or/and the rotation speed during the viewing is shifted compared to during the shooting or / And the rotation speed is slow, the slow lens puts the aforementioned video file; if the viewer stops moving, the above video file is stopped.
以下详细阐述本实施离的不同实现的方式:The following describes in detail the different implementations of this implementation:
[通过加速度传感器检测位移信息][Detecting displacement information by acceleration sensor]
智能移动终端在移动中拍摄视频时,通过加速度传感器,可以知道智能移动终端在空间上的加速度运动的方向和大小。不同智能移动终端具有不同的加速度传感器,可以分别测量智能移动终端的移动加速度和转动加速度,其加速度传感器系统接口和数据精度也各有不同,主流包括移动加速度传感器(用于测量移动加速度)和陀螺仪(用于测量转动加速度),本实现方式中我们讨论的是普及度比较高的移动加速度传感器,其它更高级的传感器能够获得更精确更灵敏的位移信息,但交互的行为原则是相同的。When the smart mobile terminal captures video on the mobile, the acceleration sensor can know the direction and size of the acceleration motion of the smart mobile terminal in space. Different intelligent mobile terminals have different acceleration sensors, which can measure the moving acceleration and rotational acceleration of the intelligent mobile terminal respectively. The acceleration sensor system interface and data precision are also different. The mainstream includes moving acceleration sensors (for measuring moving acceleration) and gyro. Instrument (used to measure the rotational acceleration), we discuss the mobile acceleration sensor with higher popularity in this implementation. Other more advanced sensors can obtain more accurate and sensitive displacement information, but the principle of interaction is the same.
采用智能移动终端拍摄视频时,通过开启智能移动终端中自带的能感知智能移动终端移动加速度的移动加速度传感器,通过说动加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移信息;在智能移动终端播放前述视频文件时,再次开启移动终端中自带的能感知智能移动终端的移动加速度的移动加速度传感器,通过移动加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端的观看过程中的位移。When the video is captured by the intelligent mobile terminal, the smart mobile terminal can be obtained by turning on the mobile acceleration sensor that can sense the movement acceleration of the intelligent mobile terminal and the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal. Displacement information during shooting; when the smart mobile terminal plays the aforementioned video file, the mobile acceleration sensor that can sense the moving acceleration of the intelligent mobile terminal, the data collected by the moving acceleration sensor, and the intelligent movement are turned on again. The time recorded by the terminal obtains the displacement during the viewing process of the smart mobile terminal.
如图1,以智能手机为例,定时采集x、y、z方向上的移动加速度,其中z值体现了以垂直于智能手机屏幕平面的运动方向,可以忽略,x和y表示横向和纵向的移动加速度,并且x和y可理解为斜向运动的坐标分解,设起始的速度为0,位移距离为0,数据采集周期为ti,瞬时采集到的加速度为ai(这个加速度已经去除了重力),则瞬时速度
Figure PCTCN2015000297-appb-000005
位移
Figure PCTCN2015000297-appb-000006
As shown in Figure 1, taking a smartphone as an example, the motion acceleration in the x, y, and z directions is periodically collected, wherein the z value reflects the direction of motion perpendicular to the plane of the smartphone screen, which can be ignored, and x and y indicate horizontal and vertical directions. The acceleration is moved, and x and y can be understood as the coordinate decomposition of the oblique motion. The initial velocity is 0, the displacement distance is 0, the data acquisition period is ti, and the instantaneous acceleration is a i (this acceleration has been removed). Gravitational speed
Figure PCTCN2015000297-appb-000005
Displacement
Figure PCTCN2015000297-appb-000006
上述位移、速度和加速度都是矢量,即包含了角度和距离信息,移动的距离可以是曲线的,参见图2中的S,我们可以简化为直线向量,见图3中的坐标中的位移矢量S1,这样只要存储一个点的信息。在这里我们设矢量S1的角度为alpha,长度为d,在原点到S1的线上,均匀取点s0、s1、s2、...、sn,如图2.a中“被均匀分解的位移矢量S1”。The above displacement, velocity and acceleration are all vectors, that is, the angle and distance information are included, and the moving distance can be a curve. See S in Fig. 2, we can simplify it to a straight line vector, see the displacement vector in the coordinates in Fig. 3. S1, so just store a point of information. Here we set the angle of vector S1 to alpha and length to d. On the line from the origin to S1, we uniformly take points s0, s1, s2, ..., sn, as shown in Figure 2.a. Vector S1".
[通过视频图像检测位移信息] [Detecting displacement information by video image]
在实际应用中,通过移动加速度传感器检测到的位移精度非常低,只能判断出大概的运动方向和位移,为了提高精度,可通过分析摄像头采集的图像信息进行辅助处理。即:采用智能移动终端拍摄视频时,先通过按帧对视频文件的视频画面检测出拍摄过程中的位移或/和旋转信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移或/和旋转信息;在智能移动终端播放前述视频文件时,打开智能移动终端的前置摄像头,拍摄前置画面,同样通过按帧对前置画面检测出位观看过程的位移或/和旋转信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在观看过程中的位移或/和旋转信息。In practical applications, the displacement accuracy detected by the moving acceleration sensor is very low, and only the approximate motion direction and displacement can be judged. In order to improve the accuracy, the image information collected by the camera can be analyzed for auxiliary processing. That is, when the video is captured by the smart mobile terminal, the direction and relative distance of the displacement or/and the rotation information during the shooting process are detected by the video image of the video file by the frame, and then the data collected by the acceleration sensor and the intelligent movement are detected. The time recorded by the terminal obtains the displacement or/and rotation information of the smart mobile terminal during the shooting process; when the smart mobile terminal plays the aforementioned video file, the front camera of the smart mobile terminal is turned on, and the front picture is taken, also by the frame pair The front picture detects the displacement of the bit view process and/or the direction and relative distance of the rotation information, and then obtains the displacement of the smart mobile terminal during the viewing process by the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal. And rotate the information.
同样以手机为例,手机在移动中拍摄视频时,可以通过按帧对视频画面检测出位移的方向和相对距离,如图4,尽管我们无法获得精确的运动距离,但我们还是可以获得相对位移大小而不关心景物的远近。同样的,在视频播放时,我们同时打开前置摄像头,拍摄前置画面,检测人脸,如果去除检测到的人脸还有背景,则可用同样的方法获得相对位移信息,如图5,注意此时显示的是播放的视频,为方便说明问题,图中显示的是摄像头采集的信息,人脸作为定位基准,不作为判断位移的信息,前置摄像头采集视频用来判断位移,非显示视频。尽管由于前置摄像头采集的背景画面跟播放视频的景物的远近不同而会造成位移快慢和距离的误差,但这并不影响系统在各自相同环境下移动快慢的处理。如果检测不到人脸,则直接用整个视频画面判断位移;如果检测到全部是人脸而没有背景画面,则不做播放处理或提供一些有趣的提示,比如屏幕可以直接显示前置摄像头采集的画面。Similarly, in the case of a mobile phone, when the mobile phone is shooting a video on the mobile, the direction and relative distance of the displacement can be detected by the frame according to the frame. As shown in Fig. 4, although we cannot obtain the precise moving distance, we can obtain the relative displacement. Size does not care about the distance of the scene. Similarly, when the video is playing, we open the front camera at the same time, take the front picture, and detect the face. If the detected face and background are removed, the relative displacement information can be obtained in the same way, as shown in Figure 5. At this time, the played video is displayed. For the convenience of explanation, the information displayed by the camera is displayed in the figure. The face is used as the positioning reference, and is not used as the information for judging the displacement. The front camera collects the video to judge the displacement, and the non-display video. . Although the background image captured by the front camera is different from the scene of the video being played, the displacement speed and distance error may be caused, but this does not affect the processing speed of the system in the same environment. If the face is not detected, the displacement is directly judged by the entire video frame; if all the faces are detected without the background image, the playback process is not performed or some interesting prompts are provided, for example, the screen can directly display the front camera collection. Picture.
[通过陀螺仪检测转动信息][Detecting Rotation Information by Gyro]
鉴于目前的业界的技术水平难以获取精确的手机位移信息,即使通过图像检测位移也会有较大计算量,而获取准确的手机转动角度信息比较容易,因此我们可以采用以转动角度来驱动播放视频方式实现这个浸入式体验会更容易。即:采用智能移动终端拍摄视频时,通过开启智能移动终端中自带的能感知智能移动终端的转动加速度的陀螺仪,通过陀螺仪采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的旋转信息;在智能移动终端播放前述视频文件时,再次开启移动终端中自带的能感知智能移动终端的转动加速度的陀螺仪,通过陀螺仪采集的数据,及智能移动终端记录的时间,获得智能移动终端的观看过程中旋转信息。In view of the current industry level of technology, it is difficult to obtain accurate mobile phone displacement information, even if the displacement is detected by the image, there is a large amount of calculation, and it is relatively easy to obtain accurate information on the rotation angle of the mobile phone, so we can drive the video by the rotation angle. It's easier to implement this immersive experience. That is, when the video is captured by the intelligent mobile terminal, the smart mobile terminal can be obtained by turning on the gyroscope which can be used to sense the rotational acceleration of the intelligent mobile terminal, the data collected by the gyroscope, and the time recorded by the intelligent mobile terminal. Rotation information during the shooting process; when the smart video terminal plays the aforementioned video file, the gyroscope which is built in the mobile terminal and can sense the rotational acceleration of the intelligent mobile terminal, the data collected by the gyroscope, and the intelligent mobile terminal record are turned on again. The time to get the rotation information during the viewing process of the smart mobile terminal.
同样以手机为例,视频观看者拿起手机以身体为轴转动时,通过陀螺仪就可以获取手机旋转角度。为方便描述,我们定义手机不同时间采集到的角度值为a0、a1、a2、...an。如图6所示:正对拿起手机,以身体为轴旋转an度角。在三维空间的角度可以投影到x、y、z三个方向的平面,根据手机当前的姿态不同,我们只选取x或y方向的角度分量的 变化值,作为离散的a0、a1、a2、...an采样值。Similarly, in the case of a mobile phone, when the video viewer picks up the mobile phone and rotates the body as the axis, the gyroscope can obtain the rotation angle of the mobile phone. For the convenience of description, we define the angle values acquired by the mobile phone at different times a0, a1, a2, ... an. As shown in Figure 6, the phone is picked up and the an angle is rotated with the body as the axis. In the three-dimensional space, the angle can be projected into the planes of x, y, and z. According to the current posture of the mobile phone, we only select the angular component of the x or y direction. The change value is taken as a discrete sample of a0, a1, a2, ... an.
以上两个方式中,描述的是基于位移或旋转的播放,即:In the above two ways, the description is based on displacement or rotation playback, namely:
播放视频时,将视频的时间变量t,替换为位移变量s,即将原先的视频由t0、t1、t2、...、tn的时间点显示各帧F0、F1、F2、...、Fn,转变为在各个空间位移点s0、s1、s2、...、sn显示各帧。如图7,系统化描述为Fn=f(tn)=f′(sn),其中Fn为当前播放帧,tn为某个时间点,sn为某个空间点,f和f′为时间和空间映射到播放帧的函数。注意,时间和空间是连续的,播放帧是离散的,应此通过下标n来表示连续到离散的有意义的取值范围。实际上手机对加速度状态信息采样也是离散的,可能在移动速度较快的情况下无法采集到某些位移点。需要将位移的取值跟帧匹配起来,从而确保位移了S1时播放完全部视频帧序列,假设播放时检测到瞬时的位移ds,视频总共有m帧,则需要播放的视频为m*ds/S1帧,取整数部分,小数部分留给后面。可以看出,每次位移传感采样可能对应需要播放的帧数可能小于1,也可能很大,比如几十帧。When playing a video, replace the time variable t of the video with the displacement variable s, that is, display the frames F0, F1, F2, ..., Fn from the time point of t0, t1, t2, ..., tn. The transition to display each frame at each spatial displacement point s0, s1, s2, ..., sn. As shown in Figure 7, it is systematically described as F n =f(t n )=f'(s n ), where F n is the current playback frame, t n is a certain point in time, and s n is a certain spatial point, f and f' is a function that maps time and space to a playback frame. Note that the time and space are continuous, the playback frame is discrete, and the subscript n should be used to represent a continuous to discrete meaningful range of values. In fact, the sampling of the acceleration state information by the mobile phone is also discrete, and some displacement points may not be collected when the moving speed is fast. The value of the displacement needs to be matched with the frame to ensure that the entire video frame sequence is played when S1 is shifted. If the instantaneous displacement ds is detected during playback, and the video has a total of m frames, the video to be played is m*ds/ S1 frame, taking the integer part, leaving the decimal part to the back. It can be seen that each time the displacement sensing sample may correspond to the number of frames that need to be played may be less than 1, or may be large, such as tens of frames.
类似的,播放视频时,将视频的时间变量t,替换为角度旋转变量a,即将原先的视频由t0、t1、t2、...、tn的时间点显示各帧F0、F1、F2、...、Fn,转变为在各个旋转角度点a0、a1、a2、...、an显示各帧。如图7,其播放帧的时间函数映射到角度函数的逻辑是一致的。Similarly, when playing a video, the time variable t of the video is replaced by the angle rotation variable a, that is, the original video is displayed by the time points of t0, t1, t2, ..., tn for each frame F0, F1, F2. .., Fn, is converted to display each frame at each of the rotation angle points a0, a1, a2, ..., an. As shown in Figure 7, the logic of the time function of the playback frame mapped to the angle function is consistent.
为了提高播放时的顺滑度,我们对计算位移变量的加速度施加一个低通滤波,表达式为beta*ai+(1-beta)ai-1,beta大于0小于1,beta越小滤波越强,建议值取0.3。用户很难做到匀速的移动,甚至位移播放的时候还会存在抖动,因此设定一个缓冲区Jitter buffer,缓冲区大小建议为10~20帧,该缓冲区输入需要播放的视频文件的帧序列编号和播放时间戳,平滑时间戳后,输出播放前述视频文件,从而消除了抖动。In order to improve the smoothness during playback, we apply a low-pass filter to calculate the acceleration of the displacement variable. The expression is beta*a i +(1-beta)a i-1 , beta is greater than 0 and less than 1, and the smaller the beta is, the smaller the filter is. The stronger the value, the recommended value is 0.3. It is difficult for the user to move at a constant speed, and even there is jitter when the playback is played. Therefore, a buffer Jitter buffer is set, and the buffer size is recommended to be 10 to 20 frames. The buffer inputs the sequence of the video file to be played. The number and play timestamp, after smoothing the timestamp, the output plays the aforementioned video file, thereby eliminating jitter.
进一步,应该对移动速度的偏差有一定容忍度,即:只要观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息偏差在预设的容错范围内,即认为观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配。比如播放时移动的速度跟录制时速度偏差在v_theta以内(建议值50%),则可按原始速率播放,超出容错偏差范围的运动才快进或慢镜头如果位移速度过快,可以考虑跳帧播放视频。为了提高易用性,我们允许观看者和录制者的位移有一些偏差;只要两者位移角度能在45度范围内匹配即可响应。如图8,相同位移方向的前后45度范围内为播放区域,相反方向前后45度为倒放区域,其它区域为无响应区域;我们将这45度范围称为容错角r_theta,我们可以根据需要调整r_theta角大小来控制两者的匹配精度。 Further, the deviation of the moving speed should be tolerated, that is, as long as the displacement or/and the rotation information during the viewing and the displacement or/and the rotation information during the shooting are within the preset tolerance range, the viewing process is considered The displacement or/and rotation information in the match matches the displacement or/and rotation information during the shooting. For example, if the speed of movement during playback is within v_theta (recommended value is 50%), it can be played at the original rate. The motion beyond the tolerance range is fast forward or slow. If the displacement speed is too fast, consider skipping the frame. Play the video. In order to improve usability, we allow some deviation between the viewer and the recorder's displacement; as long as the displacement angle of the two can match within 45 degrees, it can respond. As shown in Fig. 8, the 45° range of the same displacement direction is the play area, the opposite direction is 45 degrees for the reverse area, and the other areas are the non-response area; we call this 45 degree range the fault tolerance angle r_theta, we can Adjust the r_theta angle size to control the matching accuracy of the two.
如果采用角度旋转驱动播放视频,从易用性出发,则可以只考虑左右转动两个方向的旋转角度,即同向旋转为正常播放,逆向旋转为倒放。If the angle rotation is used to drive the video, starting from the ease of use, it is possible to consider only the rotation angles of the two directions of the left and right rotation, that is, the same direction rotation is normal playback, and the reverse rotation is reverse rotation.
不同于通常的播放方式,本方法需要通过用户移动或/和旋转手机来观看视频,提供直观简洁的交互提示是比较重要的。当智能移动终端播放前述视频文件时,同时在视频文件播放界面提示智能移动终端在拍摄过程中的位移或/和旋转信息,观看者根据视频文件播放界面提示的位移或/和旋转信息来移动或旋转智能移动终端。同样以手机为例,参见图9,手机屏幕中箭头不但指示了位移或/和旋转的方向,同时也体现了当前视频播放的进度。图10是另一种设计方案,圈内的圆球的方位指示了位移或/和旋转的方向,如果位移时让圆球处于圆圈中心则移动或/和旋转速度正好,如果移动或/和旋转过快使得视频快进,则圆球会处于反方向的位置;外圈的粗线条表示视频播放进度。应当了解,这些交互设计主要需要体现两方面信息,一是位移或/和旋转方向,二是播放进度;Different from the usual playing mode, this method needs to watch the video by the user moving or/and rotating the mobile phone, and it is important to provide an intuitive and simple interactive prompt. When the smart mobile terminal plays the foregoing video file, the video file playing interface prompts the displacement or/and rotation information of the smart mobile terminal during the shooting process, and the viewer moves according to the displacement or/and rotation information prompted by the video file playing interface or Rotate the smart mobile terminal. Similarly, taking a mobile phone as an example, referring to FIG. 9, the arrow on the screen of the mobile phone not only indicates the direction of displacement or/and rotation, but also reflects the progress of the current video playback. Figure 10 is another design. The orientation of the sphere in the circle indicates the direction of displacement or / and rotation. If the ball is placed in the center of the circle when displaced, the movement or / and the rotation speed is just right, if moving or / and rotating Too fast to make the video fast forward, the ball will be in the opposite direction; the thick line of the outer circle indicates the progress of the video playback. It should be understood that these interaction designs mainly need to reflect two aspects of information, one is displacement or / and the direction of rotation, and the other is the progress of playback;
当然,在位移播放视频画面同时,也应同时播放音频,由于观看者和录制者位移(旋转)的速度不一定能达到一致,需要对播放的音频进行处理。有两种策略,一是变速不变调,二是直接变速变调;前者保证音频信息的真实性,后者的速度可变话程度更大也更有趣。我们可以根据实际需要选取不同的策略。Of course, while playing the video screen at the same time, the audio should also be played at the same time. Since the speed of the viewer and the recorder's displacement (rotation) may not be consistent, the played audio needs to be processed. There are two strategies, one is the shifting constant shift, and the other is the direct shifting tone; the former guarantees the authenticity of the audio information, and the speed of the latter is more variable and more interesting. We can choose different strategies according to actual needs.
综上,本发明的创新之处在于:In summary, the innovation of the present invention is:
a.拍摄视频时除了保存音视频信息外,还保存了位移(旋转)信息,可让观看者另一个新的媒体维度体验视频;a. In addition to saving audio and video information, the video also saves the displacement (rotation) information, allowing the viewer to experience the video in another new media dimension;
b.全新的播放视频方式,不是传统的基于时间变量来按帧显示视频画面,而是按照位移(旋转)变量来按帧显示画面;b. The new way of playing video, instead of the traditional time-based variable to display the video picture by frame, but according to the displacement (rotation) variable to display the picture by frame;
c.可代替全景照片的拍摄和浏览体验,降低拍摄者的稳定性要求,拍摄全景角度任意大(一圈360都可以支持),拍摄方向任意(上下左右、斜角都可以),观看者的观看体验也更加丰富和有趣。 c. It can replace the panoramic photo shooting and browsing experience, reduce the photographer's stability requirements, shoot the panoramic angle arbitrarily large (a 360 can support), the shooting direction is arbitrary (up and down, left and right, bevel can be), the viewer's The viewing experience is also richer and more interesting.

Claims (9)

  1. 一种智能移动终端的视频呈现方法,其特征在于:采用智能移动终端拍摄视频时,记录下智能移动终端在拍摄过程中的位移或/和旋转信息,然后将智能移动终端在拍摄过程中的位移或/和旋转信息与拍摄的视频文件保存在一起;A video presentation method for an intelligent mobile terminal, characterized in that: when the video is captured by the intelligent mobile terminal, the displacement or/and rotation information of the intelligent mobile terminal during the shooting process is recorded, and then the displacement of the smart mobile terminal during the shooting process is recorded. Or / and rotating information is saved with the captured video file;
    当智能移动终端播放前述视频文件时,也记录下智能移动终端的观看过程中的位移或/和旋转信息,如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配,则播放前述视频文件。When the smart mobile terminal plays the aforementioned video file, the displacement or/and rotation information during the viewing process of the smart mobile terminal is also recorded, if the displacement or/and the rotation information during the viewing and the displacement or/and rotation information during the shooting process Matching, the aforementioned video file is played.
  2. 根据权利要求1所述的智能移动终端的视频呈现方法,其特征在于:采用智能移动终端拍摄视频时,通过开启智能移动终端中自带的能感知智能移动终端的转动加速度或/和移动加速度的加速度传感器设备,通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移或/和旋转信息;在智能移动终端播放前述视频文件时,再次开启移动终端中自带的能感知智能移动终端的转动加速度或/和移动加速度的加速度传感器设备,通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端的观看过程中的位移或/和旋转信息。The video presentation method of the smart mobile terminal according to claim 1, wherein when the video is captured by the smart mobile terminal, the rotation acceleration or/and the movement acceleration of the intelligent mobile terminal that is self-contained in the smart mobile terminal is turned on. The acceleration sensor device obtains the displacement or/and rotation information of the smart mobile terminal during the shooting process by the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal; when the smart mobile terminal plays the video file, the mobile terminal is turned on again. The built-in accelerometer device capable of sensing the rotational acceleration or/and the moving acceleration of the intelligent mobile terminal, the data collected by the acceleration sensor, and the time recorded by the intelligent mobile terminal, obtain the displacement or/and the rotation during the viewing process of the smart mobile terminal information.
  3. 根据权利要求2所述的智能移动终端的视频呈现方法,其特征在于:采用智能移动终端拍摄视频时,先通过按帧对视频文件的视频画面检测出拍摄过程中的位移或/和旋转信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在拍摄过程中的位移或/和旋转信息;在智能移动终端播放前述视频文件时,打开智能移动终端的前置摄像头,拍摄前置画面,同样通过按帧对前置画面检测出位观看过程的位移或/和旋转信息的方向和相对距离,然后再通过加速度传感器采集的数据,及智能移动终端记录的时间,获得智能移动终端在观看过程中的位移或/和旋转信息。The video presentation method of the smart mobile terminal according to claim 2, wherein when the video is captured by the smart mobile terminal, the displacement or/and the rotation information during the shooting process are detected by the frame of the video image of the video file. Direction and relative distance, and then the data collected by the acceleration sensor and the time recorded by the intelligent mobile terminal, obtain the displacement or/and rotation information of the smart mobile terminal during the shooting process; when the smart mobile terminal plays the aforementioned video file, turn on the smart The front camera of the mobile terminal captures the front picture, and also detects the displacement of the bit view process and/or the direction and relative distance of the rotation information by the frame, and then the data collected by the acceleration sensor, and the smart movement. The time recorded by the terminal obtains the displacement or/and rotation information of the smart mobile terminal during the viewing process.
  4. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中的位移或/和旋转速度比拍摄过程中的位移或/和旋转速度快,则快速播放前述视频文件;如果观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息不匹配,且观看过程中位移或/和旋转速度比拍摄过程中位移或/和旋转速度慢,则慢镜头放前述视频文件;如果观看者停止移动,则停止播放前述视频文件。The video presenting method of the smart mobile terminal according to claim 1 or 2 or 3, wherein if the displacement or/and the rotation information during the viewing does not match the displacement or/and the rotation information during the shooting, and the viewing The displacement or/and rotation speed in the process is faster than the displacement or/and the rotation speed during shooting, and the aforementioned video file is quickly played; if the displacement or/and rotation information during the viewing and the displacement or/and rotation during the shooting The information does not match, and the displacement or/and the rotational speed during the viewing is slower than the displacement or/and the rotational speed during the shooting, the slow motion puts the aforementioned video file; if the viewer stops moving, the video file is stopped.
  5. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:通过以下方式获得智能移动终端在观看过程或拍摄过程中的位移信息:The video presentation method of the smart mobile terminal according to claim 1 or 2 or 3, wherein the displacement information of the smart mobile terminal during the viewing process or the shooting process is obtained by:
    设智能移动终端的起始的速度为0,位移距离为0,智能移动终端的加速度传感器设备采集数据的周期为ti,智能移动终端的加速度传感器设备瞬时采集到的加速度为ai, 则智能移动终端瞬时速度
    Figure PCTCN2015000297-appb-100001
    智能移动终端的位移信息
    Figure PCTCN2015000297-appb-100002
    Let the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the data collected by the accelerometer device of the intelligent mobile terminal is t i , and the acceleration of the accelerometer device of the intelligent mobile terminal instantaneously collected is a i , then the intelligence Mobile terminal instantaneous speed
    Figure PCTCN2015000297-appb-100001
    Displacement information of intelligent mobile terminal
    Figure PCTCN2015000297-appb-100002
  6. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:通过以下方式获得智能移动终端在观看过程或拍摄过程中的位移信息:The video presentation method of the smart mobile terminal according to claim 1 or 2 or 3, wherein the displacement information of the smart mobile terminal during the viewing process or the shooting process is obtained by:
    设智能移动终端的起始的速度为0,位移距离为0,智能移动终端的加速度传感器设备采集数据的周期为ti,智能移动终端的加速度传感器设备瞬时采集到的加速度为ai,计算低通滤波后的加速度将ai’,ai’=0.3*ai+0.7*ai-1,智能移动终端瞬时速度
    Figure PCTCN2015000297-appb-100003
    智能移动终端的位移信息
    Figure PCTCN2015000297-appb-100004
    Let the starting speed of the intelligent mobile terminal be 0, the displacement distance be 0, the period of the data collected by the accelerometer device of the intelligent mobile terminal is t i , and the acceleration of the accelerometer device of the intelligent mobile terminal instantaneously collected is a i , the calculation is low The filtered acceleration will be a i ', a i '=0.3*a i +0.7*a i-1 , the instantaneous speed of the intelligent mobile terminal
    Figure PCTCN2015000297-appb-100003
    Displacement information of intelligent mobile terminal
    Figure PCTCN2015000297-appb-100004
  7. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:在智能移动终端播放前述视频文件时,设定一个缓冲区,缓冲区大小为10~20帧,该缓冲区输入需要播放的视频文件的帧序列编号,然后按帧序列顺序输出播放前述视频文件。The video presentation method of the smart mobile terminal according to claim 1 or 2 or 3, wherein when the smart mobile terminal plays the video file, a buffer is set, and the buffer size is 10 to 20 frames, and the buffer is used. The area inputs the frame sequence number of the video file to be played, and then outputs and plays the aforementioned video file in the order of the frame sequence.
  8. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息偏差在预设的容错范围内,即认为观看过程中的位移或/和旋转信息与拍摄过程中的位移或/和旋转信息相匹配。The video presentation method of the intelligent mobile terminal according to claim 1 or 2 or 3, wherein the displacement or/and the rotation information during the viewing and the displacement or/and the rotation information during the shooting are deviated in the preset fault tolerance Within the range, it is considered that the displacement or/and rotation information during the viewing matches the displacement or/and rotation information during the shooting.
  9. 根据权利要求1或2或3所述的智能移动终端的视频呈现方法,其特征在于:当智能移动终端播放前述视频文件时,同时在视频文件播放界面提示智能移动终端在拍摄过程中的位移或/和旋转信息,观看者根据视频文件播放界面提示的位移或/和旋转信息来移动或旋转智能移动终端。 The video presentation method of the smart mobile terminal according to claim 1 or 2 or 3, wherein when the smart mobile terminal plays the video file, the video file playing interface prompts the displacement of the smart mobile terminal during the shooting process or / and rotation information, the viewer moves or rotates the smart mobile terminal according to the displacement or/and rotation information prompted by the video file play interface.
PCT/CN2015/000297 2014-12-05 2015-04-30 Video presentation method for smart mobile terminal WO2016086494A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410734878.8 2014-12-05
CN201410734878.8A CN104394451B (en) 2014-12-05 2014-12-05 A kind of video presentation method of intelligent mobile terminal

Publications (1)

Publication Number Publication Date
WO2016086494A1 true WO2016086494A1 (en) 2016-06-09

Family

ID=52612279

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/000297 WO2016086494A1 (en) 2014-12-05 2015-04-30 Video presentation method for smart mobile terminal

Country Status (2)

Country Link
CN (1) CN104394451B (en)
WO (1) WO2016086494A1 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104394451B (en) * 2014-12-05 2018-09-07 宁波菊风系统软件有限公司 A kind of video presentation method of intelligent mobile terminal
US20170064289A1 (en) * 2015-08-26 2017-03-02 Holumino Limited System and method for capturing and displaying images
CN106484091B (en) * 2016-03-16 2019-05-31 葫芦科技有限公司 A kind of direction synchronous method of headset equipment
CN105933792A (en) * 2016-06-03 2016-09-07 青岛海信移动通信技术股份有限公司 Video playing method and mobile terminal
CN106294631A (en) * 2016-08-03 2017-01-04 苏州美房云客软件科技股份有限公司 The method by gyroscope loadingsequence frame picture based on iOS system
CN106250504A (en) * 2016-08-03 2016-12-21 苏州美房云客软件科技股份有限公司 The method by gyroscope loadingsequence frame picture based on Android
CN106803856A (en) * 2016-12-09 2017-06-06 重庆零度智控智能科技有限公司 video playing control method and device
DE112018001867T5 (en) 2017-04-04 2019-12-12 Canon Kabushiki Kaisha Electronic device and its control method
CN107547930A (en) * 2017-06-02 2018-01-05 北京小嘿科技有限责任公司 A kind of video play controller and control method based on mobile device rotation detection
WO2019023991A1 (en) * 2017-08-02 2019-02-07 深圳传音通讯有限公司 Panoramic photograph displaying method and device based on intelligent terminal
CN107396018A (en) * 2017-08-09 2017-11-24 维沃移动通信有限公司 A kind of kinescope method, mobile terminal and computer-readable recording medium
CN107707817B (en) * 2017-09-27 2019-12-17 维沃移动通信有限公司 video shooting method and mobile terminal
CN108491138A (en) * 2018-01-16 2018-09-04 华南理工大学 A kind of mobile screen self-adapting regulation method based on Multi-sensor Fusion
CN108989660A (en) * 2018-06-08 2018-12-11 Oppo广东移动通信有限公司 Control method, control assembly, electronic equipment and the storage medium of slide assemblies
CN108924453A (en) * 2018-06-19 2018-11-30 中国联合网络通信集团有限公司 Pushing video streaming method and system, reporting unit
CN112738404B (en) * 2020-12-30 2022-11-25 维沃移动通信(杭州)有限公司 Electronic equipment control method and electronic equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1214513A (en) * 1997-10-15 1999-04-21 天津三维显示技术有限公司 Motion attitude record system for six-freedom video camera
US20070229397A1 (en) * 2004-03-03 2007-10-04 Volo, Llc Virtual reality system
KR20110103692A (en) * 2010-03-15 2011-09-21 한국과학기술연구원 Panorama video display device and display control method using user motion
CN103279187A (en) * 2013-05-09 2013-09-04 西安电子科技大学 Method for constructing multi-scene virtual panorama space and intelligent terminal
CN103562791A (en) * 2011-04-18 2014-02-05 眼见360股份有限公司 Apparatus and method for panoramic video imaging with mobile computing devices
CN104394451A (en) * 2014-12-05 2015-03-04 宁波菊风系统软件有限公司 Video presenting method for intelligent mobile terminal

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8576276B2 (en) * 2010-11-18 2013-11-05 Microsoft Corporation Head-mounted display device which provides surround video
CN109284264B (en) * 2012-09-16 2021-10-29 吴东辉 Image file format, image file generating method, image file generating device and application
CN103900477A (en) * 2012-12-28 2014-07-02 北京三星通信技术研究有限公司 Method for judging relative displacement of handheld device, and handheld device
CN103106335B (en) * 2012-12-29 2016-03-16 北京百度网讯科技有限公司 The method of 3-D view, device and mobile terminal is shown by mobile terminal records
CN103530799B (en) * 2013-10-22 2017-07-11 惠州Tcl移动通信有限公司 Realize that palm sees the browsing method and 3D house viewing systems in room according to 3D picture libraries

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1214513A (en) * 1997-10-15 1999-04-21 天津三维显示技术有限公司 Motion attitude record system for six-freedom video camera
US20070229397A1 (en) * 2004-03-03 2007-10-04 Volo, Llc Virtual reality system
KR20110103692A (en) * 2010-03-15 2011-09-21 한국과학기술연구원 Panorama video display device and display control method using user motion
CN103562791A (en) * 2011-04-18 2014-02-05 眼见360股份有限公司 Apparatus and method for panoramic video imaging with mobile computing devices
CN103279187A (en) * 2013-05-09 2013-09-04 西安电子科技大学 Method for constructing multi-scene virtual panorama space and intelligent terminal
CN104394451A (en) * 2014-12-05 2015-03-04 宁波菊风系统软件有限公司 Video presenting method for intelligent mobile terminal

Also Published As

Publication number Publication date
CN104394451B (en) 2018-09-07
CN104394451A (en) 2015-03-04

Similar Documents

Publication Publication Date Title
WO2016086494A1 (en) Video presentation method for smart mobile terminal
RU2679316C1 (en) Method and device for playback of video content from any location and at any time
US11798594B2 (en) Systems and methods for generating time lapse videos
US10750092B2 (en) Systems and methods for stabilizing videos
JP5769813B2 (en) Image generating apparatus and image generating method
US20230377605A1 (en) Systems and methods for generating time-lapse videos
JP5865388B2 (en) Image generating apparatus and image generating method
US9313405B2 (en) Image processing device, image processing method and program
US10116839B2 (en) Methods for camera movement compensation for gesture detection and object recognition
JPWO2013069047A1 (en) Image generating apparatus and image generating method
WO2017032336A1 (en) System and method for capturing and displaying images
JP2014053794A (en) Information processing program, information processing apparatus, information processing system, and information processing method
US10447926B1 (en) Motion estimation based video compression and encoding
US20180307352A1 (en) Systems and methods for generating custom views of videos
US11622076B2 (en) Systems and methods for stabilizing videos
US10165186B1 (en) Motion estimation based video stabilization for panoramic video from multi-camera capture device
CN107197339B (en) Display control method and device of film bullet screen and head-mounted display equipment
US20230308758A1 (en) Systems and methods for assessing stabilization of videos
US20230319405A1 (en) Systems and methods for stabilizing videos
US11336832B1 (en) Systems and methods for horizon leveling videos
US11985425B2 (en) Systems and methods for stabilizing videos
WO2023275920A1 (en) Trigger signal generation device and mobile terminal
TW201738649A (en) Corresponding method between panorama video and message and system thereof having cloud software calculator to obtain an angle value and a first time point from the cloud database, and calculate and search a picture of panorama video corresponding to the message
WO2019222059A1 (en) Systems and methods for providing rotational motion correction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15864523

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15864523

Country of ref document: EP

Kind code of ref document: A1