RU2743956C1 - Method for creating vertically oriented video - Google Patents

Abstract

FIELD: video playback means.SUBSTANCE: loading video from server into player. Video content of the metadata specifying resolution and cropping parameters and corresponding to the individual user device is checked. If metadata are detected, a video frame is obtained and a focal point is set at its center. Field of view is determined proceeding from the frame and reduction or increase of video resolution within this field of view on the basis of metadata. Viewing area is displayed such that the vertical size of the view region corresponds to the vertical dimension of the user device, and the focus point remains in the center of the viewing area and in the center of the user device. When the user rotates the device, the repeated display of the frame is performed such that the focus point remains in the center of the device, and the horizontal size of the viewing area corresponds to the horizontal size of the user device, and portions of the screen above the viewing region and under it are transmitted in black. Further frames of video, layers and/or effects are continued to be rendered according to steps described above.EFFECT: technical result consists in improvement of efficiency of reproduced video for playback device.25 cl, 6 dwg

Description

Cross-references to related claims

[0001] This application is an ordinary Provisional Patent Application No. 62836426, filed April 19, 2019.

Background of the invention

The technical field to which the invention relates

[0002] The present invention relates to a system and method for playing video in a manner that optimizes a field of view (eg, a field of view in a player) for a specific resolution and screen orientation of a user device.

Prior art

[0003] Initially, when making films and (later) television content, there were only a few video formats and aspect ratios. Standard film was shot at 4: 3 and TV footage was shot at 16: 9. Both ratios were designed for horizontal orientation, as there was no need to create or play vertical video.

[0004] In recent years, with the proliferation of electronic devices, a problem has arisen when playing content with standard definition and aspect ratio, since many computer monitors, mobile phones and tablets have different aspect ratios and resolutions and can have vertical and horizontal orientation (orientation can be changed in playback time when turning the smartphone or tablet by the user). Videos recorded at only one resolution may not play well on some screens and orientations.

[0005] Accordingly, a video playback engine is needed that is automatically optimized for the user's device.

The essence of the invention

[0006] the Present invention relates to video playback at different resolutions and screen orientations that match the user's device and can largely eliminate the disadvantages of existing prior art.

[0007] In one embodiment of the invention, there is provided a method for playing video in a format that corresponds to a user device, the method comprising downloading a video from a server; uploading video to the player; checking for the presence of metadata in the video indicating the resolution and crop / stock settings that correspond to a specific user's device; if metadata is found, then (i) a frame is taken from the video and a focal point is set in the center of the frame; (ii) the view of the frame is limited and the resolution is reduced based on the metadata; (iii) the field of view is reproduced so that the vertical size of the field matches the vertical size of the user device, and the focal point remains in the center of the frame and in the center of the user device (note that the frame itself does not change - the original frame is usually used to form the field of view, while edges (top / bottom or left / right) are not displayed); (iv) when the device is rotated by the user, the frame is rendered so that the focal point remains in the center of the device and the horizontal size of the field of view matches the horizontal size of the user's device; and (v) rendering of subsequent video frames continues in accordance with clauses. (i) - (iv). Note that the video file itself does not change, only the viewing area changes.

[0008] If necessary, the user enters left margin, right margin, top margin, bottom margin, timecode, and / or frame number as metadata. If necessary, the user uses a software tool to visually define the displayed area. If necessary, the user sets the values of the parameters of the frame, which are similar to the values of the parameters of the previous frame. Metadata is structured as XML or JSON if necessary. If necessary, the user's device is any smartphone, tablet, desktop computer or TV. If necessary, the player detects changes in resolution in the user's device due to rotation and sets the rendering of the video in vertical or horizontal mode.

[0009] If necessary, the player determines the rotation of the device and displays only a specified / limited part of the video depending on the device's screen resolution (vertical or horizontal). If necessary, each field of view or part of the frame can be expanded or shrunk, or the entire video (displayed in the field of view) can be fitted to the "canvas" of the player. If necessary, regions in the video are specified based on context and differences between frames in pixel positions. If necessary, the player determines the focal point based on the most pronounced change of pixels in the frame. If necessary, the automatic selection of the displayed part of the video changes during playback.

[0010] If necessary, in response to the movement of the device by the user, the display area is shifted in the indicated direction, similar to 360 ° devices. The player supports two playback modes, if necessary: (1) the frame is completely embedded in the screen, and any stock space is rendered in black, and (2) the frame is stretched to its maximum size, and the rest of the frame remains outside the screen. If necessary, the playback mode is set by the user. If necessary, the playback mode is specified by metadata. If necessary, the timecode or frame number is used to rotate the frame and the field of view parameters. If necessary, as a result of turning the device, a smooth transition occurs with a step-by-step change of parameters from the first set to the second.

[0011] Additional features and advantages of the invention will be set forth in the description that follows, and in part will be obvious from the description, or may be learned by practice of the present invention. The advantages of the invention are realized and achieved by means of the structure detailed in the description, the claims, and in the accompanying drawings.

[0012] It is to be understood that the foregoing general description and the following full description are approximate and illustrative, and are intended to serve as the basis for the subsequent presentation of the invention in the application.

Brief description of graphic materials

[0013] Accompanying graphics are included to provide a deeper understanding of the invention and are an integral part of this document. They illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0014] Description of the drawings:

[0015] FIG. 1 depicts a situation in which the user has expanded the video to fit vertically, with part of the frame outside the device (image on the left), and on the right, where the video is horizontally fitted but black space remains in the vertical direction.

[0016] FIG. 2 shows an example of the layout for an aspect ratio of the device screen 18: 9.

[0017] FIG. 3 depicts a full frame (which is much larger than the device) that is displayed as a field of view so that the vertical dimension matches the device and portions of the field of view in the horizontal direction extend beyond the device;

[0018] FIG. 4 is a block diagram of the algorithm of the player;

[0019] FIG. 5 is a block diagram of an example of a mobile device that may be used in the invention.

[0020] FIG. 6 shows a block diagram of an example implementation of a mobile device.

[0021] FIG. 7 depicts the concept of content layers.

[0022] FIG. 8 shows a method for dividing a frame into fragments.

Detailed Description of the Preferred Embodiments of the Invention

[0023] The following will present detailed descriptions of preferred embodiments of the present invention, examples of which are depicted in the accompanying drawings.

[0024] The present invention provides a system and method for a multi-resolution video player. A method is proposed for marking and playing video streams depending on the screen resolution of the device. The proposed method has its own data format (hereinafter referred to as the VV format), which is created in the editor and can be played by the player. First, the author enters parameter values according to the frame size, frame position, and timecode. Parameter values can be stored in a database, in a file as metadata, or in some other form. For example, a file header can contain parameter values for several possible user devices (for example, iPhone 1, 2, 3, 5, 7, 10, Samsung Note 3, Samsung Note 4, iPad 2, etc.). Alternatively, the database can be maintained on the user's device, populating it by loading a parameter file from a known source, where the parameter file is created by the content author and uploaded to the server along with the video file itself.

[0025] After that, the declared player plays the video, showing only the area that corresponds to the previously entered parameter values. In addition, the player determines the resolution for each frame and increases or decreases it in the viewing area depending on the type of device, its orientation and screen resolution (according to the data in the VV format).

[0026] As an example, the user may enter the following values to set the display area: left margin, right margin, top margin, bottom margin, time code, or frame number. VV data can be provided for any type of device, any resolution, and / or for a vertical or horizontal resolution. The user can use a software tool to visually define the displayed area.

[0027] As an additional example, the user may set a VV data label for a specific frame, indicating that this frame has the same parameter values as the previous one. As another example, data can be structured in any text format like XML, JSON, db, etc. Data can be stored in any kind of storage, such as file metadata, database, file system, etc. The player can play videos on any device, including mobile devices, tablets, desktop computers and TVs.

[0028] As another example, the player can detect changes in resolution on the device by moving the screen to a vertical or horizontal mode. As another example, the player can detect device rotation by displaying video in a resized / reduced viewport based on the device's screen resolution (vertical or horizontal). As another example, the player can enlarge or reduce each viewport or all of the displayed video resized to fit the player's canvas. The frame itself does not change - usually the original frame is used to obtain the field of view, and the edges (top / bottom or left / right) are not displayed.

[0029] The translation to VV format can be performed without user input. Regions can be defined based on context and differences between frames in pixel colors and pixel group positions. The focus of the frame can be set based on the most pronounced change of pixels in the frame. VV formatting can be changed automatically during playback if necessary. The user can rotate and physically move the device so that the displayed area changes in the specified direction, similar to a 360 ° video.

[0030] In an exemplary embodiment, an author (ie, a user preparing data for a player) launches an editor (ie, a program for visual editing VV data) and marks the selection areas in each frame for vertical resolution. Then, the author downloads the video and VV data to a player (i.e., a video playback program that can read the VV data). Then the user of the mobile device launches the application and uses a special VV player, with the device turned horizontally, opens the video in it and watches it with the original resolution, for example, 1024 × 768 pixels.

[0031] After downloading the video, the player checks its VV data and sets the possible resolutions. During playback, the user can rotate the device from horizontal to vertical. The player records user activity and device orientation changes, and then checks if there is a type of vertical resolution in the VV data. If VV data for a vertical type device (ie screen orientation) is detected, the player reads it and applies it to each frame. For example, frame 37 has the following parameters in VV format: left stock: 220 pixels, right stock: 280 pixels, top stock: 4 pixels, bottom stock: 14 pixels. When the player reads frame 37, it only displays a 524 × 750 pixel area with appropriate borders from the edges at the original video resolution, while not displaying the rest. When displaying zoomed in / out frames as view areas, the player can zoom in or out to fit the player canvas.

[0032] If the next frame, i. E. frame 38 is marked with a special label "prf", then the same edge parameters will be used to display it. If frame 39 has other parameter values, the player will read them and display a different area. When the user changes the screen orientation, the VV player checks for the specified resolution (or orientation) in the VV data. If such information is available, the player will continue playing using this data set; otherwise, it will play the video at its original resolution.

[0033] In another embodiment, two videos at different resolutions may be combined within a single resolution “canvas”. When the first video is played back at the first resolution (and field of view parameters) and the device is rotated, the second video (with its own resolution and field of view parameters) is “skipped” by a predetermined time interval from the beginning (i.e., by the length of the already played part of the first video ) and is reproduced from this point on. At the next turn of the device, the first video is “skipped” for the corresponding time (the played part of the second video), and playback continues from this point.

[0034] In yet another embodiment, the player can be a software development kit that can be integrated into various environments as a code library.

[0035] An example of the VV format is shown below:

frame relative to screen

scaling relative to the frame, stretched to the minimum side of the screen

frame relative to position 0 in degrees

anonymized value change based on key action when the change is smooth within the key sync parameters. For example, during frames 40-50, the value will smoothly change from 123 to 111

[0036] The VV format includes a focus state conversion time code that is given in seconds relative to the previous frame.

[0037] For each frame state (a frame state is a set of parameters that describe the current distortion; for example, above the field of view parameters are specified in the frame state), the following data is usually provided:

• Stocks in relative units, where 1 is the width or height of the full frame. For example, Margin.R =. 1 means the right margin is 0.1 * (original frame width). This parameter is optional. If it is absent, then the frame is displayed in its original size.

• Focus Coordinates, the focal point that is located in the center of the viewing screen when the player is playing a video to support the same format on different screens. The focal point is specified in absolute or relative parameters. For example (0.3,0.5) in absolute terms - 30 percent from the left and 50 percent from the bottom; if the parameter is specified in absolute terms, then this value can also be (120pix, 100pix), i.e. the focal point is 120 pixels from the left and 100 pixels from the bottom

A "state transformation type" animation that describes transformation rules (fade, accelerate or fast, fade, or instant). Animation describes the transformation between frames (previous and current) that is displayed in the video. Animation can be represented in two ways: 1. A number of parameters that need to be changed, and the unit of time (or frame number) in which you want to apply the change; 2. Two coordinates, between which movement occurs, the VV algorithm creates intermediate values

[0038] FIG. 1 depicts a situation in which the user has expanded the video to fit vertically, with part of the frame outside the viewport (image on the left), and on the right, where the video is horizontally fitted, but there is black space in the vertical direction. FIG. 2 shows an example of the layout for an aspect ratio of the device screen 18: 9. FIG. 3 shows a full frame (which must be larger than the device's resolution), which is displayed as a field of view so that the vertical size matches the device, and parts of the field of view in the horizontal direction extend beyond the device. The focus of the frame is located exactly in the center of the device screen. The outline of the device is approximate.

[0039] FIG. 4 shows a block diagram of the algorithm of the player.

[0040] The player supports video playback in at least two modes: in mode 1, the frame is fitted to the screen, i. E. it fits completely within the screen, and the rest of the space is filled with black; in mode 2, the frame is stretched along its longest side, and the parts remaining outside the screen are not displayed. It is possible to set the mode by the user (pressing / touching the screen) or forced inclusion for a specific video.

[0041] Video playback process:

[0042] On the user's screen, the video frame is stretched by the player along the longest side.

[0043] The synchronous algorithm controls the position conversion.

[0044] When a frame change occurs during playback, the video is displayed with invisible edges outside the view area according to the ratio set in the markup, and then using the focal point, the resulting frame is positioned in the center of the user's screen. This point is located exactly in the physical center of the device screen, and the frame is aligned on it.

[0045] If a transformation animation is provided at the time of transformation, the values are gradually changed to the predetermined ones according to a certain algorithm.

[0046] The current player implementation is as follows:

[0047] Video and audio are reproduced by streaming full-frame video from the server. All markup and frame focus file is loaded before playback. The synchronous algorithm monitors the time and issues instructions for conversion. Streaming video playback occurs in standard mode, and at the same time, using a request via http, a 4v file with a description of the format is received, and in addition, all operations are performed according to the V.

[0048] The concepts described above also apply to other effects. The proposed format supports a number of additional effects and allows you to add layers over frames during video playback. These effects can be applied to the video layer and to the audio track. Effect information is added to the metadata.

[0049] Since all the effects are displayed only in the player, the original video remains unchanged. To use this method, instructions are added to the verticalization file that contain the required parameters.

[0050] An Effects tag containing an array of applied effects is added to the .4v metadata file. Each effect, in turn, has its own set of parameters such as timecode, type, etc.

[0051] An example of a portion of a verticalization file containing effects is shown below:

[0052] Applying effects can be thought of as working with layers. See FIG. 7, which depicts the concept of layers. The base frame is not shown, and instead the viewer sees a vertical frame, which is rendered into a separate layer. If the effect is related to simple frame distortion, then it is applied directly to the verticalized frames layer. If the effect requires additional content or some changes to an individual layer, a new layer is created and displayed on top of the verticalized frames layer.

[0053] The following are effects as part of a basic player. By controlling the layers and parameter sets indicated below, you can create new effects that you just need to load into the player and describe in the new version of the verticalization data. As a result, users will be able to overlay the video with any distortions or layers of changes that can be viewed during playback, or manipulate the display of video frames in real time.

1. Chromakey

This effect allows you to add a layer with a different video background, such as a color, still image, or other video. In other words, this layer is a canvas on which you can draw anything. It has the following parameters:

[0054] Replace - determining the position of a color pixel that will be the background color, and adding a different color of this pixel to the layer displayed on top of the video; and

[0055] Delta - Determination of the tolerance level for the position of the pixel.

[0056] A separate label (IMG = still image path; VIDEO = video file path; or COLOR = new color) is used to define the content and position of the text layer over the video for displaying the replaced selected text to the user.

2. Sound effects

[0057] The audio track can also be modified using the above approach, adding an additional audio track to the existing one. The user can apply the effect to the original audio track or, in some cases, replace it with another track. Usually, another audio stream is created (it has its own parameters, for example, volume and other parameters, and is output to one of the channels). The following illustrative sound effects are available:

[0058] - EFFECT OF LOW-FREQUENCY FILTER

[0059] Cutoff freq - cutoff frequency of the low-pass filter, Hz (range from 10.0 to 22000.0, default = 5000.0).

[0060] Resonance - Q-factor of the LPF resonance curve (range 1.0 to 10.0, default = 1.0).

[0061] - HIGH FREQUENCY FILTER EFFECT

[0062] Cutoff freq - cutoff frequency of the high-pass filter, Hz (range from 10.0 to 22000.0, default = 5000.0).

[0063] Resonance - Q-factor of the resonance HPF curve (range 1.0 to 10.0, default = 1.0).

[0064] These two effects are low and high pass filters, respectively. Their main parameter is the cutoff freq in Hz.

[0065] - ECHO EFFECT

[0066] Adds an echo to the sound track, described by the following parameters: delay - defines the echo delay from 10 to 5000 ms; and decay - specifies the decay of the echo from 0 to 1, where 1 is no decay and 0 is full decay. Options:

[0067] Delay - echo delay in ms (from 10 to 5000, default = 500).

[0068] Decay - Delayed echo decay, from 0 to 100%. 100% = no attenuation, 0% = full attenuation (i.e. simple one-line delay). Default = 50%

[0069] Max channels - max, number of channels

[0070] Drymix - volume of the original signal transmitted to the output, from 0 to 100%. Default = 100%

[0071] Wetmix - the volume of the echo signal transmitted to the output, from 0 to 100%. Default = 100%

[0072] - FLANGER

[0073] Adding a flanger described by the following parameters: Drymix (percent of original sound), Wetmix (percent of flanger), and Rate - a frequency from 0.1 to 20 Hz. Options:

[0074] Drymix - Percentage of the original signal transmitted to the output, from 0.0 to 100.0%. Default = 45%

[0075] Wetmix is the percentage of the flanger signal sent to the output. from 0.0 to 100.0%. Default = 55%

[0076] Depth - depth from 0.01 to 1.0. Default = 1.0.

[0077] Rate is a rate from 0.1 to 20 Hz. Default = 10 Hz

[0078] - DISTORTION EFFECT

[0079] Adds a distortion layer described by the Distortion parameter; 0 to 1, default = 0.5.

[0080] - HEIGHT

[0081] Adds an offset in pitch, which is described by the following parameters: pitch - pitch, a multiplier from 0.5 to 2.0; and FFT size - FFT size, from 256.0 to 4096.0.

[0082] These effects are examples of those that can be included in a basic player. The underlying technology also supports other effects that can be applied to video by adding them to the verticalization file. XML.

3. Aspect Ratio

[0083] Frames can be distorted algorithmically, and the layer can be cropped or resized in real time based on the specified parameters. These effects are removed after the video ends.

[0084] The parameters of these effects have the following arguments: X and Y, which determine the position of the frame relative to the center of the screen; W and H, which define the width and height of the frame. You can also add a CROP label to define layer stocks: left (L), right (R), top (T) and bottom (B).

4. Complex shots

[0085] To preserve a significant portion of the frame, it can be divided into fragments that are displayed in different parts of the screen, see 1, 2, 3 in FIG. 8. This effect allows you to put the user's device into a multi-screen mode, in which several fragments of the original frame are displayed simultaneously:

[0086] For example, a user screen can be divided into three sections, each of which displays a different portion of the original frame.

[0087] These parts of the screen are defined by the following parameters:

[0088] index is a unique screen identifier used to associate a fragment with its corresponding portion of the screen.

[0089] pos - the location of the virtual screen on the user's device, which is determined relative to the actual screen size (ie, the virtual screen width 0.1 is equal to 10% of the actual screen width of the device).

[0090] size is the size of the virtual screen relative to the actual screen size.

[0091] The content of the screen is set by the following parameters:

[0092] Index - a link to a part of the screen that will show the fragment; pos - the initial position of the fragment; crop - the size of the fragment.

[0093] Screens can be superimposed on each other, but in this case, each screen will be displayed according to its serial number, for example, screen 1 will be superimposed on screen 5.

5. Layers

[0094] Additional layers can contain external content, such as still images, video, or text, which can be used to add titles or change the viewing experience of the original video by adding layers.

[0095] Replacing the background

[00104] Applying additional sound to the player (ie, playing sound effects) (tone, frequency)

[00105] The verticalization format may also contain audio parameters.

[00115] Normalizing effect

[00116] Fade in time - time for the effect to fade in seconds (from 0 to 20000.0, default = 5000.0 seconds).

[00117] Lowest volume - the lowest volume (from 0.0 to 1.0, default = 0.10).

[00118] Maximum amp - maximum gain (20.0 to 100000.0, default = 20x).

[00119] Parametric Equalizer Effect

[00120] Center freq is the frequency in Hz at which the gain is applied (range 20.0 to 22000.0, default = 8000.0 Hz).

[00121] Octave Range is the number of octaves in which gain is applied (centered at the center frequency) (0.20 to 5.00, default = 1.0 octave).

[00122] Frequency Gain - applied gain (0.05 to 3.00, default = 1.00 - no gain).

[00123] Height Shift Effect

[00124] Pitch is a multiplier of pitch (0.5x to 2.0x, default 1.0x).

[00125] FFT Size - FFT size (from 256.0 to 4096.0, default = 1024.0).

[00126] Overlap - overlap (1 to 32, default = 4).

[00127] Max channels - the maximum number of channels (0 to 16, default = 0 channels).

[00128] Frame aspect ratio (cropping, resizing)

[00129] In real time, the frame may appear distorted by changing (eg cropping) the verticalized layer of the frame in accordance with the parameters of the selected effect.

Complex shot

[00141] The composite frame effect can be used to verticalize to display multiple parts of the same frame or different frames at the same time. For example, a dialogue between two characters can be shown on the same screen, with one part of the frame displayed in the upper half of the screen and the other part of the frame in the lower half of the screen.

[00157] Cutting or splicing parts of a movie: showing in real time only some parts of video frames using a cropping effect:

Extra layer / mask

[00165] Additional content layers can be superimposed on the video, which can completely overlap the verticalized video.

[00166] Additional layers can be video streams and still images. Still images can have an alpha channel. Video streams can have a background with a mask or color.

[00167] The verticalization format, together with the time codes of the layers, may contain their descriptions.

[00194] FIG. 5 is a block diagram of an example of a mobile device 59 in which the invention may be used. For example, mobile device 59 may be a personal digital assistant, cell phone, network device, camera, smartphone, Enhanced General Packet Radio Service (EGPRS) mobile phone, network base station, media player, navigation device, email device, game console, or a combination. two or more of these or other data processing devices.

[00195] In some embodiments, the mobile device 59 has a touchscreen 73. The touchscreen 73 may utilize liquid crystal display (LCD) technology, light emitting polymer display (LPD) technology, or other technology. The touchscreen 73 can respond to tactile contact with the user.

[00196] In some implementations, the touch screen 73 may include a multi-touch screen 73. The multi-touch screen 73 can, for example, handle multiple simultaneous touches, including processing data related to the pressure, degree, and / or position of each touch point. This processing facilitates interaction with the device through multiple fingers, chord commands, and other interactions. You can also use other touch screen technologies, such as a screen that makes contact with a stylus or other means.

[00197] In some implementations, the mobile device 59 may display one or more graphical user interfaces 73 to provide a user with access to various system objects and to communicate information to the user. In some implementations, the graphical user interface may include one or more display objects 74, 76. In the illustrated example, screen objects 74, 76 are graphical representations of system objects. Some examples of system objects include device functions, applications, windows, files, alerts, events, and other identifiable system objects.

[00198] In some implementations, the mobile device 59 may perform the functions of multiple devices, such as a telephone device designated as telephone object 91; an e-mail device specified as an e-mail object 92; a network data transmission device specified in the form of a web object 93; Wi-Fi base station devices (not shown); and a media processing device indicated as a media player object 94. In some implementations, certain screen objects 74, such as telephone object 91, email object 92, web object 93, and media player object 94, may be displayed on menu bar 95. In some implementations, device functionality may be accessed from a top-level graphical user interface such as the one depicted in the figure. When you touch one of the objects 91, 92, 93 or 94, the corresponding function can be activated.

[00199] In some implementations, the mobile device 59 may perform network distribution. For example, this functionality allows a user to use the mobile device 59 and its adjacent network while on the move. In particular, mobile device 59 can provide Internet access (eg, Wi-Fi) to other wireless devices in the vicinity. For example, mobile device 59 can be configured as a base station for one or more devices. Therefore, the mobile device 59 can provide network access to other wireless devices or deny access.

[00200] In some implementations, when a device function is called, the graphical user interface of the mobile device 59 is modified, enlarged, or replaced with another user interface or user interface elements to facilitate user access to certain functions associated with the corresponding device functionality. For example, in response to a user touching a telephone object 91, the touchscreen graphical user interface 73 may display objects associated with various functions of the telephone; similarly, when the email object 92 is tapped, the graphical user interface can display objects associated with various email functions; when the web object 93 is touched, the graphical user interface can display objects associated with various web navigation functions; upon touching the multiplayer object 94, the user interface can display objects associated with various media processing functions.

[00201] In some embodiments, the environment or state of the top-level graphical user interface can be returned by pressing a button 96 located near the bottom of the mobile device 59. In some embodiments, each of the respective device functionalities may have a corresponding "home" the screen objects that are displayed on the touch screen 73, and the restoration of the graphical user interface environment is possible by clicking on the "original" screen object.

[00202] In some embodiments, the top-level graphical user interface may include additional screen objects 76, for example, a short message service (SMS) object, a calendar object, a camera object, a calculator object, a stock object, a weather object, a map object, a note object. , a clock object, an address book object, a settings object, and an app store object 97. For example, touching an SMS display object may open an SMS environment with related functions; similarly, when a display object is selected, the corresponding environment or functionality of the object can be opened.

[00203] Additional and / or other screen objects may also be displayed on the graphical user interface. For example, if device 59 acts as a base station for other devices, then one or more "connection" objects may appear in the graphical user interface, indicating that there is a connection. In some embodiments of the invention, you can customize the displayed objects 76, for example, you can define the displayed objects 76, and you can download additional applications or other software that provides other functions and corresponding objects of the screen.

[00204] In some embodiments, the mobile device 59 may include one or more input / output devices and / or sensors. For example, speaker 60 and microphone 62 can be used to facilitate voice-activated functionality such as telephone and voice mail functions. In some embodiments, an up / down button 84 may be provided to adjust the volume of speaker 60 and microphone 62. Mobile device 59 may also have an on / off button 82 for audible indication of incoming phone calls. In some embodiments, a loudspeaker 64 may be provided for hands-free conversation, for example to perform hands-free functions. A headphone and / or microphone jack 66 may also be provided.

[00205] In some embodiments, a proximity sensor 68 may be provided to detect the proximity of the mobile device 59 to the user's ear and then disable the touch screen 73 to prevent accidental function calls. In some embodiments, the touch screen 73 can be turned off to further save power when the mobile device 59 is positioned near the user's ear.

[00206] Other sensors can also be used. For example, in some embodiments, the ambient light sensor 70 may be used to adjust the brightness of the touch screen 73. In some embodiments, the acceleration sensor 72 may be used to detect the movement of the mobile device 59 as indicated by the direction arrows. Accordingly, screen and / or media objects can be displayed according to an orientation determined by the sensor, such as vertical or horizontal. In some embodiments, the mobile device 59 may have circuits or sensors to determine location, for example, in a global positioning system (GPS) or others (for example, systems using Wi-Fi access points, television signals, cellular networks, uniform resource locators ( URL)). In some embodiments, a positioning system (eg, a GPS receiver) can be integrated into mobile device 59 or provided as a separate device that can communicate with mobile device 59 via an interface (eg port 90) to provide location-based services.

[00207] Mobile device 59 may also have a camera lens and sensor 80. In some embodiments, the camera lens and sensor 80 may be positioned on the rear surface of mobile device 59. The camera may capture still images and / or video.

[00208] Mobile device 59 may also have one or more wireless communication subsystems, for example, 802.11b / g communication device 86 and / or BLUETOOTH communication device 88. Other communication protocols can also be supported, including 802.x communication protocols (e.g. WiMax, Wi-Fi, 3G, LTE), Code Division Multiple Access (CDMA), Global System for Mobile Communications (GSM), Enhanced GSM for Data ( EDGE), etc.

[00209] In some embodiments, port 90 is used, such as a universal serial bus (USB) port, a PC port, or other wired port connection. For example, port 90 can be used to establish wired communications with other computing devices, such as communication devices 59, network access devices, a personal computer, printer, or other processing devices that can receive and / or transmit data. In some embodiments, port 90 allows the mobile device 59 to synchronize with the host device using one or more protocols, such as TCP / IP, HTTP, UDP, or another known protocol. In other embodiments, TCP / IP over USB can be used.

[00210] FIG. 6 is a block diagram 2200 of an example implementation of a mobile device 59. A mobile device 59 may include a memory interface 2202, one or more data processors, image processors and / or central processing units 2204, and a peripheral interface 2206. Memory interface 2202, one or more processors 2204 and / or peripheral interface 2206 can be separate components or embedded in one or more integrated circuits. The various components in the mobile device 59 can be connected via one or more communication buses or signal lines.

[00211] Sensors, devices, and subsystems for performing various functionalities may be coupled across a peripheral interface 2206. For example, motion sensor 2210, light sensor 2212 and proximity sensor 2214 for the orientation, illumination, and proximity functions described above may be connected via peripheral interface 2206. Other sensors 2216 may also be connected to peripheral interface 2206, such as positioning systems (GPS receiver), temperature sensor, biometric sensor, or other primary transducer, to provide appropriate functionality.

[00212] A camera subsystem 2220 and an optical sensor 2222, such as a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) optical sensor, can be used to perform camera functions such as taking photos or video clips.

[00213] Communication functions may be performed through one or more wireless systems 2224, which may include radio frequency receivers and transmitters and / or optical (eg, infrared) receivers and transmitters. The particular design and implementation of communications subsystem 2224 may depend on the communications networks in which mobile device 59 is intended to operate. For example, mobile device 59 may include communications subsystems 2224 designed to operate on a GSM network, GPRS network, EDGE network, Wi-Fi network, or WiMax, BLUETOOTH networks. In particular, wireless subsystems 2224 may include hosting protocols that allow device 59 to be configured as a base station for other wireless devices.

[00214] Audio subsystem 2226 may be associated with speaker 2228 and microphone 2230 to facilitate voice-activated functionality such as voice recognition, voice responses, digital recording, and telephony features.

[00215] I / O subsystem 2240 can include touch controller 2242 and / or other input controllers 2244. Touch controller 2242 can be connected to touch screen 2246. Touch screen 2246 and touch controller 2242 can, for example, detect contact and movement or stop when Multiple sensor technologies, including but not limited to capacitive, resistive, infrared, and SAW technologies, and other proximity sensor arrays or other elements to determine one or more points of contact with the 2246 touchscreen.

[00216] Other input controllers 2244 can be connected to other input / control devices 2248, such as one or more buttons, toggle switches, thumbwheel, infrared port, USB port, and / or pointer, such as a stylus. One or more buttons (not shown) can include an up / down button to adjust the 2228 speaker and / or 2230 microphone.

[00217] In one embodiment, pressing a button for one period of time unlocks the touch screen 2246; and by pressing the button for another period of time longer than the first press, the mobile device 59 can be turned on or off. The user can customize the functions of one or more buttons. The 2246 touchscreen can also be used for virtual or touch buttons and / or keyboard.

[00218] In some embodiments, the mobile device 59 may represent recorded audio or video files, such as MP3, AAC, and MPEG files. In other embodiments, the mobile device 59 may have an MP3 player function. Therefore, the mobile device 59 can have a 32-pin connector compatible with an MP3 player. Other I / O and control devices can also be used.

[00219] Memory interface 2202 may be associated with memory 2250. Memory 2250 may include high-speed random access memory and / or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or flash memory (such as NAND , NOR). The 2250 memory can store a 2252 operating system such as Darwin, RTXC, LINUX, UNIX, OS X, ANDROID, IOS, WINDOWS, or an embedded operating system such as VxWorks. Operating system 2252 may contain instructions for operating basic system services and performing hardware-dependent tasks. In some embodiments, operating system 2252 may be a kernel (eg, a UNIX kernel).

[00220] Memory 2250 may also store communication instructions 2254 for communicating with one or more additional devices, one or more computers, and / or one or more servers. Memory 2250 may include graphical user interface instructions 2256 for performing graphical user interface data processing, including presentation, navigation, and selection in an app store; instructions 2258 for processing sensor data and performing their functions; instructions 2260 for processing data and performing telephone functions; instructions 2262 for the implementation of processes and functions associated with electronic messages; instructions 2264 for implementing processes and functions associated with a web browser; instructions 2266 for the implementation of processes and functions related to the processing of media data; instructions 2268 for the implementation of processes and instructions related to GPS and navigation; instructions for the 2270 camera to perform its associated processes and functions; and / or other 2272 software instructions for performing other processes and functions.

[00221] All of the above instructions and applications may correspond to a set of instructions for performing one or more of the functions described above. These instructions do not need to be implemented as separate programs, procedures or modules. The 2250 memory may contain additional instructions or few instructions. In addition, various functions of the mobile device 59 may be implemented in hardware and / or software, including one or more signal processing and / or embedded special application circuits.

[00222] From the foregoing description of the preferred embodiment of the invention, it will be apparent to those skilled in the art that the described method and apparatus have a number of advantages.

[00223] It should also be understood that various modifications, adaptations, and alternative embodiments of the invention are possible within the scope and spirit of the present invention. The invention is defined by the claims below.

Claims (53)

1. A method for playing video in a format that corresponds to a user's device, this method includes the following operations: downloading video from the server; loading video into the player; checking for the presence of metadata in the video that sets the resolution and cropping parameters and corresponding to the individual user's device; if metadata is found then (i) obtaining from a video frame and setting a focal point in its center; (ii) determining the field of view based on the frame and decreasing or increasing the resolution of the video within this field of view based on the metadata; (iii) displaying the field of view such that the vertical size of the field of view matches the vertical size of the user's device, and the focal point remains in the center of the field of view and in the center of the user's device; (iv) when the user rotates the device, re-displaying the frame so that the focal point remains in the center of the device, and the horizontal size of the view area matches the horizontal size of the user's device, and the portions of the screen above and below the view area are rendered in black; and (v) the continuation of rendering subsequent video frames, layers and / or effects in accordance with paragraphs. (i) - (iv). 2. The method according to claim 1, characterized in that the user enters the left margin, the right margin, the upper margin, the lower margin, the time code and / or the frame number as metadata or data. 3. The method according to claim 1, wherein the user uses a software tool to visually define the displayed area. 4. The method according to claim 1, characterized in that the user sets the parameter values associated with the frame, similar to the values of the parameters associated with the previous frame. 5. The method according to claim 1, characterized in that the metadata is structured in the form of XML or JSON. 6. The method according to claim 1, characterized in that the user's device is any smartphone, tablet, desktop computer or TV. 7. The method according to claim 1, characterized in that the player detects changes in resolution in the user's device in connection with rotation and sets the rendering of video, layers, effects in vertical or horizontal mode. 8. The method according to claim 1, characterized in that the player detects the rotation of the device and displays the video in enlarged, reduced or original resolution within the viewing area depending on the device screen resolution (vertical or horizontal). 9. The method of claim 1, further comprising enlarging or reducing each view area or all of the video for insertion into the "canvas" of the player. 10. The method of claim 1, wherein the regions in the video are defined based on context and differences between frames in pixel positions. 11. The method according to claim 1, characterized in that the focal point is determined based on the most pronounced change of pixels in the frame. 12. The method according to claim 1, characterized in that the zoom in / out of the video in the field of view changes automatically during playback. 13. The method according to claim 1, characterized in that in response to the movement of the device by the user, the displayed area is shifted in the specified direction, similar to a 360 ° video. 14. The method of claim 1, wherein the player supports two playback modes: (1) the frame is fully embedded in the screen, and any stock space is transmitted in black, and (2) the frame is stretched to its maximum size, and the rest of the frame remains outside the screen. 15. The method according to claim 14, wherein the playback mode is specified by the user. 16. The method according to claim 14, wherein the playback mode is specified by metadata or data. 17. The method according to claim 1, characterized in that a timer is used to rotate the frame and crop parameters. 18. The method according to claim 1, characterized in that as a result of the rotation of the device, there is a gradual and smooth change in parameters from the first set to the second. 19. The method according to claim 1, characterized in that the metadata contains an effect label indicating the effects and their parameters. 20. The method of claim 19, wherein applying the effects comprises defining additional layers in the effect frame. 21. The method of claim 19, wherein the effects include displaying a layer with a color, still image, or other video that matches the background of the original video. 22. The method of claim 19, wherein the effects comprise a low pass sound effect, a high pass filter sound effect, an equalizer sound effect, a pitch change sound effect, a distortion effect, a flanger effect, a resonance effect, and / or an echo effect. 23. The method according to claim 19, wherein the effects include changing the aspect ratio or dividing the video frame into several frames within the field of view. 24. A method for playing video in a format that corresponds to a user's device, while this method includes the following operations: downloading video from the server; loading video into the player; loading a file with metadata specifying the resolution and cropping parameters for user devices; Perform the following operations using resolution and crop settings that are appropriate for the specific device you are using: (i) recognizing a frame from the video and determining a focal point in its center; (ii) determining the field of view based on the frame and reducing the resolution based on metadata or data; (iii) displaying the field of view such that the vertical size of the field of view matches the vertical size of the user device, and the focal point remains in the center of the frame and in the center of the user device; (iv) when the user rotates the device, re-display the frame so that the focal point remains in the center of the device, and the horizontal size of the field of view matches the horizontal size of the user's device, and the portions of the screen at the top and bottom of the field of view are rendered in black; and continuation of rendering layers of effects and subsequent frames of video in the field of view according to paragraphs. (i) - (iv). 25. A system for playing video in a format that corresponds to the user's device, while such a system includes: processor and memory associated with the processor, with the processor configured to do the following: downloading video from the server; loading video into the player; loading a file with metadata specifying the resolution and cropping parameters for user devices; performing the following operations using the resolution and cropping settings that are appropriate for the specific device of the user (i) recognizing a frame from the video and determining a focal point in its center; (ii) determining the field of view based on the frame and reducing the resolution based on metadata or data; (iii) displaying the field of view such that the vertical size of the field of view matches the vertical size of the user's device and the focal point remains in the center of the frame and in the center of the user's device; (iv) when the user rotates the device, re-displaying the frame so that the focal point remains in the center of the device, and the horizontal size of the field of view matches the horizontal size of the user's device, and the portions of the screen at the top and bottom of the field of view are rendered in black; and continuation of rendering layers of effects and subsequent frames of video in the field of view according to paragraphs. (i) - (iv).

Images