WO2014188235A1

WO2014188235A1 - Creation of a cinemagraph file

Info

Publication number: WO2014188235A1
Application number: PCT/IB2013/054290
Authority: WO
Inventors: Lasse Laaksonen; Arto Lehtiniemi; Mikko Tammi
Original assignee: Nokia Inc
Current assignee: Nokia Inc
Priority date: 2013-05-24
Filing date: 2013-05-24
Publication date: 2014-11-27
Anticipated expiration: 2015-11-24

Abstract

An apparatus enables a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface. The apparatus further models content of the at least one animated portion of the cinemagraph in a vector graphic format, and uses the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition. The apparatus further creates a cinemagraph file for the cinemagraph with the adjusted motion.

Description

Creation of a cinemagraph file

FIELD OF THE DISCLOSURE The invention relates to the field of cinemagraphs and more specifically to the creation of a cinemagraph file.

BACKGROUND Cinemagraphs constitute a mixture between a photograph and a video. They provide the illusion that a viewer is watching a still image that contains some movement or alternatively a video that contains many static elements. The selection of the moving and static elements is typically made such that the cinemagraph provides a surprise for the viewer or that it enhances the mood of the images. Cinemagraphs are well suited for creating short motion sequences to make the still images more life-like and interesting or to draw the attention of the viewer to a particular aspect of the image.

Cinemagraphs can be created manually based on a sequence of captured pictures using a dedicated application. The sequence of pictures can be either a sequence of photographs or a sequence of images represented by video frames. Cinemagraphs can also be created automatically.

For creating a cinemagraph, desired moving elements may be selected from the sequence of pictures. One of the pictures may be selected for the still parts of the cinemagraph, and the moving elements may be filtered out from this picture. The animation for these elements may be created based on a plurality of the pictures. The animation can be provided as a sequence of frames that is repeated in a loop. The plurality of the pictures that form the basis for the animation may be selected for example such that the result appears as a seamless loop of a movement.

SUMMARY OF SOME EMBODIMENTS OF THE INVENTION

A method is described which comprises enabling, by an apparatus, a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface. The method further comprises modeling, by the apparatus, content of the at least one animated portion of the cinemagraph in a vector graphic format, and using the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition. The method further comprises creating, by the apparatus, a cinemagraph file for the cinemagraph with the adjusted motion.

Moreover a first apparatus is described, which comprises means for realizing the actions of the presented method. The means of the apparatus can be implemented in hardware and/or software. They may comprise for instance at least one processor for executing computer program code for realizing the required functions, at least one memory storing the program code, or both. Alternatively, they could comprise for instance circuitry that is designed to realize the required functions, for instance implemented in a chipset or a chip, like an integrated circuit. In general, the means may comprise for instance one or more processing means.

Moreover a second apparatus is described, which comprises at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause at least one apparatus at least to perform the actions of the presented method.

Any of the described apparatuses may be a module or a component for a device, for example a chip. Alternatively, any of the mentioned apparatuses may be a mobile or stationary device. Any of the described apparatuses may further comprise only the indicated components or one or more additional components.

In certain embodiments, the described methods are information providing methods, and the described apparatuses are information providing apparatuses.

In certain embodiments of the described methods, the methods are methods for creating cinemagraph files. In certain embodiments of the described apparatuses, the apparatuses are apparatuses for creating cinemagraph files. Moreover, a system is described, which comprises any of the presented apparatuses as a first device and at least one other device, for example a device comprising a camera.

Moreover a non-transitory computer readable storage medium is described, in which computer program code is stored. The computer program code causes at least one apparatus to perform the actions of the presented method when executed by at least one processor.

The computer readable storage medium could be for example a disk or a memory or the like. The computer program code could be stored in the computer readable storage medium in the form of instructions encoding the computer-readable storage medium. The computer readable storage medium may be intended for taking part in the operation of a device, like an internal or external hard disk of a computer, or be intended for distribution of the program code, like an optical disc. It is to be understood that also the respective computer program code by itself has to be considered an embodiment of the invention. The computer program code could also be distributed to several computer readable storage mediums.

It is to be understood that the presentation of the invention in this section is merely by way of example and non-limiting.

Other features of the present invention will become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the invention, for which reference should be made to the appended claims. It should be further understood that the drawings are not drawn to scale and that they are merely intended to conceptually illustrate the structures and procedures described herein. BRIEF DESCRIPTION OF THE FIGURES

Fig. 1 is a schematic block diagram of an example embodiment of an apparatus;

Fig. 2 is a flow chart illustrating an example embodiment of a method;

Fig. 3 is a schematic block diagram of an example embodiment of a mobile device; Fig. 4 is a schematic block diagram of an example embodiment of a system;

Fig. 5 is a flow chart illustrating an example operation in the mobile device of Figure 3 or in the system of 4;

Fig. 6 is a sketch of a first example of a cinemagraph;

Fig. 7 is a sketch of the first example of a cinemagraph with examples of associated pivot points and an example of an associated movement vector;

Fig. 8 is a sketch of the first example of a cinemagraph illustrating a movement of pivot points by a user;

Fig. 9 is a sketch of the first example of a cinemagraph illustrating a change of a

movement trail by a user;

Fig. 10 are sketches of a second example of a cinemagraph illustrating movements of a pivot point by a user;

Fig. 1 1 is a schematic block diagram of an example embodiment of an apparatus;

Fig. 12 is a schematic block diagram of an example embodiment of an apparatus; and Fig. 13 schematically illustrates example removable storage devices.

DETAILED DESCRIPTION OF THE FIGURES

Figure 1 is a schematic block diagram of an example embodiment of an apparatus. Apparatus 100 comprises a processor 101 and, linked to processor 101 , a memory 102. Memory 102 stores computer program code for creating a cinemagraph file. Processor 101 is configured to execute computer program code stored in memory 102 in order to cause an apparatus to perform desired actions. Memory 102 is thus an example embodiment of a non-transitory computer readable storage medium, in which computer program code is stored.

Apparatus 100 could be a mobile device like a camera, a mobile terminal, a mobile computing device or a laptop, but it could also be a stationary device like a personal computer (PC). Apparatus 100 could equally be a component, like a chip, circuitry on a chip or a plug-in board, for any device. Optionally, apparatus 100 could comprise various other components, like a camera, a communication interface configured to enable an exchange of data with other apparatuses, a user interface like a touchscreen, a further memory, a further processor, etc.

An operation of an apparatus will now be described with reference to the flow chart of Figure 2. The operation is an example embodiment of a method according to the invention. Processor 101 and the program code stored in memory 102 cause an apparatus to perform the operation when the program code is retrieved from memory 102 and executed by processor 101. The apparatus that is caused to perform the operation can be apparatus 100 or some other apparatus, for example but not necessarily a device comprising apparatus 100.

The apparatus enables a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface, (action 201)

The apparatus furthermore models content of the at least one animated portion of the cinemagraph in a vector graphic format, and uses the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition, (action 202) It is to be understood that all or part of the content of the at least one animated portion of the cinemagraph can be modeled in a vector graphic format. The apparatus furthermore creates a cinemagraph file for the cinemagraph with the adjusted motion, (action 203)

The invention proceeds from the consideration that it may be useful to enable a user to further modify a cinemagraph that has been created in a conventional manner.

Certain embodiments of the invention therefore provide that a user is enabled to change a characteristic of a motion in a cinemagraph. The change defined by the user is implemented in the cinemagraph using a vector graphic representation of concerned content of the cinemagraph. The motion in the cinemagraph may be adjusted by partly or entirely replacing the original animated portion or by supplementing the original animated portion. Replacing an original animated portion completely may have the effect that changes in quality of a presented moving element during the playback of the cinemagraph are avoided.

Enabling a user to change the characteristic of a motion may have the effect that more advanced motion patterns, including unnatural motion patterns, can be achieved and that a longer, dynamically developing movement can be obtained. Certain embodiments may thus allow creating motion beyond captured movement in a cinemagraph in a simple and user friendly way. Without limiting the scope of the claims, this may have the effect of providing a novel experience with cinemagraphs that add value to existing solutions. Making use of a vector graphic representation of concerned content for implementing the change may have the effect that the change can be realized in a particularly simple manner with a good quality. A simple pendulum, for instance, is basically a straight line that includes rod and weight and that may be represented by a single vector. Such a vector can be easily moved and/or extended in response to a user input.

Apparatus 100 illustrated in Figure 1 and the method illustrated in Figure 2 may be implemented and refined in various ways.

In an example embodiment, adjusting the motion in the at least one animated part of the cinemagraph may comprise adjusting the modeled content in vector graphic format. Creating the cinemagraph file may then comprise converting the adjusted content in vector graphic format into a raster format for combining the adjusted content with non-adjusted portions of the cinemagraph. Without limiting the scope of the claims, this may have the effect that the motion can be changed efficiently, while the created cinemagraph file may be in a conventional raster format, like an animated graphics interchange format (GIF) file.

In an example embodiment, the motion in the at least one animated portion of a cinemagraph is analyzed automatically, and at least one adjustment element defining a characteristic of the motion is determined automatically. The user may then be enabled to define the desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph by presenting the at least one determined adjustment element via a user interface and by enabling the user to modify at least one presented adjustment element. Without limiting the scope of the claims, this may have the effect that it is particularly easy for the user to define a change of at least one characteristic of a motion in at least one animated portion of a cinemagraph.

It is to be understood, however, that in another example embodiment the user may be enabled to define the desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph by adding an adjustment element defining a characteristic of the motion via the user interface. Without limiting the scope of the claims, this may have the effect that the required computation power may be reduced and that the cinemagraph file may be created faster. It may further have the effect that the user may have a greater flexibility in changing the characteristics of a motion.

Adjustment elements could comprise a representation of fixed reference structures of a movement, movement vectors defining an extent of oscillating movements, motion trails defining a path of a particular object, and any other representation that is suited to define a characteristic of a motion in a cinemagraph.

In an example embodiment, enabling a user to define at least one fixed reference structure for the motion and to cause a presentation of a representation of the at least one fixed reference structure on a screen at a location desired by a user. Such a fixed reference structure could be a pivot for the motion, like a pivot point for an oscillating or rotating movement or a pivot axis for an oscillating or rotating movement. It could equally be a more complex structure, though. Alternatively or in addition, an example embodiment could comprise automatically determining a fixed reference structure for the motion, presenting a representation of the fixed reference structure to the user on a screen, and enabling the user to modify the representation of the fixed reference structure. Again, such a fixed reference structure could be a pivot for the motion or a more complex structure. Alternatively or in addition, an example embodiment could comprise automatically determining a movement vector for the motion in the at least one animated portion, presenting a representation of the movement vector to the user on a screen, and enabling the user to modify the representation of the movement vector. Alternatively or in addition, an example embodiment could comprise enabling a user to define a movement trail for an object on a screen. The movement trail could be defined for instance by defining a desired final position of the object, which could then have to be reached on the shortest way, or it could be defined by drawing the entire trail or a plurality of stations of the trail. Alternatively or in addition, an example embodiment could comprise automatically determining a movement trail of an object in the at least one animated portion, presenting a representation of the movement trail to the user on a screen, and enabling the user to change the representation of the movement trail. Alternatively or in addition, an example embodiment could comprise enabling a user to define on a screen elements that are allowed to be subject to scaling in order to enable the desired change of at least one characteristic of a motion. Without limiting the scope of the claims, this may allow taking account of the fact that the representation of some moving objects, like people, might be less suited for scaling in case the impression of distortions is to be avoided. Elements displayed on a screen could be modified or generated in any suitable manner by a user, for instance directly via the screen in the case of a touch screen or otherwise using a mouse or keypad, etc.

In an example embodiment, adjusting the motion in the at least one animated part of the cinemagraph comprises adjusting a speed of the motion. The speed of the motion could be adjusted for example by changing a rate of frames forming an animated portion of the cinemagraph. When changing a rate of frames by increasing a rate of frames, data for additional frames could be obtained by blending or interpolating existing frames. Adjusting the speed may have the effect that a cinemagraph with a change of motion resulting from a user definition may appear more realistic.

In an example embodiment, the speed of a motion is adjusted automatically based on physical assumptions that are predefined for at least one type of movement, like an oscillating movement.

In an example embodiment, the cinemagraph comprises a plurality of animated portions and adjusting the motion in the at least one animated portion of the cinemagraph comprises adjusting a speed of the motion for a subset of the plurality of animated portions of the cinemagraph only. Without limiting the scope of the claims, this may have the effect that it can be taken into account that a user may change the characteristic of a motion only for a subset of the plurality of animated portions. Alternatively or in addition, a speed of the motion could be adjusted differently for different animated portions of the cinemagraph. Without limiting the scope of the claims, this may have the effect that it can be taken into account that a user may change the characteristic of a motion differently for different animated portions of the cinemagraph.

In an example embodiment, the created cinemagraph file may be provided for a presentation of the cinemagraph on a screen and/or for storage in a memory and/or for transmission to another apparatus.

In an example embodiment, the original cinemagraph is generated by capturing a sequence of pictures; selecting at least one animated portion automatically or based on a user input; and generating the cinemagraph with the at least one selected animated portion. An automatic selection of an animated portion can comprise analyzing the pictures and detecting motion from picture to picture. It is to be understood that the original cinemagraph may be generated partially or entirely by the same apparatus or by another apparatus.

Figure 3 is a schematic block diagram of an example mobile device 300.

Mobile device 300 could be for example a camera, a mobile phone like a smartphone, some other mobile terminal or a laptop, etc.

It comprises a processor 301 that is linked to a first memory 302, to a second memory 304, to a camera 305 and to a touchscreen 306.

Processor 301 is configured to execute computer program code, including computer program code stored in memory 302, in order to cause mobile device 300 to perform desired actions. It is to be understood that processor 301 may comprise or be connected to a random access memory (not shown) as a working memory.

Memory 302 stores computer program code for creating a cinemagraph file. The computer program code may comprise for example similar program code as memory 102. In addition, memory 302 may store computer program code implemented to realize other functions, for example computer program code for creating a cinemagraph or for supporting the creation of a cinemagraph, as well as any kind of other data. Processor 301 and memory 302 may optionally belong to a chip or an integrated circuit 303, which may comprise in addition various other components, for instance a further processor or memory.

Memory 304 can equally be accessed by processor 301. It is configured to store user data. In addition, memory 304 could store other data. It could be an integrated part of mobile device 300 or a removable part of mobile device 300, like a memory card. It could also be external to mobile device 300.

Camera 305 could be a photographic camera or a video camera. Touchscreen 306 is an example of a user interface. Mobile device 300 may comprise other or additional user interface components, like a touch insensitive display, a keypad or buttons, etc. It is to be understood that functions corresponding to those of touchscreen 306 could also be provided, for example, by means of a display that is not touch sensitive and some separate input means, like keys.

Mobile device 300 could comprise various other components, like a transceiver or some other communication interface enabling a direct or indirect data exchange with other devices.

Component 303 or mobile device 300 could correspond to example embodiments of an apparatus according to the invention.

Figure 4 is a schematic block diagram of an example embodiment of a system, which comprises a first device 400 and a second device 410.

By way of example, first device 400 is assumed to be a PC, but it could also be some other device. PC 400 is or comprises an apparatus supporting the creation of cinemagraph files. PC 400 comprises at least one processor 401, a first memory 402, a second memory 404 and a touchscreen 406, which may be configured and arranged in a similar manner as corresponding components of mobile device 300. In the case of PC 400, the functions corresponding to those of touchscreen 306 of mobile device 300 could also be provided by means of a display that is not touch sensitive and some separate input means, like a mouse. In contrast to mobile device 300, PC 400 may but does not have to comprise a camera. In contrast to mobile device 300, PC 400 comprises in any case a communication interface 407 enabling a data exchange with other devices via a direct and/or indirect connection.

Second device 410 is a device comprising a camera. Thus, it could be a camera or a device comprising an integrated camera, like a smartphone. Second device 410 is configured to enable an exchange of data with PC 400 via a direct and/or indirect connection.

A direct connection could be for instance a Bluetooth connection or a cable connection, etc. An indirect connection could include for instance a connection via Internet, WLAN and/or a cellular communication network, etc. Example operations at mobile device 300 of Figure 3 will now be described with reference to Figure 5. Processor 301 and some of the program code stored in memory 302 cause mobile device 300 to perform the presented operations when the program code is retrieved from memory 302 and executed by processor 301.

Upon a user input, camera 305 captures a picture sequence, (action 51 1) The obtained picture sequence may be stored in memory 304.

The pictures are segmented, and motion is detected from picture to picture in each segment, (action 512) Next, animated portions are selected in the pictures, (action 513) The selection may be performed manually by a user- as will be discussed further below - or automatically. Both, the manual selection and the automatic selection may be performed in a conventional manner. For an automatic selection, those segments may be selected, for instance, which comprise the largest amount of motion or a motion exceeding a predetermined measure. A cinemagraph may then be created in a conventional manner using the selected animated portions.

Furthermore, the motion determined in action 512 and the selection performed in action 513 are analyzed for determining movement vectors, (action 514) Based on the analysis and possibly some user input - as will be discussed further below -, adjustment elements characterizing a motion in the selected animated portions are determined for presentation, (action 515) Such adjustment elements may be a representation of pivot points, movement vectors or movement trails. A pivot point may be selected automatically based, for instance, on an analysis of an oscillating or rotational movement. Thus, a fixed end point of a moving part may be selected as a pivot point, or alternatively a central point of a moving part - for example of a rotating disc-shaped object - may be selected as a pivot point.

The cinemagraph with the selected animated portion or portions created in action 513 and the adjustment elements determined in action 515 may be presented on touchscreen 306 as an example of a user interface (UI). (action 516) The cinemagraph may be presented in an animated manner, in a static manner or such that a user may leaf through the frames of the cinemagraph. In the latter two approaches, the selected animated portion or portions could be highlighted in some manner. The latter approach enables a user to modify each frame separately.

A user may now edit the presentation, (action 517)

On the other hand, a user editing may relate to the selection of animated portions. Any user editing in action 517 that relates to the selection of animated portions is processed in the scope of action 513, resulting in an updated presentation in action 516.

If the animated portions are selected manually in action 513, at first the entire sequence of pictures or an automatic pre-selection of portions could be presented via touchscreen 306 in action 516. A user may then define an animated portion or animated portions via the user interface, and the final animated portions are processed in action 513 based on this user input to create the cinemagraph. Later on, a user may be enabled to request a change of the selected animated portion or portions. When such a request is detected, a corresponding feedback is given and the pictures are processed in action 513 based on this user feedback to create a new cinemagraph.

On the other hand, a user editing may relate to desired changes of motion characteristics. Any user editing in action 517 that relates to desired changes of motion characteristics is processed in the scope of action 515, resulting in an updated presentation in action 516.

The user may select new adjustment elements for desired locations on touchscreen 306 and/or change adjustment elements that are already presented on touchscreen 306.

The adjustment elements of interest for changing the characteristic of a motion could comprise at least one pivot point of an oscillating or rotational movement. In case pivot points of such motions are not determined automatically, a user could define pivot points. In case pivot points of motions are determined automatically in action 515, a user could shift the representation of the pivot points on the screen. The pivot point for a single oscillating or rotational movement could also be set to different positions from frame to frame of the cinemagraph. It is to be understood that instead of or in addition to pivot points, some other fixed reference structure of a movement could be modified.

The adjustment elements of interest for changing the characteristic of a motion could furthermore comprise at least one movement vector defining the extent of movement of an object, in particular of an oscillating movement. A user could define movement vectors for certain objects in the animated portion or portions or change the length of presented movement vectors. In general, the length of an existing movement vector would be changed by extending it to create a new effect, since the effect of reducing the length of a movement vector could also be achieved with a potentially higher quality simply by using fewer pictures of the picture sequence. However, it would also be possible to reduce the length of a first movement vector, while keeping or extending the length of a second movement vector, so that the original relation of movements to each other can be changed. The adjustment elements of interest for changing the characteristic of a motion could furthermore comprise at least one movement trail for a certain object. A user could define such a movement trail on touchscreen 306 or change a movement trail that is represented on touchscreen 306. A modified or newly defined movement trail vector may be the basis not only for adjusting the extent of an existing movement, but for generating completely new movements.

There could be many other adjustment elements of interest for changing the characteristic of a motion. For example, a user could define a desired end position of an object undergoing an oscillating movement. In case this new position requires an extension of a link to a pivot point, a user may define in addition the connection of this link to the object, so that the extension may only be performed by scaling the link, not the object in order to avoid a deformed representation of the object.

Summarized, information about any user input via touchscreen 306 is received and evaluated by processor 301 to see whether it can be understood as a request to edit the presentation on touchscreen 306. The selection of animated portions is updated in action 513 and adjustment elements are created or modified in action 515 based on the information and an updated presentation is shown in both cases in action 516 via touchscreen 306. When the user has completed the editing of the presentation, a cinemagraph file is created, (action 518)

To this end, the original content of the selected animated portions - or at least the content of those portions affected by the changes defined by the user - is modeled by converting it into a vector graphic representation. The changes defined by the user are then applied to the vector graphic representation. The movement is adapted by calculating the new size, position and orientation of each vectorized element for a plurality of frames based on the real capture and the motion characteristics defined by the user via pivots, motion vectors or motion trails, etc, for the respective element. The easiest way to create a stretched object, for example, may be to stretch the associated pixels. For many content types, this approach will result in a decent quality. It is to be understood, however, that any other approach for creating texture of a desired perceptual quality of the resulting visual data could be used as well. The speed of the new movements may be adjusted automatically as well. The speed can be adjusted by adding frames. The change of speed may be limited to a specific group of animated portions only, and it may be different for each portion.

The new content in vector graphic format is converted into raster format again.

The raster format frames for each animated portion may then be combined with the original raster format static portions of the cinemagraph in a cinemagraph file.

The file may be provided for presentation on touchscreen 306 and/or for storage in memory 304 and/or for transmission to some other device.

The same or similar operations could be performed by corresponding components of PC 400 of Figure 4, except that the picture sequence in action 31 1 would be captured by an external camera 410 instead of an internal camera 305. In a variation of such a system, camera 410 could also be responsible for automatically creating the original cinemagraph. PC 400 could then be responsible for modifying existing cinemagraphs only using actions 512 and 514-518.

It is to be understood that the operation presented in Figure 5 could be modified in many ways and that the order of actions could be changed. For example, it would be possible to analyze the motion and to create motion vectors in action 514 for the pictures as a whole and thus, optionally, before animated portions are selected in action 513. Creating motion vectors only for the selected animated portions and thus after action 513 may have the effect, though, that processing power can be saved.

Furthermore, the creation and adjustment of a vector graphic representation in action 518 could be performed not only at the end of the process, but continuously. The resulting modified animated portion(s) could equally be presented on touchscreen 306 - either in vector graphic format or in converted raster format - to provide the user with an immediate feedback. The original movement and the modeled movement could be presented for instance with different colors or with different intensities on touchscreen 306 in action 516. This may be achieved, for example, by means of a simultaneous presentation of overlaying image layers for both movements. Processor 301 together with parts of the software in memory 302 that are responsible for adjusting the movement according to a user input can also be seen as a dynamic movement engine. Processor 301 together with parts of the software in memory 302 that are responsible for adjusting the speed of movements can also be seen as a physics engine. Some examples of the change of characteristics of a motion in an animated portion of a cinemagraph will now be illustrated with reference to Figures 6-10.

Camera 306 could be used for capturing a sequence of pictures of a scene with two children on a respective swing seat of a swing set, and another person standing by.

A conventional cinemagraph made of this scene could, for example, have the child on the left swing seat and the chains of this swing animated, while the other child and the people and objects around the swing set remain static. A sketch of one view of such a cinemagraph is presented in Figure 6. The animated portion may have been selected automatically or manually. For the static portion, one of the pictures of the sequence of pictures is selected, while the animated portion is provided by a repetition of a sequence of frames that have been extracted from suitable pictures of the sequence of pictures at a location corresponding to the selected animated portion. By analyzing the motion in the animated portion, motion vectors and pivot points may be determined. Figure 7 is a sketch of the same cinemagraph as the one in Figure 6, but here, some adjustment elements are presented on screen 306. More specifically, automatically detected pivot points are represented by large black dots 601 , and an automatically detected movement vector is presented by a thick lined double-headed arrow 602. The dots 601 thus represent the original pivot points of the movement of the swing, while the double-headed arrow 602 represents the original extent of the swinging motion. It is to be understood that fewer or more adjustment elements may be presented. In addition, a link between the representation of the movement vector 602 and the representation of the pivot points 601 is indicated by thick dotted lines 603.

A user may now wish to have the swing oscillating with some other movement point.

To this end, the user may shift the representation of the pivot points down the chains of the swing, as shown in Figure 8 by large black dots 61 1. A user could be enabled, for example, to touch the representation of a pivot point on touch screen 306 and to drag it to another place. Alternatively or in addition, it could be provided that when a user taps a place on touch screen 306, the representation of a pivot point "jumps" to the tapped position. For supporting a shift in the case of a representation of several pivot points as in Figure 8, it could be provided that a user first has to tap onto a representation of a particular pivot point and only then to a desired new location for this particular pivot point. It is to be understood that in an alternative embodiment, the natural pivot points are not determined and presented automatically. In this case, the user could set the representation of the pivot points right away to the desired positions on the chains, and a corresponding representation of these pivot points is then provided on touch screen 306. For enabling a user to set a representation of a new pivot point to a desired position, it could be provided that when a user taps a place on touch screen 306, a representation of a pivot point is presented at the tapped position.

Alternatively, several available types of adjustment elements - comprising a representation of a pivot point - could be presented at some location on touch screen 306. The user could then touch the representation of a pivot point and drag an automatically created copy to a desired position on touch screen 306, or tap first onto the representation of a pivot point and then to the desired position. The available adjustment elements could be visible automatically or become visible upon some user action, like a double tap onto touch screen 306 or a single tap onto some edge of touch screen 306, etc. Thus, in example embodiments, it may be very easy for the user to manipulate and/or create pivot points and other adjustment elements. It is to be understood that many other approaches may be used for supporting the shifting or creating of a representation of a pivot point or some other adjustment element, for instance by defining suitable hovering gestures, etc.

The content of the animated portion is modeled by converting it into a vector graphic representation. The vector graphic representation may be used to create a new sequence of frames for the animated portion using the original motion vector and the new pivot points defined by the user for an adjusted movement.

When the user shifts the representation of the pivot points 61 1 halfway down the chains of the swing as illustrated in Figure 8, and this is implemented in the vector graphic

representation, the upper half of the chains, represented by thick solid lines 615, remains static, while the lower half of the chains and the child, represented by thick dashed lines 613, are animated. If the animation was limited to the lower part of the chains and the child in a conventional cinemagraph instead, there would be gaps in the chains and the animation would appear very unnatural. By defining new pivot points and adjusting a vector graphic representation accordingly, such gaps can be avoided. The swing now simply swings around a new pivot point halfway down the chains.

A reason for such a modification may be to enter a surprising effect to the cinemagraph. Alternatively, the child on the swing could have been captured for example with a camera sensor having a high resolution of 3000x2000 pixels. The desired size of the cinemagraph could be only 800x600 pixels. Instead of resizing the image, the user is able to fit the motion into a smaller cropped area by changing the pivot points. As a result, the whole movement is visible in the cropped region showing just the highlights.

In a variation of this example, flexibility and realism could be added to the user-altered effects by automatically adjusting the speed of the movement in addition. When altering the pivot points of the motion of the chains that appear to suspend the swing - that is, the point where the motion starts - also the physics of the swinging motion is altered in real life. In other words, by making the moving part of the chain shorter, the speed of the motion is increased when assuming that the height the swing seat reaches remains the same. This effect may be modeled for example by varying the frame rate of the animation during the presentation. In some embodiments, techniques such as frame blending or interpolation may be exploited in altering the frame rate. The speed adjustment could be implemented for instance based on a general rule that can be applied for any oscillating movement, using the distance between pivot points and main animated part as well as the maximum height reached by the main animated part - defined for example by the length of the movement vector - as input variables.

In further variation, it is possible to alter the location of the user-defined pivot points from frame to frame of an animated portion. As a result, the pivot points would appear to slide up and down the chains of the swing set as the animation progresses. In order to support such a use case, a vector graphics representation of both parts around the pivot point could be defined; that is, on the one hand of the static part of the chain, part of which moves during the animation, and on the other hand of the moving part of the chain. When an automatic speed adjustment is used, the speed of the motion could be automatically altered according to the change in the pivot position.

A user may not only modify pivot points, but also representations of other adjustment elements, like the movement vector presented in Figure 7. A user could pull and extend the representation of the movement vector 602 to this end. In the vector graphic format, the movement of the modeled swing could then be extended accordingly. As a result, the child would appear to be swinging higher than in reality in a seamless loop. The speed can be changed dynamically to correspond to a real word scenario using the above described speed adjustment. A user could also create new movement trails. An example for such a change is illustrated by the sketch of Figure 9.

Here, the user changed the representation of a bi-directional movement vector into a representation of a unidirectional circular movement trail 622 around the original pivot points, represented again by large black dots 601 , and thus the horizontal pole of the swing set. The vector graphic representation of the swing is used to create a sequence of frames taking account of this change. In addition, the speed may be adapted based on predefined rules as mentioned before. As a result, the child appears to be swinging with such a speed that she is going around the pole of the swing set. Of course, this does not seem to be a real movement but an interesting special effect, since the child is always facing the viewer.

With a newly defined movement trail, some content may not only have to change position and orientation. It may also have to be stretched or compressed for enabling it to move beyond the area it reaches in the captures sequence of pictures.

For example, the length of the chains of the swing set of Figure 9 may in reality be 2 meters. A newly defined movement trail may provide that the chains have to get longer and reach 5 meters when the child swings high in the air. In order to present such longer chains, a part of the original image needs to be stretched. If simply the entire moving content was stretched, the shape of the child would be altered too, which will generally not be desired. On the other hand, it is less disturbing to stretch various other objects such as the chains of a swing set. This means that the chains or a part of the chains should be modeled and adjusted separately. The beginning of the stretchable part of the chains may be given by the original (or shifted) representation of the pivot points, shown in Figure 9 again as large black dots 601. A useful end of the stretchable part of the chains may be defined by the user. In the example of Figure 9, a user may define further small black dots 626 on the screen that separate the stretchable part of the chains from the child. This enables a graphics algorithm in the computer program code to separately model the part of the chain between the dots 601 and 626, represented by thick dotted lines 623, in order to be able to alter the movement with good perceptual quality. As the user selects the positions he wants the child to reach by defining the movement trail, the length of the selected part of the chains is automatically adapted to the respective position in relation to the pivot points. The child and the rest of the chains, represented by thick dashed lines 624, is moved as defined by the movement trail without stretching.

Similarly, the chains may be required to be compressed with some defined movement trails, which may be handled in a corresponding manner. Another example is schematically presented in Figure 10. In this example, an original cinemagraph presents a person, and a selected animated portion is a moving right arm of the person. By moving the pivot point of the movement along the arm, the user is able to select which part of the arm is moving, thus creating magical new joints to the person. Figure 10 comprises a respective sketch of the same person on the left hand side, in the middle and on the right hand side. In each case, the position of the pivot point - marked by a respective black dot 631 , 632 and 633 - has been set by a user to a different position on the arm. The resulting movements are indicated by dotted lines. Each of the sketches of Figure 10 shows in addition an arrow 635. The arrow may represent a movement vector. By changing the length of the movement vector, a user could adjust in addition the extent of the movement of the respective moving part of the arm.

While the examples have been presented for modifying oscillating motions, it is to be understood that other types of motion can be modified in a similar manner.

Certain embodiments may thus allow creating motion beyond captured movement in a cinemagraph in an easy and user friendly way. They may allow including larger and dynamically developing movement to a short picture sequence while keeping the file size reasonable, enabling the creation of new movement patterns that do not exist in the source content, altering the points of movement and automatically adapting the movement in the frame sequence of a cinemagraph to that change. In general, certain embodiments may offer new surprising, playful and entertaining elements to cinemagraphs. Any presented connection in the described embodiments is to be understood in a way that the involved components are operationally coupled. Thus, the connections can be direct or indirect with any number or combination of intervening elements, and there may be merely a functional relationship between the components. Further, as used in this text, the term 'circuitry' refers to any of the following:

(a) hardware-only circuit implementations (such as implementations in only analog and/or digital circuitry)

(b) combinations of circuits and software (and/or firmware), such as: (i) to a combination of processor(s) or (ii) to portions of processor(s)/ software (including digital signal

processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone, to perform various functions) and

(c) to circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even if the software or firmware is not physically present. This definition of 'circuitry' applies to all uses of this term in this text, including in any claims. As a further example, as used in this text, the term 'circuitry' also covers an implementation of merely a processor (or multiple processors) or portion of a processor and its (or their) accompanying software and/or firmware. The term 'circuitry' also covers, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone.

Any of the processors mentioned in this text could be a processor of any suitable type. Any processor may comprise but is not limited to one or more microprocessors, one or more processor(s) with accompanying digital signal processor(s), one or more processor(s) without accompanying digital signal processor(s), one or more special-purpose computer chips, one or more field-programmable gate arrays (FPGAS), one or more controllers, one or more application-specific integrated circuits (ASICS), or one or more computer(s). The relevant structure/hardware has been programmed in such a way to carry out the described function.

Any of the memories mentioned in this text could be implemented as a single memory or as a combination of a plurality of distinct memories, and may comprise for example a read-only memory (ROM), a random access memory (RAM), a flash memory or a hard disc drive memory etc.

Moreover, any of the actions described or illustrated herein may be implemented using executable instructions in a general-purpose or special-purpose processor and stored on a computer-readable storage medium (e.g., disk, memory, or the like) to be executed by such a processor. References to 'computer-readable storage medium' should be understood to encompass specialized circuits such as FPGAs, ASICs, signal processing devices, and other devices.

Example embodiments using at least one processor and at least one memory as a non- transitory data medium are shown in Figures 1 1 and 12.

Figure 1 1 is a schematic block diagram of a device 710. Device 710 includes a processor 712. Processor 712 is connected to a volatile memory 713, such as a RAM, by a bus 718. Bus 718 also connects processor 712 and RAM 713 to a non-volatile memory 714, such as a ROM. A communications interface or module 715 is coupled to bus 718, and thus also to processor 712 and memories 713, 714. Within ROM 714 is stored a software (SW) application 717. Software application 717 may be a navigation application, although it may take some other form as well. An operating system (OS) 720 also is stored in ROM 714. Figure 12 is a schematic block diagram of a device 810. Device 810 may take any suitable form. Generally speaking, device 810 may comprise processing circuitry 812, including one or more processors, and a storage device 813 comprising a single memory unit or a plurality of memory units 814. Storage device 813 may store computer program instructions that, when loaded into processing circuitry 812, control the operation of device 810. Generally speaking, also a module 81 1 of device 810 may comprise processing circuitry 812, including one or more processors, and storage device 813 comprising a single memory unit or a plurality of memory units 814. Storage device 813 may store computer program instructions that, when loaded into processing circuitry 812, control the operation of module 81 1. The software application 717 of Figure 1 1 and the computer program instructions 817 of Figure 12, respectively, may correspond e.g. to the computer program code in memory 102, memory 302 or memory 402.

In example embodiments, any non-transitory computer readable medium mentioned in this text could also be a removable/portable storage or a part of a removable/portable storage instead of an integrated storage. Example embodiments of such a removable storage are illustrated in Figure 13, which presents, from top to bottom, schematic diagrams of a magnetic disc storage 900, of an optical disc storage 901 , of a semiconductor memory circuit device storage 902 and of a Micro-SD semiconductor memory card storage 903.

The functions illustrated by processor 101 in combination with memory 102, by processor 301 in combination with memory 302, by processor 401 in combination with memory 402, by integrated circuit 303 or by integrated circuit 403 can also be viewed as means for enabling a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface; means for modeling content of the at least one animated portion of the cinemagraph in a vector graphic format and for using the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition; and means or creating a cinemagraph file for the cinemagraph with the adjusted motion. The program codes in memories 102, 302 and 402 can also be viewed as comprising such means in the form of functional modules. Figures 2 and 5 may also be understood to represent example functional blocks of computer program codes supporting a creation of cinemagraph files.

It will be understood that all presented embodiments are only examples, and that any feature presented for a particular example embodiment may be used with any aspect of the invention on its own or in combination with any feature presented for the same or another particular example embodiment and/or in combination with any other feature not mentioned. It will further be understood that any feature presented for an example embodiment in a particular category may also be used in a corresponding manner in an example embodiment of any other category.

Claims

s claimed is:

A method comprising:

enabling, by an apparatus, a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface;

modeling, by the apparatus, content of the at least one animated portion of the cinemagraph in a vector graphic format, and using the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition; and

creating, by the apparatus, a cinemagraph file for the cinemagraph with the adjusted motion.

The method according to claim 1 , wherein adjusting the motion in the at least one animated part of the cinemagraph comprises adjusting the modeled content in vector graphic format, and wherein creating the cinemagraph file comprises converting the adjusted content in vector graphic format into a raster format for combining the adjusted content with non-adjusted portions of the cinemagraph.

The method according to claim 1 or 2, further comprising automatically analyzing the motion in the at least one animated portion of a cinemagraph and automatically determining at least one adjustment element defining a characteristic of the motion, wherein enabling the user to define the desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph comprises presenting the at least one determined adjustment element via a user interface and enabling the user to modify at least one presented adjustment element.

The method according to one of claims 1 to 3, wherein enabling a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph comprises at least one of:

enabling a user to define at least one fixed reference structure for the motion and to cause a presentation of a representation of the at least one fixed reference structure at a desired location on a screen; enabling a user to define at least one pivot for the motion and to cause a presentation of a representation of the at least one pivot at a desired location on a screen;

automatically determining a fixed reference structure for the motion, presenting a representation of the fixed reference structure to the user on a screen, and enabling the user to modify the representation of the fixed reference structure;

automatically determining a pivot for the motion, presenting a representation of the pivot to the user on a screen, and enabling the user to modify the representation of the pivot;

automatically determining a movement vector for the motion in the at least one animated portion, presenting a representation of the movement vector to the user on a screen, and enabling the user to modify the representation of the movement vector; enabling a user to define a movement trail for an object on a screen;

automatically determining a movement trail of an object in the at least one animated portion, presenting a representation of the movement trail to the user on a screen, and enabling the user to change the representation of the movement trail; and enabling a user to define on a screen elements that are allowed to be subject to scaling in order to enable the desired change of at least one characteristic of a motion.

The method according to one of claims 1 to 4, wherein adjusting the motion in the at least one animated part of the cinemagraph comprises at least one of:

adjusting a speed of the motion;

adjusting a speed of the motion by changing a rate of frames forming the at least one animated portion; and

adjusting a speed of the motion by increasing a rate of frames forming the at least one animated portion, wherein data for additional frames is obtained by one of blending and interpolation of existing frames.

The method according to one of claims 1 to 5, wherein the cinemagraph comprises a plurality of animated portions and wherein adjusting the motion in the at least one animated portion of the cinemagraph comprises one of:

adjusting a speed of the motion for a subset of the plurality of animated portions of the cinemagraph only; and adjusting a speed of the motion differently for different animated portions of the cinemagraph.

The method according to one of claims 1 to 6, further comprising providing the cinemagraph file for at least one of:

a presentation of the cinemagraph on a screen;

storage in a memory; and

transmission to another apparatus.

The method according to one of claims 1 to 7, further comprising the preceding actions of

capturing a sequence of pictures;

analyzing the pictures and detecting motion from picture to picture;

selecting at least one animated portion one of automatically and based on a user input; and

generating the cinemagraph with the at least one selected animated portion.

An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause at least one apparatus at least to perform:

enable a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph via a user interface;

model content of the at least one animated portion of the cinemagraph in a vector graphic format, and use the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition; and

create a cinemagraph file for the cinemagraph with the adjusted motion.

The apparatus according to claim 9, wherein for adjusting the motion in the at least one animated part of the cinemagraph, the computer program code is configured to, with the at least one processor, cause the at least one apparatus to adjust the modeled content in vector graphic format, and wherein for creating the cinemagraph file, the computer program code is configured to, with the at least one processor, cause the at least one apparatus to convert the adjusted content in vector graphic format into a raster format for combining the adjusted content with non-adjusted portions of the cinemagraph.

The apparatus according to claim 9 or 10, wherein the computer program code is configured to, with the at least one processor, cause the at least one apparatus to automatically analyze the motion in the at least one animated portion of a cinemagraph and to automatically determine at least one adjustment element defining a characteristic of the motion, wherein enabling the user to define the desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph comprises presenting the at least one determined adjustment element via a user interface and enabling the user to modify at least one presented adjustment element.

The apparatus according to one of claims 9 to 11, wherein for enabling a user to define a desired change of at least one characteristic of a motion in at least one animated portion of a cinemagraph, the computer program code is configured to, with the at least one processor, cause the at least one apparatus to perform at least one of:

enable a user to define at least one fixed reference structure for the motion and to cause a presentation of a representation of the at least one fixed reference structure at a desired location on a screen;

enable a user to define at least one pivot for the motion and to cause a presentation of a representation of the at least one pivot at a desired location on a screen;

automatically determine a fixed reference structure for the motion, present a representation of the fixed reference structure to the user on a screen, and enable the user to modify the representation of the fixed reference structure;

automatically determine a pivot for the motion, present a representation of the pivot to the user on a screen, and enable the user to modify the representation of the pivot;

automatically determine a movement vector for the motion in the at least one animated portion, present a representation of the movement vector to the user on a screen, and enable the user to modify the representation of the movement vector; enable a user to define a movement trail for an object on a screen; automatically determine a movement trail of an object in the at least one animated portion, present a representation of the movement trail to the user on a screen, and enable the user to change the representation of the movement trail; and

enable a user to define on a screen elements that are allowed to be subject to scaling in order to enable the desired change of at least one characteristic of a motion.

The apparatus according to one of claims 9 to 12, wherein for adjusting the motion in the at least one animated part of the cinemagraph, the computer program code is configured to, with the at least one processor, cause the at least one apparatus to perform at least one of:

adjust a speed of the motion;

adjust a speed of the motion by changing a rate of frames forming the at least one animated portion; and

adjust a speed of the motion by increasing a rate of frames forming the at least one animated portion, wherein data for additional frames is obtained by one of blending and interpolation of existing frames.

14. The apparatus according to one of claims 9 to 13, wherein the cinemagraph comprises a plurality of animated portions and wherein for adjusting the motion in the at least one animated portion of the cinemagraph, the computer program code is configured to, with the at least one processor, cause the at least one apparatus to perform at least one of: adjust a speed of the motion for a subset of the plurality of animated portions of the cinemagraph only; and

adjust a speed of the motion differently for different animated portions of the cinemagraph.

The apparatus according to one of claims 9 to 14, wherein the computer program code is configured to, with the at least one processor, cause the at least one apparatus to provide the cinemagraph file for at least one of:

a presentation of the cinemagraph on a screen;

storage in a memory; and

transmission to another apparatus.

16. The apparatus according to one of claims 9 to 15, wherein the computer program code is further configured to, with the at least one processor, cause the at least one apparatus to perform the following:

capture a sequence of pictures;

analyze the pictures and detecting motion from picture to picture;

select at least one animated portion one of automatically and based on a user input; and

generate the cinemagraph with the at least one selected animated portion. 17. The apparatus according to one of claims 9 to 16, further comprising at least one of:

a camera;

a touchscreen;

a communication interface configured to enable an exchange of data with other apparatuses; and

a further memory.

18. The apparatus according to one of claims 9 to 17, wherein the apparatus is one of:

a camera;

a device comprising a camera;

a mobile terminal;

a mobile device;

a personal computer;

a stationary device;

a component configured to be used in a device; and

a chip configured to be used in a device.

19. A non-transitory computer readable storage medium in which computer program code is stored, the computer program code when executed by a processor causing at least one apparatus to perform the following:

model content of the at least one animated portion of the cinemagraph in a vector graphic format, and use the vector graphic format for adjusting the motion in the at least one animated part of the cinemagraph based on the user definition; and create a cinemagraph file for the cinemagraph with the adjusted motion.