TW201411552A - An image enhancement apparatus - Google Patents

Abstract

A method comprising: analysing at least two images to determine at least one object mutual to the at least two images, the object having a periodicity of motion; generating an animated image based on the at least two images, wherein the at least one object is animated; determining at least one audio signal associated with the at least one object; and combining the at least one audio signal with the animated image to generate an audio enabled animated image.

Description

Image enhancement device Field of invention

The present invention is directed to providing an additional function for images. More particularly, but not exclusively, the present invention provides display devices that provide additional functionality for images displayed in the mobile device.

Background of the invention

Many portable devices, such as mobile phones, are equipped with a display window such as a glass or plastic to provide information to the user. Moreover, such display windows are now commonly used as touch inputs. In some more advanced devices, the device is equipped with a transducer suitable for producing audible feedback.

Imagery and animated images are well known to us. Animated images or moving photo images can provide the illusion that the view is watching a video. The moving photo is usually a static photo in which a subtle and repetitive action takes place. These are particularly useful because they can be transmitted or transmitted between devices using significantly smaller bandwidths than conventional video.

Summary of invention

According to a first aspect, the present invention provides a method, comprising: analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion; generating one based on the at least two images An animated image, wherein the at least one object is animated; determining at least one audio signal associated with the at least one object; and combining the at least one audio signal and the animated image to produce an audio-enabled animated image.

Determining the at least one audio signal associated with the at least one object may include: receiving the at least one audio signal, wherein at least a portion of the at least one audio signal is substantially captured at the same time as the at least two images; Filtering the at least one audio signal.

Receiving the at least one audio signal can include receiving the at least one audio signal from the at least one microphone.

Filtering the at least one audio signal may include: determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal to remove the at least one foreground sound source from the at least one audio signal to generate a ring sound audio a signal as at least one audio signal associated with the at least one object.

Filtering the at least one audio signal to select at least one portion of the at least one audio signal may include: determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal to extract the at least one audio signal At least one foreground source to generate at least one foreground source audio signal as at least one audio signal associated with the at least one object.

Determining that the at least one audio signal associated with the at least one object can include: receiving an indication that the at least one object is common to the at least two images; indicating that the at least one object is common to the at least two images; and based on the pointing The object generates the at least one audio signal.

The pointing at least one object may include: determining a location associated with the at least two images; indicating at least one object based on a location of the at least two images; selecting at least one of the at least one object.

It is pointed out that the at least one object may include pattern recognition analysis of the at least one object common to the at least two images to indicate the object.

Determining that the at least one audio signal associated with the at least one object may include at least one of: receiving the at least one audio signal from an external audio database; receiving the at least one audio signal from an internal audio database; and synthesizing The at least one audio signal.

The method may further include: generating an audio stream model to control the presentation of the audio signal; and processing the at least one audio signal using the audio stream model.

The generating the audio stream model may include: determining a tone of the audio signal; determining a volume of the audio signal; determining a playback speed of the audio signal; determining a repetition period of the audio signal; determining a start of the audio signal; and determining an end of the audio signal.

The method may further comprise time synchronizing the at least one audio signal with the animated image to generate a synchronized audio enabled animation image.

And analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, and further comprising: receiving the at least two images from an image source; and outputting the at least two images Going to a display; and receiving at least one user input to determine the at least one object common to the at least two images.

And analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, and further comprising: correlating and analyzing the at least two images to determine at least one candidate object, the at least one The candidate objects are common to the at least two images and have a periodic motion; and at least one of the at least one candidate object is selected as the at least one object.

A device can be configured to perform the methods as described herein.

A computer program product comprising program instructions can cause a device to perform a method as described herein.

One method can be as essentially described in the present invention and as illustrated in the accompanying drawings.

A device can be as essentially described in the present invention and as illustrated in the accompanying drawings.

According to a second aspect, the present invention provides an apparatus comprising at least one processor and at least one memory having one or more program instruction codes stored therein, the at least one memory and the instruction code being configured to be at least A processor together causes the apparatus to perform at least: analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion; generating an animated image based on the at least two images, wherein the at least one object is animated; determining at least one audio signal associated with the at least one object; and combining the at least one audio signal with the animation The image is used to produce an animated image that is audio enabled.

Determining that the at least one audio signal associated with the at least one object is operative to: receive the at least one audio signal, wherein at least a portion of the at least one audio signal is substantially captured at the same time as the at least two images And filtering the at least one audio signal.

Receiving the at least one audio signal may cause the apparatus to perform receiving the at least one audio signal from the at least one microphone.

Filtering the at least one audio signal may cause the apparatus to perform: determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal to remove the at least one foreground sound source from the at least one audio signal to generate a ring sound An audible audio signal as at least one audio signal associated with the at least one object.

Filtering the at least one audio signal to select at least one portion of the at least one audio signal may cause the apparatus to perform: determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal to extract from the at least one audio signal The at least one foreground source is generated to generate at least one foreground source audio signal as at least one audio signal associated with the at least one object.

Determining that the at least one audio signal associated with the at least one object is executable by the apparatus to: receive an indication that the at least one object is common to the at least two images; indicating that the at least one object is common to the at least one object Two images; and generating the at least one audio signal based on the pointed object.

Indicates that the at least one object can cause the apparatus to perform: determining a location associated with the at least two images; indicating at least one object based on a location of the at least two images; selecting at least one of the at least one object.

It is pointed out that the at least one object can cause the apparatus to perform a pattern recognition analysis on the at least one object common to the at least two images to indicate the object.

Determining at least one audio signal associated with the at least one object may cause the apparatus to perform at least one of: receiving the at least one audio signal from an external audio library; receiving the at least one audio signal from an internal audio library; And synthesizing the at least one audio signal.

The apparatus is further executable to: generate an audio stream model to control the presentation of the audio signal; and use the audio stream model to process the at least one audio signal.

Generating the audio stream model may cause the apparatus to perform: determining a tone of the audio signal; determining a volume of the audio signal; determining a playback speed of the audio signal; determining a repetition period of the audio signal; determining a start of the audio signal; and determining an end of the audio signal .

The apparatus is further operative to perform time synchronization of the at least one audio signal with the animated image to produce a synchronized audio enabled animation image.

Parsing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, which further enables the apparatus to perform: receiving the at least two images from an image source; outputting the at least two And displaying the image on a display; and receiving the at least one user input to determine the at least one object common to the at least two images.

Parsing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, which is further executable by the device: correlating the at least two images to determine at least one candidate object, The at least one candidate object is common to the at least two images and has a periodic motion; and at least one of the at least one candidate object is selected as the at least one object.

According to a third aspect, the present invention provides a device comprising: means for analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion; The image creates a component of an animated image, wherein the at least one object is animated; a member that can determine at least one audio signal associated with the at least one object; and a component that can be combined with the at least one audio signal and the animated image Produces an audio-enabled animated image.

The means for determining the at least one audio signal associated with the at least one object may comprise: means for receiving the at least one audio signal, wherein at least a portion of the at least one audio signal is substantially identical to the at least two images Time is captured; and means for filtering the at least one audio signal.

The means for receiving the at least one audio signal may include A component of the at least one audio signal can be received from at least one microphone.

The means for filtering the at least one audio signal may comprise: means for determining at least one foreground sound source of the at least one audio signal; means for filtering the at least one audio signal to remove the at least one from the at least one audio signal The foreground sound source produces a ringtone audio signal as at least one audio signal associated with the at least one object.

The means for filtering the at least one audio signal to select at least one portion of the at least one audio signal may comprise: means for determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal from the at least one The at least one foreground sound source is extracted from the audio signal to generate at least one foreground sound source audio signal as at least one audio signal associated with the at least one object.

Determining, by the member, the at least one audio signal associated with the at least one object, the means for receiving an indication, the indication indicating that the at least one object is common to the at least two images; indicating that the at least one object is common to the a member of at least two images; and a member that can generate the at least one audio signal based on the pointed object.

The member indicating the at least one object may include: a member that can determine a position associated with the at least two images; a member that indicates at least one object based on a position of the at least two images; from the at least one object Select at least one component.

The member indicating the at least one object can include a member that can perform pattern recognition analysis for the at least one object common to the at least two images to indicate the object.

The means for determining the at least one audio signal associated with the at least one object may comprise at least one of: means for receiving the at least one audio signal from an external audio library; receiving the audio data from an internal library a component of at least one audio signal; and a component that can synthesize the at least one audio signal.

The device may further include: a component that can generate an audio stream model to control the presentation of the audio signal; and a component that can use the audio stream model to process the at least one audio signal.

The means for generating the audio stream model may include: means for determining an audio signal tone; means for determining an audio signal volume; means for determining an audio signal playback speed; means for determining an audio signal repetition period; determining audio The component at which the signal begins; and the component that determines the end of the audio signal.

The device can further include means for synchronizing the at least one audio signal with the animated image to generate a synchronized audio-enabled animated image.

And analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, the component further comprising: a component capable of receiving the at least two images from an image source; And outputting the at least two images to a component on a display; and receiving, by the at least one user, a component of the at least one object that determines the commonity of the at least two images.

Analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion, the member having The method may further include: correlating the components of the at least two images to determine at least one candidate object, the at least one candidate object being common to the at least two images and having a periodic motion; and selecting the at least one candidate object At least one of the members of the at least one object.

According to a fourth aspect, the present invention provides an apparatus, comprising: an analyzer configured to analyze at least two images to determine at least one object common to the at least two images, the object having a periodic motion; The generator is configured to generate an animated image based on the at least two images, wherein the at least one object is animated; an audio clip generator configured to determine at least one audio signal associated with the at least one object; A combiner is configured to combine the at least one audio signal and the animated image to produce an audio enabled animated image.

The audio clip generator can include: an input configured to receive the at least one audio signal, wherein at least a portion of the at least one audio signal is substantially captured at the same time as the at least two images; and The filter is configured to filter the at least one audio signal.

The input can include a microphone input configured to receive the at least one audio signal from at least one microphone.

The filter can include: an essential audio signal analyzer configured to determine at least one foreground sound source of the at least one audio signal; and further configured to filter the at least one audio signal to be shifted from the at least one audio signal The at least one foreground sound source is generated to generate a ring sound audio signal as at least one audio signal associated with the at least one object.

The filter can include: an essential audio signal analyzer configured to determine at least one foreground sound source of the at least one audio signal; and further configured to filter the at least one audio signal to extract from the at least one audio signal The at least one foreground source is generated to generate at least one foreground source audio signal as at least one audio signal associated with the at least one object.

The audio clip generator can include: a composite audio generator configured to receive an indication that the at least one object is common to the at least two images; configured to indicate that the at least one object is common to the at least two images And further configured to generate the at least one audio signal based on the pointed object.

The composite audio generator configured to indicate the at least one object can include: a position estimator configured to determine a location associated with the at least two images; an object identifier configured to be based on the at least The position of the two images indicates at least one item; and an item selector is configured to select at least one of the at least one item.

The composite audio generator configured to indicate the at least one object can include a pattern recognition analyzer configured to perform pattern recognition for the at least one object common to the at least two images to indicate the object .

The composite audio generator can include at least one of: an input configured to receive the at least one audio signal from an external audio library; an input configured to receive the at least one from an internal audio library Audio signal; an input is configured to be available from a note The memory body receives the at least one audio signal; and a synthesizer is configured to synthesize the at least one audio signal.

The apparatus can further include: an audio clip embedder configured to generate an audio stream model to control the presentation of the audio signal; and configured to process the at least one audio signal using the audio stream model.

The audio clip embedder configured to generate the audio stream model can be configured to: determine a tone of the audio signal; determine a volume of the audio signal; determine a playback speed of the audio signal; determine a repetition period of the audio signal; determine an audio signal Start; and determine the end of the audio signal.

The apparatus can further include a synchronizer configured to time synchronize the at least one audio signal with the animated image to produce a synchronized audio enabled animated image.

The analyzer can include: an input configured to receive the at least two images from an image source; an input configured to output the at least two images onto a display; and a user interface input configured to be Receiving at least one user input to determine the at least one object common to the at least two images.

The analyzer can include: a correlator configured to correlate the at least two images to determine at least one candidate object, the at least one candidate object being common to the at least two images and having a periodic motion; The selector is configured to select at least one of the at least one candidate object as the at least one item.

A computer program product stored in a medium can make A device can perform the method as described herein.

An electronic device can include a device as described herein.

A wafer set can include a device as described herein.

10‧‧‧Electronic devices

11‧‧‧Touch input module

12‧‧‧ display

13‧‧‧Transceiver (TX/RX)

15‧‧‧ processor

16‧‧‧ memory

17‧‧‧Code section

18‧‧‧Inner data section

101‧‧‧ camera

103‧‧‧Video/Image Analyzer

105‧‧‧Activity Photo Generator

107‧‧‧Audio Clip Generator

109‧‧‧Mixer and synchronizer

171‧‧‧User Interface Input Controller

173‧‧‧ Area of Interest Selector

181~187‧‧‧

201‧‧‧Synthetic audio generator

203‧‧‧Intrinsic Audio Signal Analyzer

205‧‧‧Audio Clip Embedder

151‧‧‧Audio Clip Database

171‧‧‧Microphone/Microphone Array

301‧‧‧Feature/Sub-Region Correlator

303‧‧‧Clip Requester

305‧‧‧Clip Returner/Clip Selector

401‧‧‧ ring sound detector

501‧‧‧ audio stream model generator

503‧‧‧Optical Streaming Processor

601~609‧‧‧

701~709‧‧‧

801~805‧‧‧

901~907‧‧‧

1001~1007‧‧‧

1301‧‧‧1 video image

1303‧‧‧Region 1

1305‧‧‧Region 2

1313‧‧‧Selected area of interest R ₁

1315‧‧‧Selected area of interest R ₂

1321‧‧‧Selected audio area

1323‧‧‧Selected video sub-area R ₁

1325‧‧‧Selected video sub-area R ₂

1401‧‧‧Animated Video Sub-area

1403‧‧‧Video sequence

1405‧‧‧ ringtone sequence

For a better understanding of the present invention, reference will now be made to the accompanying drawings, by way of example, in which: FIG. 1 schematically shows a device that is suitable to use some embodiments; FIG. 2 schematically shows an audio enhancement session An example of a moving photo generator; Figure 3 schematically illustrates a video/image analyzer as shown in Figure 2, in accordance with some embodiments; Figure 4 schematically illustrates an image as shown in Figure 2, in accordance with some embodiments. A flowchart of the operation of the video/image analyzer shown in FIG. 3; FIG. 5 schematically illustrates an audio clip generator as shown in FIG. 2, in accordance with some embodiments; FIG. 6 illustrates A synthetic audio generator as shown in FIG. 5 is shown; FIG. 7 schematically illustrates an intrinsic audio signal analyzer as shown in FIG. 5, in accordance with some embodiments; Figure 8 schematically illustrates an audio clip embedder as shown in Figure 5; Figure 9 schematically illustrates an audio enhanced action photo generator as shown in Figure 2, in accordance with some embodiments. one of Operational flow chart; Figure 10 schematically illustrates an operational flow diagram of a composite audio generator as shown in Figure 6 in accordance with some embodiments; Figure 11 schematically illustrates one as in Figure 7 in accordance with some embodiments. An operational flow diagram of the illustrated intrinsic audio signal analyzer; FIG. 12 schematically illustrates an operational flow diagram of an audio clip embedder as shown in FIG. 8, in accordance with some embodiments; 13 schematically illustrates an operational flow diagram of a mixer and synchronizer as shown in FIG. 2; according to some embodiments, FIG. 14 illustrates an example of sub-region selection in a region of interest of an image; In some embodiments, Figure 15 illustrates an example of a time plan for video and audio formation of an audio enhanced moving picture.

Detailed description of the preferred embodiment

The concept of an embodiment of the present invention is to combine an audio signal into a moving photo (animated image) during the process of generating a moving photo or an animated image. This can be implemented in the illustrated example of the present invention by generating and embedding metadata using at least one of an intrinsic audio signal and a synthesized audio signal that includes an audio effect signal or can be coupled to the audio. The chain of effect signals, and in a manner such that the resulting moving photo can be enhanced by the audio effect.

High-quality photos and video are a great way to get a review of the experience. Moving photos or animated images are seen as an extension of the photo and are produced using post-production techniques. The photo of the meeting A member is provided to move objects that are common between images, or to move objects in an area of a still or static image. For example, the design or aesthetic element allows for elements with subtle movements while the rest of the image is still. This action or animation feature is repeated in some moving photos. In the following description and patent application, such terms as objects, common objects, or objects may be considered to refer to any object that is shared (or common) between images and used to create a moving photo or animated object. Element, object or component. For example, the images used as an input may be a video, the content of which is a moving toy train that is essentially a stationary background. In such an example, the object, the subject, the common item, the area, or the element may be the toy train that provides the dynamic or subtle motion element in the animated image while the remaining images are stationary. It is to be understood that the commonality of the object or subject does not necessarily mean that the object, subject, or element is substantially identical in a single frame. However, when the object moves or appears to be moving, there is usually a large degree of correlation between subsequent image objects. For example, by making the train appear larger/smaller, the object or object of the toy train can appear to be driving toward or away from the viewer in a one-frame picture, or It is the change in the appearance of the toy train that makes the toy train appear to be driving away or heading towards the viewer.

In other words, the size, shape, and position of the image area indicated as the object, target, or element can be changed between images, but within the image is a selected entity One grid There is a degree of correlation between the pictures (compared to the composition of the still image, which has a perfectly perfect correlation between the pictures in one frame).

A moving photo can be seen in some ways as a potential natural progression of image viewing: from grayscale (black and white photography) to color, from color to high resolution color images, from completely static to localized in photos. exercise. However, revisiting the edge of an experience without audio will still be considered incomplete, and the moving photo is still unable to present audio.

Therefore, the question will be how to generate a moving photo or an animated image so that the audio can be added to it.

Usually a moving photo (or a photo or animated image that will be an action) is constructed from a sequence of video, where the audio is likely to be existing or accompanying the video. However, when attempting to link the audio to the moving photo, the recorded audio in the scene cannot be completely attached to the motion image, and the additional audio should be selected and selectively processed. For example, we can't repeat an audio clip (similar to adding an action to a sub-region of an image) because it can become unpleasant.

Another problem associated with adding audio is that when two people are talking in a conversation, repeating a particular sequence of audio produces an unnatural example. For example, repeating audio clips over and over again ("Hey, Ravi, are you?") would be unnatural. Therefore, it is to be understood that one or a portion of the audio clips or audio signals can be played repeatedly or in a loop; and one or a selected type of audio should not be played repeatedly, even when the image is moved. It can only be played once during loop playback.

Another problem with using looped audio to link to video images in moving photos or animated images is to accurately loop audio signals or audio clips. For example, a looping or repeating audio signal may bounce or pause when it is played repeatedly, which can lead to an unnatural or offensive experience.

It is to be understood that a moving photo is generally understood to have a repeatable, subtle moving element (or target or object), however in some cases the audio may be attached to an animated image or An object or moving element that is not repeatable in the photo. For example, add a lightning/thundering sound to a moving photo. As such, in some embodiments, the audio clip or signal can be a single instance playback element within an animated scene of a visual motion component. In the examples below, motion photography or moving photos can be viewed as covering all combinations or single or repeatable image motions and audio clip playback.

1 is a schematic block diagram of an example electronic device 10 upon which an embodiment of the present invention can be implemented. This embodiment of the device 10 is configured to provide an improved imaging experience.

In some embodiments, the device 10 is a mobile terminal, mobile phone or user device for operation in a wireless communication system. In other embodiments, the device is any suitable electronic device configured to process video and audio material. In some embodiments, the apparatus is configured to provide an image display such as, for example, a digital camera, a portable audio player (MP3 player), and a portable video player (MP4 player). . In other embodiments, the device may be any suitable touch The interface's electronic devices (which may or may not display information), such as a touch screen or trackpad, are configured to provide feedback when the touch screen or trackpad is touched. For example, in some embodiments, the touchpad can be a touch sensitive keypad that may not be labeled thereon in some embodiments; while in other embodiments there may be physical indicia or logos on its front window. In such an embodiment, the user can be informed by a physical indicator of the location to be touched, such as a raised profile, or a printed layer that can be illuminated by a light guide.

The device 10 includes a touch input module or user interface 11 that is linked to a processor 15. The processor 15 is further linked to a display 12. The processor 15 is further linked to a transceiver (TX/RX) 13 and a memory 16.

In some embodiments, the touch input module 11 and/or the display 12 are separate or separable from the electronic device, and the processor receives signals from the touch input module 11 and/or transmits and receives the signals. The device 13 or other suitable interface sends a signal to the display 12. Moreover, in some embodiments, the touch input module 11 and display 12 are components of the same component. In such an embodiment, the touch input module 11 and the display 12 may be referred to as the display component or the touch display component.

In some embodiments, the processor 15 can be configured to execute a variety of code. In some embodiments, the implemented code may include programs, such as audio signal parsing and decoding of image data; touch processing; input simulation; or haptic effect analog code, wherein the touch input module input is Detection and processing; effect feedback signal generation, where electrical A signal may be generated that, when delivered to a transducer, may produce tactile or tactile feedback to the user of the device; or an actuator process configured to generate an actuator signal for driving the actuator . For example, in some embodiments, the implemented code may be stored in the memory 16, more specifically in a code segment 17 of the memory 16, whenever needed The processor 15 is available for retrieval. In some embodiments, the memory 15 can further provide a section 18 for storing data, for example, storing data that has been processed in accordance with the application, such as, for example, virtual audio signal data.

In some embodiments, the touch input module 11 can implement any suitable touch screen interface technology. For example, in some embodiments the touch screen interface can include a capacitive sensor configured to be sensitive to a finger appearing above or above the touch screen interface. The capacitive sensor can comprise an insulator (such as glass or plastic) coated with a transparent conductor (eg, indium tin oxide-ITO). Since the human body is also a conductor, touching the surface of the screen causes a distortion of the local electrostatic field, which can be measured by a change in capacitance. Any suitable technique can also be used to determine the location of the touch. The location can be passed to the processor, which can calculate how the user's touch is related to the device. The insulator protects the conductive layer from dirt, dust or residue from the fingers.

In some other embodiments, the touch input module can be a resistive sensor that includes several layers of thin metal conductive layers separated by a narrow gap. When an object, such as a finger, When a point on the outer surface of the panel is pressed, the two metal layers become conductive at that point: the panel then behaves like a pair of voltage dividers with connected outputs. This physical change therefore results in a change in the current that is registered as a touch event and sent to the processor for processing.

In some other embodiments, the touch input module can further utilize techniques to determine a touch event, such as visual inspection, projection capacitance detection, infrared detection, surface acoustic wave detection, dispersion signal technology, and sound pulse recognition. For example, visual inspection as described above refers to a camera that can be positioned below or above the surface to detect the location of the finger or touch object. In some embodiments, it is to be understood that "touch" can be defined as both a physical contact and a "hovering touch", while the latter is not actually in contact with the sensor, but when the object is very close to the sensing When it is used, it has an effect on the sensor.

The touch input module described herein is an example of a user interface input. It is to be understood that in some other embodiments, any other suitable user interface input can be used to provide input using a user interface, for example, selecting an item, object, or region from a display screen. In some embodiments, the user interface input can thus be a keyboard, mouse, keypad, joystick, or any suitable indicator device.

In some embodiments, the apparatus 10 can implement the processing technique at least in part by hardware, in other words, the processing performed by the processor 15 can be implemented at least partially by hardware without the need for Software or firmware to operate the hardware.

In some embodiments, the transceiver 13 is capable of communicating with other electronic devices, for example, in some embodiments, through a wireless communication network.

The display 12 can include any suitable display technology. For example, the display component can be located below the touch input module and project an image through the touch input module to be viewed by the user. The display 12 can employ any suitable display technology such as a liquid crystal display (LCD), a light emitting diode (LED), an organic light emitting diode (OLED), a plasma display unit, a field emission display (FED), surface conduction electrons. A light-emitting display (SED), and an electrophoretic display (also known as an electronic paper, e-paper, or electronic ink display). In some embodiments, the display 12 employs one of display techniques that utilize a light guide to project to a display window.

Figure 2 shows an example of an audio enhanced action photo generator. Further, with respect to Fig. 9, the operation of an audio enhanced moving photo generator as shown in Fig. 2 will be further described.

In some embodiments, the audio enhanced motion photo generator includes a camera 101. The camera 101 can be any suitable video or image capture device. The camera 101 can be configured to capture images and pass the images or video data to a video/image analyzer 103.

The operation of capturing video images is shown in step 601 of FIG.

In some embodiments, the example audio enhanced active photo generator includes a video/image analyzer 103. The video/image analyzer 103 can be configured to receive video or video data from the camera 101 and analyze video images of the candidate object or sub-region for use in making motions/motions Mixing of pictures and sounds.

The output of the video/image analyzer 103 in the form of video or video material selected for the candidate video action and the object or sub-area can be transmitted to the moving photo generator 105. The video/image analyzer 103 can be further configured to output data indicative of candidate objects or sub-regions suitable for audio insertion to the audio clip generator 107.

In some embodiments, the example audio enhanced motion photo generator includes a moving photo generator 105. The moving photo generator 105 is configured to receive the video and any video action selection data and generate a suitable moving photo material. In the example below, the moving photo generator is configured to generate animated image material, however, as described herein, in some embodiments, the animation may be subtle or unremovable from the image (in other words) In other words, the image is basically a static image). The moving photo generator 105 can be any suitable moving photo or animated image generating component configured to generate material in a suitable format such that the moving photo viewer can produce an image with any action element. In some embodiments, the moving photo generator 105 can be configured to output the moving photo material to a mixer and synchronizer 109.

The operation of generating the moving photo material is shown in step 605 of FIG.

Moreover, in some embodiments, the example audio enhanced active photo generator includes an audio clip generator 107. The audio clip generator 107 can be configured to generate an audio clip or audio signal component that can be inserted into or incorporated into the moving photo video or image material. In some In an embodiment, the audio clip generator 107 can be configured to receive information from the video/image analyzer 103 such that the audio clip generator 107 is configured to select based on the information transmitted from the video/image analyzer 103. An audio clip audio signal component.

The operation of generating the audio clip is shown in step 607 of FIG.

In some embodiments, the apparatus includes a mixer and synchronizer 109 configured to receive the video image from the mobile photo generator 105 and receive the audio signal from the audio clip generator 107, and is configured The two signals can be mixed and synchronized in a suitable manner.

In some embodiments, the mixer and synchronizer 109 can be configured to receive the video material from the mobile photo generator 105 and receive the audio data or audio signal from the audio clip generator 107, mixing the audio sum. The video material, and the audio and video data are synchronized in a manner such that the mixer and synchronizer 109 outputs an enhanced moving picture. In some embodiments, the enhanced moving photo is a moving photo with meta-data containing audio data (or a link to an audio file) to be output with the video output. .

The operation of mixing and synchronizing the audio clips in the animated video material is shown in step 609 of FIG.

An example of a video/image analyzer 103 is shown in more detail in FIG. Further, with respect to Figure 4, the operation of the video/image analyzer 103 as shown in Figure 3 will be described in more detail.

The video/image analyzer 103 described in the present invention is equipped It is arranged to receive the video or video material from the camera and to determine an object, region or sub-region selected as an animation candidate, in other words, an object or sub-region having an associated periodicity. In some embodiments, the video/image analyzer 103 can be configured to indicate an image or video object or sub-region candidate to be audio inserted. For example, FIG. 14 shows an example video image 1301 having various indicated candidate sub-regions.

The top portion of the figure shows that the video image 1301 contains a first object or region having an associated periodicity, a region 1 having a center at a position X ₁ , Y ₁ 1303, and a second having a correlated periodicity. Object or area, area 2, centered at position X ₂ , Y ₂ 1305.

In some embodiments, the video/image analyzer 103 can be configured to select one of the objects or regions as a candidate region (or a "region of interest"). Thus, as shown in the middle portion of the figure, the video/image analyzer can select Region 1 and Region 2, shown as R ₁ , 1313, and R ₂ , 1315, respectively.

In some embodiments, the video/image analyzer 103 can be configured to select all of the image as a region of interest inserted relative to the audio clip. Thus, as shown in the bottom portion of FIG. 14, the video/image analyzer can be configured to select an entire image 1301 containing the selected video object or sub-region, shown as R ₁ , 1323, R ₂ , 1325. And the audio selection area 1321.

In some embodiments, these objects or regions may be automatically selected using some sorting method based on sub-region selection rules, semi-automatically selected using some user input, or manually selected (in other words, entirely by the Use the user interface to choose).

In some embodiments, the video/image analyzer 103 includes a user interface input controller 171. The user interface input controller 171 can be configured to receive a user interface input, such as a touch screen interface or any suitable user interface input, such as a keyboard, keypad, mouse, or any suitable indicator. In some embodiments, the user interface input controller can be configured to determine a selection input from the user interface input and communicate the selection input to the region of interest selector 173 along with the location of the selection input. In some embodiments, the user input controller is configured to operate as the same user interface output, for example, configured to output the video or image data to a suitable touch screen display and from those Make this choice.

The operation of receiving the camera data/displaying the camera data is shown in step 181 of FIG.

Additionally, the operation of receiving the user interface input is shown in step 183 of FIG.

In some embodiments, the video/image analyzer 103 includes a region of interest selector 173. The region of interest selector 173 can be configured to receive the camera material and the user interface input control data and input selections from the user interface to determine an object, sub-area or "region of interest".

In some embodiments, the region of interest selector 173 operates in an automated or semi-automatic manner to select any suitable object or sub-region with associated periodicity among the video or image material. In other words, the The video or video data is monitored for a number of pictures, and the associated periodicity within an object or region of the video image is indicated and selected as a region of interest. These objects or "regions of interest" can be used to generate an action portion of the moving photo or, in some embodiments, select an area that provides an audio clip to be blended with the image material.

The region of interest selector 173 can be configured to output the selected object or "region of interest" material. Thus, in some embodiments, the region of interest selector 173 can be configured to output the selected object, sub-region or "region of interest" value to the audio clip generator 107 and to the camera video or video material. It is transmitted to the moving photo generator 105 together.

The operation of outputting the region of interest data is shown in step 187 of FIG.

FIG. 5 shows an example of an audio clip generator 107.

In some embodiments, the audio clip generator 107 includes a composite audio generator 201. In some embodiments, the composite audio generator 201 can be configured to receive selected objects, sub-areas, or "regions of interest" information from the video/image analyzer 103. Moreover, in some embodiments, the composite audio generator 201 can be configured to be coupled to an audio clip library 151. The audio clip database 151 can be any suitable library or linked audio signal library. For example, in some embodiments, the audio clip database 151 can be a library of audio clips stored on the Internet or in the "cloud." Moreover, in some embodiments, the audio clip database 151 can be an audio signal or a link. A collection or collection of audio signals that are stored in the memory of the device.

An example of the composite audio generator 201 is shown in more detail in FIG. Moreover, FIG. 10 illustrates a flow chart of the operation of the composite audio generator 201, in accordance with some embodiments.

In some embodiments, the composite audio generator 201 includes a feature/sub-region correlator 301. The feature/sub-region correlator 301 is configured to receive an input from the video/image analyzer 103.

The operation of receiving the feature/sub-region selection information is displayed in step 701 of FIG.

In some embodiments, the feature/sub-region correlator 301 can be configured to associate a feature to the received object, sub-region (or "region of interest") selected by the video/image analyzer. . A feature can be any tag or theme on which an associated audio clip can be found (and then used to produce a composite audio effect).

For example, the user of the device may use the camera to capture a lush green forest scene and an elephant's video, most of which are captured by green plants, and the elephant's nose is the only scene in the scene. The moving ingredients. In this example, based on the periodicity of the image, the video/image analyzer 103 can be configured to select the elephant nose as a suitable image or video object or sub-region due to the periodicity of the sub-region.

The feature/sub-region correlator 301 can then be configured in such an example to associate or indicate the elephant nose with an "elephant" feature The selection of objects or sub-areas.

In some embodiments, the feature/sub-region correlator 301 can be configured to automatically indicate the object or determine an association based on the shape and/or action. For example, a "point-and-click" system can be used in which the image object, sub-area or image is transmitted to a server that can identify the image object or sub-area. In other words, the feature/sub-region correlator is not implemented in the device.

In some embodiments, the feature/sub-region correlator 301 can perform the sub-region association or identification based on location or other sensor input assistance. For example, the image is taken at a zoo or in an elephant park, the location of the zoo area/elephant park can be identified, the associated feature is determined, and associated with the object or sub-area .

In some embodiments, the feature/sub-region correlator 301 can additionally receive a user input to enable a manual or semi-automatic feature association. For example, with the previous example, when the user is prompted to indicate the object or sub-area, by the user inputting "elephant", the object or sub-area indicated by the elephant nose is associated. Moreover, in some embodiments, the feature/sub-region correlator can provide a plurality of suggested features based on the image, and the user can select one of the suggested feature lists (semi-automatic mode). For example, using the previous image example, the feature/sub-region correlator can return a list of possible features (such as the "point-and-see" method from the inside or outside of the device), such as "large "Image", "Grey Snake", "Giant Mouse", etc., and the user can select one of the possible features.

The associated feature recognition can then be sent to a clip please Seeker 303.

The operation of associating the feature to the object or sub-area is shown in step 703 of FIG.

In some embodiments, the composite audio generator 201 includes a clip requester 303. In some embodiments, the clip requestor 303 can be configured to receive the associated feature from the feature/sub-region correlator 301. The clip requestor 303 can then be configured to output an audio signal, audio clip or audio sample that most closely resembles or contains the audio presentation of the feature to the audio library 151. Thus, for example, for an object or feature identified as an elephant, the clip requester 303 can request an elephant audio clip or audio signal/sample from the audio library 151.

The operation of requesting a clip associated with the feature or identifying the object from the audio library is shown in step 705 of FIG.

In some embodiments, the composite audio generator 201 includes a clip passer/clip selector 305. The clip passer/clip selector 305 is configured to receive from the audio library 151 an appropriate audio library audio clip or audio signal/sample associated with the feature or identified object. For example, a series of audio clips or audio signals/samples may be returned from the audio library 151 for an elephant. In some embodiments, the clip passer/clip selector 305 can be configured to present these example audio clips or audio signal/sample candidates and allow the user to select one of the example candidates, in other words, The operation of selecting this audio clip is semi-automatic. In some embodiments, the clip passer/clip selector 305 can be configured to automatically select the clip, for example, selecting the maximum correlation The audio clip or audio sample.

The operation of returning an associated clip from the audio library is shown in step 707 of FIG.

In some embodiments, the clip passer/clip selector 305 can output the selected audio clip to the audio clip embedder 205.

The operation of outputting an audio clip to the audio clip embedder 205 is shown in step 709 of FIG.

For example, in an embodiment where the environment is noisy, such a camera is used to capture a photo or video image, and the audio (eg, the audio clip is an elephant sound in the example discussed) is A post-processing step. In addition, for example, in some embodiments, it is possible to capture a photo or a video image without any sound. Therefore, the method for making a moving photo to have audio is post-processing, by adding The synthesized audio is combined to synthesize the audio. In the embodiment described herein, there should be no need to open any separate applications or programs at the user, or spend hours generating the audio. For example, in some embodiments, the "point-to-live" match (in other words, the association and audio clip request and return operations) will be effective immediately. As described herein, the storage of the audio clip will occur after that, and the user can undo the audio effect at a later point in time.

In some embodiments, the audio clip generator 107 includes an intrinsic audio signal analyzer 203. The intrinsic audio signal analyzer 203 can be configured to receive an audio signal input from a microphone or microphone array 171.

FIG. 7 illustrates an example of an intrinsic audio signal analyzer 203, in accordance with some embodiments. In addition, FIG. 11 depicts the operation of the example essence audio signal analyzer 203 as shown in FIG.

In some embodiments, the microphone/microphone array 171 can be configured to be integrated with the device or, in some embodiments, separate from the device. In embodiments where the microphone/microphone array 171 is physically separate from the device, the microphone/microphone array 171 can be configured to transmit audio signals from the microphone or microphone array to the device, particularly the audio clip generator 107. .

The operation of receiving the audio signal from the microphone/microphone array is shown in step 801 of FIG.

In some embodiments, the intrinsic audio signal analyzer 203 includes a ring sound detector 401.

The ring sound detector 401 can be configured to receive the audio signal from the microphone/microphone array 171 and determine an environmental or surround sound component of the audio signal, which is representative of the video captured at the same time. Surround audio signal. For example, in some embodiments, the microphone array includes two microphone configurations to capture a stereo audio signal. However, it is to be understood that in some embodiments, the audio can be a multi-channel or mono audio signal. In some embodiments, the ring sound detector 401 is configured to extract a primary or directional component from the audio signal captured by the entire audio scene to return the ring field audio component.

It is to be understood that in some embodiments, the ring sound detector 401 can be configured to select a component of a particular direction instead of the ring. sound. For example, in some embodiments, the region of interest is shown as a particular region component, and the ringtone detector 401 can be configured to receive an input from the video/image analyzer 103 and select an Represents the audio signal that identifies the direction of the object or sub-area.

In some embodiments, the ring sound detector 401 can detect, for example, an audio signal from the surrounding environment of the device, such as a wind blow, a raining sound, a sound of a wave, or generally natural. sound. In some embodiments, the ring sound detector 401 is configured to apply a non-negative matrix factorization on the input audio channel audio material.

For example, the non-negative matrix factorization can be summarized as follows: Let Xr and Xl be the left and right channel audio data, respectively. In some embodiments, the first step of non-negative matrix factorization is to compute a common variance matrix C, given by

Where: σ _L ² : energy in the left channel

σ _R ² : energy in the right channel

ρ: the number of correlations between the two channels.

In some embodiments, when the value of σ is less than 0, non-negative matrix factorization is not performed because it means that the left and right channels exhibit a negative correlation, which can be considered a ring signal.

In some embodiments, when the value of ρ is greater than zero, the dominant component of the signal can be estimated using a non-negative matrix factorization of rank 1 of matrix C. This produces an estimate of the standard deviations σ _L and σ _R .

Moreover, in some embodiments, based on the standard deviation of the left and right channels, the weights are calculated as follows

In some embodiments, the obtained weights can be used to generate weighted sum components and weighted difference components to obtain the primary component and the ringtone component, respectively.

The operation of determining the ring sound from the audio signal by non-negative matrix factorization is shown in step 803 of FIG.

In some embodiments, the ring sound detector 401 is configured to output the filtered ring sound audio signal to the audio clip embedder 205.

Outputting the ring sound signal to the embedder operation is shown in step 805 of FIG.

In some embodiments, the audio clip generator 107 includes an audio clip embedder 205. FIG. 8 shows an example embedder 205. Further, the operation of the audio clip embedder 205 shown in FIG. 8 is described in FIG.

The audio clip embedder 205 is configured to receive at least one of the synthesized audio generator 201 and the intrinsic audio signal analyzer 203 The resulting audio clips the audio signal and configures the audio signal in a manner such that the audio signal has a format and form suitable for mixing and synchronizing.

In some embodiments, the audio clip embedder 205 includes an audio stream model generator 501. The audio stream model generator 501 is configured to generate an audio stream model suitable for applying to the audio signal. The audio stream creation model can be configured to configure a particular audio effect. For example, the model can be configured to configure a simple switch or mix operation (switching of the audio clip on and off) to a more complex operation, such as a change in pitch or beat in the audio signal. .

In some embodiments, the audio stream model generator 501 can be configured to determine whether the audio clip should be output for a specified length of time, or for a particular start time and end time, for a defined period of time. For example, as shown in FIG. 15, for a ring sound from the entire video sequence, the model is such that the audio stream model is defined to output the audio signal during the entire animation. In other words, the audio stream model outputs the ring-tone audio clip of the T-second time from time 0 seconds to time T seconds, and the audio clip is played continuously. However, in some embodiments, the generated model can be defined to output an audio signal only in a portion of the video sequence. In some embodiments, the audio stream model generator can be configured to further define an appropriate tone or volume process for the beginning and end of the audio signal clip such that when the end of a loop of the audio signal is connected to There is no obvious discontinuity at the beginning of a loop.

In other words, in some embodiments, the model can be configured to define a loop and a deformation of the audio signal such that the experienced audio signal is continuous and not intermittent (when the audio signal is looped, Has a pause, or a sudden volume or pitch change).

In some embodiments, the audio stream model generator can generate an audio stream model for each of the audio effects to be output. For example, a ringtone audio clip can be played over the entire length of the video/video and has a composite audio clip that is played only on one portion of the entire length.

The generation of the audio stream model is shown in step 901 of FIG.

The audio stream model generator 501 can be configured to output the audio stream model to an audio stream processor 503.

In some embodiments, the audio clip embedder 205 can include an audio stream processor 503. The audio stream processor 503 can be configured to receive audio signals to be streamed from the synthesized audio generator 201 or the intrinsic audio signal analyzer 203 (or both). In other words, the model generated by the audio stream model generator 501 is used to generate an output audio signal based on the model. In other words, the audio signal can be selected, filtered, switched, mixed in any suitable manner such that the model is followed.

Receiving the audio signal to be embedded is shown in step 903 of FIG.

The operation of applying the streaming model to the audio signal is shown In step 905 of FIG.

In some embodiments, the audio stream processor 503 can be configured to output the audio signal to the mixer and synchronizer 109.

The operation of outputting the streamed audio signal to the mixer and synchronizer is shown in step 907 of FIG.

The operation of an example mixer and synchronizer is shown in FIG.

In some embodiments, the mixer and synchronizer 109 can be configured to receive the video material.

The operation of receiving the video material is shown in step 1001 of FIG.

Additionally, the mixer and synchronizer 109 can be configured to receive the audio material from the audio clip generator 107.

The operation of receiving the audio material is shown in step 1003 of FIG.

In some embodiments, the mixer and synchronizer 109 can synchronize or combine the audio material with the video material. For example, in some embodiments, the video as shown in FIG. 15 has two regions of interest, wherein the first meaningful periodicity has a center position at the pixel X ₁ , Y ₁ and a length of t _{1 .} Two seconds of the center, which produces an animated video sub-region 1401; and a second meaningful periodicity whose center position is at pixel X ₂ , Y ₂ , and the length is two seconds centered on t ₂ , is displayed in video sequence 1403 Medium; the captured video length is T, and there is a ring sound from the entire video sequence that will be used for loop playback. The ring sound has been embedded (processed) according to the audio stream model to produce a ring sound sequence 1405. The mixer and synchronizer can be configured to synchronize the audio signal to the image and the image animation. For example, as shown in FIG. 15, the audio signal or clip can be synchronized at the beginning of the video or sound cycle 1405.

In some embodiments, the mixer and synchronizer 109 can then be configured to mix or multiplex the data to form a moving photo or animated video metadata file containing image or video material and audio signal data. In some embodiments, such mixing or multiplexing of data may result in a file containing at least some of the following: video material, audio material, sub-region identification data, and time synchronization data, in any suitable format. In some embodiments, the mixer and synchronizer 109 can output metadata or archive output data.

It should be appreciated that the term user device is intended to cover any suitable type of wireless user device, such as a mobile phone, a portable data processing device, or a portable web browser. Furthermore, it will be understood that the term acoustic channel is intended to encompass sound outlets, channels and resonant cavities, and such a sound channel may be integrally formed with the transducer or as a mechanical integration of the transducer with the device. a part of.

In general, the design of various embodiments of the present invention can be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in a firmware or software executable by a controller, microprocessor or other computing device, although the invention is not Limited to this. Although the various aspects of the invention have been illustrated and described in terms of a block diagram, a flow diagram, or some other graphical representation, it can be fully understood. Yes, the block diagrams, devices, systems, techniques, or methods described herein may be implemented, but are not limited to, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers. Or other computing devices, or some combination thereof.

The design of an embodiment of the present invention can be implemented in a computer software that can be executed by a data processor of the mobile device, located in the processor entity; or in hardware; or in software and hard a combination of bodies. In addition, it should be noted in this regard that any of the logic flow blocks in the figures may represent: steps of the program; or interconnected logic circuits, modules and functions; or program steps and logic circuits, A combination of modules and functions. The software can be stored in: physical media such as a memory chip or a memory block implemented inside the processor; magnetic media such as a hard disk or a floppy disk; and optical media such as a DVD and its data variant, CD.

The memory used in the design of embodiments of the present invention may be of any type suitable for the local technical environment, and any suitable data storage technique may be used, such as semiconductor based memory devices, magnetic memory devices, and System, optical memory devices and systems, fixed memory and removable memory. The data processor can be any type suitable for the local technical environment, and can include one or more general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits ( ASIC), gate level circuits and processors based on multi-core processor architecture, but are not limited to the examples described.

Embodiments of the invention may be implemented by various components, such as integrated bodies The circuit module is designed to be.

As used in this application, the term "circuitry" refers to all of the following: (a) implemented only in hardware (such as implementations using analog and/or digital circuits only) and (b) A combination of circuitry and software (and/or firmware), such as: (i) a combination of processors, or (ii) a portion of a processor/software (including a digital signal processor), a software, and a memory device To form a device, such as a mobile phone or server, to perform various functions, and (c) a circuit, such as a microprocessor or a portion of a microprocessor that requires software or firmware to operate even if the software or The firmware does not actually exist.

This definition of "circuitry" applies to all uses of this term in the application of the present invention, including all patent application scopes. As a further example, as used in the application of the present application, the term "circuitry" also encompasses an implementation of only one processor (or multiple processors) or a software associated with it (or their). And/or the implementation of a part of the firmware. The term "circuitry" also encompasses, for example, and if applicable to the elements of the particular patent application, a baseband integrated circuit or application processor integrated circuit of a mobile telephone, or a server, A similar integrated circuit in a cellular network device, or other network device.

This previous description has provided a complete and well-documented description of example embodiments of the invention, by way of exemplary and non-limiting example. However, for those skilled in the relevant art, Numerous modifications and adaptations will become apparent in the light of the appended claims. However, all such and similar modifications of the teachings of the present invention are still within the scope of the invention as defined in the appended claims.

10‧‧‧Electronic devices

11‧‧‧Touch input module

12‧‧‧ display

13‧‧‧Transceiver (TX/RX)

15‧‧‧ processor

16‧‧‧ memory

17‧‧‧Code section

18‧‧‧Inner data section

Claims (15)

An apparatus comprising at least one processor and at least one memory having stored therein one or more program instruction codes, the at least one memory and the instruction code being configured to cause the apparatus to be at least with the at least one processor Performing: analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion; generating an animated image based on the at least two images, wherein the at least one object is made into an animation; Determining at least one audio signal associated with the at least one object; and combining the at least one audio signal with the animated image to generate an audio enabled animated image. The apparatus of claim 1, wherein the determining at least one audio signal associated with the at least one object causes the apparatus to: receive the at least one audio signal; and filter the at least one audio signal. The apparatus of claim 2, wherein the receiving at least one audio signal causes the apparatus: receiving at least one portion of the at least one audio signal from the at least one microphone is substantially the same as the at least two images captured At the time. A device according to any one of claims 2 or 3, wherein at least one tone is filtered The frequency signal causes the apparatus to perform at least one of: determining at least one foreground sound source of the at least one audio signal; filtering the at least one audio signal to remove the at least one foreground sound source from the at least one audio signal to generate a ring sound And an audible audio signal as at least one audio signal associated with the at least one object; and filtering the at least one audio signal to extract the at least one foreground sound source from the at least one audio signal to generate a primary sound component as the at least one audio Signal. The apparatus of any one of the preceding claims, wherein the determining at least one audio signal associated with the at least one object causes the apparatus to: receive an indication that the at least one object is common to the at least two images; indicating that the at least one object is common to The at least two images; and generating the at least one audio signal based on the pointed object. The apparatus of claim 5, wherein the at least one object is indicative of the apparatus: determining a location associated with the at least two images; indicating at least one object based on the location of the at least two images; and selecting the at least one object. The apparatus of any one of claims 5 or 6, wherein the at least one object causes the apparatus to: perform execution of the at least one object common to the at least two images Type identification analysis. The apparatus of any one of the preceding claims, wherein the determining at least one audio signal associated with the at least one object causes the apparatus to perform at least one of: receiving the at least one audio signal from an external audio library; from an internal The audio database receives the at least one audio signal; receives the at least one audio signal from a memory; and synthesizes the at least one audio signal. The apparatus of any one of claims 2 to 8, wherein receiving the at least one audio signal further causes the apparatus to: generate an audio stream model to control a presentation of the at least one audio signal; and process the audio stream model using the audio stream model The at least one audio signal. The device of claim 9, wherein the audio stream model is generated such that the device performs at least one of: determining a program of an audio signal; determining a volume of the audio signal; determining a playback speed of the audio signal; determining a repetition period of the audio signal ; determine the start time of the audio signal; and determine the end time of the audio signal. A device as in any one of the preceding claims, wherein the device is activated To: time synchronize the at least one audio signal with the animated image to generate a synchronized audio enabled animation image. The apparatus of any one of the preceding claims, wherein analyzing the at least two images to determine the at least one object common to the at least two images further enables the device to: receive the at least two images from an image source; output the at least two And displaying the image on a display; and receiving the at least one user input to determine the at least one object common to the at least two images. The apparatus of any one of the preceding claims, wherein analyzing the at least two images to determine the at least one object common to the at least two images further enables the apparatus to: correlate the at least two images to determine at least one candidate object, At least one candidate object is common to the at least two images and has a periodic motion; and the at least one candidate object is selected as the at least one object. An electronic device comprising the devices of claims 1 to 13. A method comprising: analyzing at least two images to determine at least one object common to the at least two images, the object having a periodic motion; generating an animated image based on the at least two images, wherein the at least one object Being animated; determining at least one audio message associated with the at least one object And combining the at least one audio signal and the animated image to generate an audio-enabled animated image.