WO2007004139A2 - Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file - Google Patents

Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file Download PDF

Info

Publication number
WO2007004139A2
WO2007004139A2 PCT/IB2006/052156 IB2006052156W WO2007004139A2 WO 2007004139 A2 WO2007004139 A2 WO 2007004139A2 IB 2006052156 W IB2006052156 W IB 2006052156W WO 2007004139 A2 WO2007004139 A2 WO 2007004139A2
Authority
WO
WIPO (PCT)
Prior art keywords
image file
electronic image
metadata
file
audio
Prior art date
Application number
PCT/IB2006/052156
Other languages
French (fr)
Other versions
WO2007004139A3 (en
Inventor
Felix H. G. Ogg
Petrus J. L. J. Van De Laar
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2007004139A2 publication Critical patent/WO2007004139A2/en
Publication of WO2007004139A3 publication Critical patent/WO2007004139A3/en

Links

Classifications

    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/034Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43Querying
    • G06F16/438Presentation of query results
    • G06F16/4387Presentation of query results by the use of playlists
    • G06F16/4393Multimedia presentations, e.g. slide shows, multimedia albums
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G11B27/32Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
    • G11B27/327Table of contents
    • G11B27/329Table of contents on a disc [VTOC]
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10Digital recording or reproducing
    • G11B20/10527Audio or video recording; Data buffering arrangements
    • G11B2020/10537Audio or video recording
    • G11B2020/10546Audio or video recording specifically adapted for audio data
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00Record carriers by type
    • G11B2220/60Solid state media
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00Record carriers by type
    • G11B2220/60Solid state media
    • G11B2220/61Solid state media wherein solid state memory is used for storing A/V content

Definitions

  • Method of associating an audio file with an electronic image file system for associating an audio file with an electronic image file, and camera for making an electronic image file
  • the invention relates to a method of associating an audio file with an electronic image file.
  • the invention also relates to a system for associating an audio file with an electronic image file and to a camera for making an electronic image file.
  • an electronic image file may be a video file, an electronic image file, a slide show, an e-mail comprising one or more images, etc.
  • an electronic image file may comprise several sub- files or it may be a collection of image files (such as e.g. a slide show, which can be considered to be a sequence of image files).
  • a person may manually select an audio file which he knows and which is to be coupled with an image file.
  • An example of a method and device for selecting an audio file to be associated with a slide show is e.g. known from United States patent application
  • the known techniques described in the preceding paragraph relate to composing suitable music related to the content of the electronic file.
  • the result is a newly composed piece of music matching the content of the file.
  • such a method first of all requires a sophisticated analysis of the content of the image, and, secondly, will often result in newly composed music.
  • the reliving experience of an image is disturbed by newly composed music, because this music was unknown before creation of the image file.
  • the emotional impact of newly composed music is unpredictable and often off the mark.
  • Providing a better choice of background sound to accompany an electronic image may augment the emotional or psychological impact of the presentation of the image file.
  • the method of associating an audio file with an electronic image file is characterized in that, for an electronic image file having metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, the information of said metadata is made an input for an automatic search procedure wherein one or more electronic data bases of existing audio files are searched, wherein the information of said metadata of the electronic image file is compared with similar parameters of existing audio files so as to find one or more matching audio files, and an association between the one or more matching audio files and the electronic image file is established.
  • the wording "having metadata' is to be understood to mean that the metadata are available for the search. They may form a part of the electronic image file itself, or, in other words, the electronic image comprises said metadata as an integral part of the electronic image file, which is the case in preferred embodiments. However, they may alternatively be stored separately, e.g. in a volatile memory such as a DRAM when the metadata are generated just before they are used in an automatic search or on a server.
  • the information for the metadata is often easily obtainable, but content analysis can also be used to generate metadata, e.g. by recognizing objects (like the Eiffel Tower in Paris) in order to determine a geographical location.
  • the metadata indicate the where (geographical location of creation) and/or when (time of creation) and/or some indication of cultural or ethnical background of the person who created the file (language of the file).
  • time includes the date as well as the time (e.g. Monday June 20, 2005 , 7.24 AM) or one of these data, preferably both these data.
  • Information on the geographical location is easily obtainable by GPS or other location-determining means, which is often available and becomes more and more standard in electronic equipment for creating electronic image files.
  • the time of creation is routinely kept as metadata.
  • the language of the file may be made metadata for electronic image files by attaching information on the language setting of the camera to the image file. Attaching such information may be done permanently, or for the purpose of the method.
  • both geographical location and time of creation are input for the automatic search procedure.
  • the data bases may be public data bases, i.e. general access data bases or private data bases, i.e. data bases particular to the creator of the image file or with limited access.
  • a two-step automatic search procedure is preferably performed, wherein, in one step, public data bases are searched and, in another step, one or more private data bases are searched and a selection of the results of both searches is associated with the electronic image file.
  • the search steps may be taken independently of each other, after which the selection is made, or one of the steps may follow the other, using the results of the first search step in the second search step.
  • suitable audio files such as popular songs, in general with the creator's own taste or cultural background, as exemplified in the creator's own data base. Making such a selection increases the possibility of finding the right audio file to be associated with the image file.
  • the method is characterized in that one or more of the following data is a further input for the automatic search procedure: d. personal data of the creator of the file, namely one or more of the creator's nationality, the creator's native language, the creator's gender, the creator's age. If no access can be made to a private data base during the automatic search, the search may nevertheless be substantially improved by having personal data of the creator as an input. Cultural setting, gender and/or age of the creator of the file as an input of the automatic search will improve the end result.
  • one or more of the following data is a further input for the automatic search procedure: e. the geographical location of a receiver the personal data of a receiver.
  • the object of sending an electronic image file to a receiver is often to convey or relay a message to the receiver.
  • the result of the automatic search procedure will generally be better suited to make the most powerful impression.
  • the invention also relates to a system for associating an audio file with an electronic image file.
  • the system is characterized in that it comprises an associator for associating an audio file with an electronic image file, said associator comprising a metadata unit for determining metadata of an electronic image file and a search unit having an input for the metadata of the image file, wherein said metadata indicate one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, wherein the search unit is arranged to search databases comprising audio files for audio files matching the information of the metadata, and the associator is arranged to establish an association between the result of the search and the image file.
  • Preferred embodiments of the system correspond to embodiments of the above system in accordance with the preferred embodiments of the associating method.
  • All of the above embodiments of the invention relate to electronic image files having metadata with which, after creation, a suitable audio file is to be associated.
  • the automatic search is instigated at the time of creation and at the time of creation of the metadata.
  • the camera device When taking a photo with a camera, which may be incorporated in a mobile phone, or PDA, the camera device immediately makes the metadata and contacts data bases outside the device, for instance, the Internet, searches a suitable piece of background music or sound or a selection of suitable pieces of music on the basis of the geographical location, and time, and/or language, and the user hears the music or may select the music or sound, and the data on the selected music is associated with the photo.
  • the automatic search can thus be done substantially immediately before or after creation of the electronic image file. This gives the creator of the file a facility which was impossible before, namely finding a suitable piece of music while creating the file and associating it with the electronic image.
  • an appropriate audio file with the electronic image file at the time of shooting, e.g. an audio file which captures the mood of the occasion. It is often difficult, if not practically impossible to find the appropriate background sound to an image or video weeks or months after shooting the picture or film. Immediate association allows a better selection.
  • the data base outside the camera may be a local data base, i.e. a data base which is particular to or situated near the locality at which the electronic image file is created.
  • the camera is characterized in that it comprises a metadata creator for creating metadata to an image file, the metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c.
  • the language of the electronic image file comprises a communicator for communicating the metadata to an outside data base of audio files so as to instigate, in said outside data base, a search for matching audio files, and in that it comprises a receiver for receiving results of the instigated search and an associator for associating the results with the image file, and/or in that the camera comprises a data base memory and a means for searching the data base memory for audio files matching the metadata.
  • Preferred embodiments of the camera of the invention correspond to various embodiments of the method and system as described above.
  • an 'associator', 'metadata creator', 'communicator' etc. are to be broadly understood and comprise e.g. any piece of hardware, any circuit or sub-circuit designed to establish an association, create metadata, communicate, etc. as described, as well as any piece of software (computer program or sub-program or set of computer programs, or program code or codes) designed or programmed to perform such a function in accordance with the invention, as well as any combination of pieces of hardware and software acting as such, alone or in combination, without being restricted to the given examples of embodiments.
  • One program may combine several functions.
  • the invention is also embodied in any computer program comprising program code means for performing a method according to the invention when said program is run on a computer as well as in any computer program product comprising program code means stored on a computer-readable medium for performing a method according to the invention when said program is run on a computer, as well as any program product comprising program code means for use in a system according to the invention for performing the action or actions specific for the invention.
  • An association between the audio file or files and the electronic image file may be established in several manners, some of which are: directly coupling the audio file found to the image file. This can be done, for instance, when the audio file itself is already in the possession of the owner of the image file; making a direct link between the audio file and the image file, which can also be done when the audio file is already in the possession of the owner of the image file; tagging the image file with information on the audio file, enabling the user to find the audio file.
  • the invention also relates to a method of associating an audio file with an electronic image file, the method comprising the steps of: translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and associating the one or more image files having the desired metadata with the audio file.
  • the invention further relates to a system for associating an audio file with an electronic image file, the system comprising: a translator for translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, a search unit for searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and an associator for associating the one or more image files having the desired metadata with the audio file.
  • a translator for translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file
  • a search unit for searching one or more electronic databases of existing image files for one or more image files having the desired metadata
  • a user can be offered the possibility to select an audio file instead of one or more image files.
  • a search for image files matching the audio file is performed instead of a search for an audio file matching the one or more image files.
  • the desired metadata describe an electronic image and not an audio file.
  • the invention also relates to a method and system for associating an audio file with an electronic image file, which is characterized in that, for an electronic audio file having metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic audio file; b. the time of creating the electronic audio file; c. the language of the electronic audio file, the information of said metadata is made an input for an automatic search procedure wherein one or more electronic data bases of existing image files are searched, wherein the information of said metadata of the electronic audio file is compared with similar parameters of existing image files so as to find one or more matching image files of said data base and an association between the one or more matching image files and the electronic audio file is established.
  • This aspect of the invention in which an audio file associated with an image file is found is the mirror image of finding an electronic image file associated with an audio file.
  • An example is a sound recording of a jazz orchestra or opera performance or a recording of an interview. Knowing the time and place, it is possible for a user to compare this with similar video recordings or photos to find a matching image file.
  • Fig. 1 schematically illustrates an example of a method and a system according to the invention.
  • Fig. 2 schematically illustrates a method and a camera according to the invention.
  • algorithm description for this case retrieve the GPS locations from each file metadata (MD) of the 150 pictures stored by the camera.
  • a locality may be selected and/or an encompassing locality (e.g. 'Paris' if "Eiffel Tower” and "Louvre” are provided) may be composed from the multiple localities. Pictures not having the selected or encompassing locality may optionally be removed from the slide show.
  • getting all performing artists of all songs in the user's collection using an Internet database to retrieve each artist's nation of birth; selecting music from an artist born in the nation, preferably near the localities at which the pictures were taken; looking up the language spoken in that nation (simple table available on the Internet); for each song in the collection, retrieving the lyrics-text; analyze the lyrics of each song to evaluate the language of the song; - selecting a song in the language spoken in the nation where the pictures were taken.
  • the dotted lines in Fig.l schematically indicate a system 4 according to the invention.
  • the collection C may be stored within the system itself, or outside the system which is schematically indicated by the dotted rectangle around collection C.
  • Example IB The dotted lines in Fig.l schematically indicate a system 4 according to the invention.
  • a user selects 150 pictures taken in the French Riviera during the summer holiday of 2004.
  • the user wants to make a slide show with audio.
  • the method and the system are adapted to select a hit song that matches the pictures.
  • the algorithm below finds locally popular music titles ("hit songs") at creation time of the image, in this case localized towards France or the French Riviera.
  • Priority can be determined by occurrence and chart position.
  • the system according to the invention would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and for instigating a search in historical hit lists on or via e.g. the Internet, retrieving the information on suitable hit lists and associating (e.g. as metadata) the information on suitable songs with the images.
  • a video is shot in Paris early in the morning.
  • Most cameras have a language setting, but each user has the tendency to use his mother tongue, or at least a language he is familiar with.
  • the language setting of the camera is French.
  • the system according to the invention would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and language and for instigating a search in song lists (which form the data bases for the search) on or via e.g. the Internet, retrieving the information on suitable hit lists and associating (e.g. as metadata) the information on suitable songs with the images.
  • metadata comprise information on location (the GPS data) and time (the date markers), and language and for instigating a search in song lists (which form the data bases for the search) on or via e.g. the Internet, retrieving the information on suitable hit lists and associating (e.g. as metadata) the information on suitable songs with the images.
  • a user selects 25 pictures of a random roll for a slide show. All of these pictures were taken in April 2002.
  • the system selects music from the user's collection, based on what was considered a hit in April 2002, or based on what he listened to then, as follows:
  • the system would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and for instigating a search in the user's song collection (forming the data base for the search), retrieving the information on suitable songs and associating (e.g. as metadata) the information on suitable songs with the images.
  • the system may be a camera if the program and information on the song collection is loaded in the camera itself.
  • a system is installed in the Louvre which comprises a data base of suitable audio tracks in various languages, and/or even directed to various ages.
  • the camera contacts said system e.g. using Bluetooth, Near Field Communication (NFC), Wireless (WIFI) or wired communication (USB), communicates the location, and possibly the language and/or the age of the owner to the system of the Louvre museum and receives from said system one or more suitable songs or titles of songs which are added as metadata to the photo.
  • the user may listen to the songs or short samples of the songs at any time and pay for and download the song he prefers. It is to be noted that, in such circumstances, the coupling of the metadata to geographical location and time of creation is implicit because the camera is there at that time.
  • the system according to the invention would be the combination of the camera and the system installed in the Louvre museum.
  • the camera is a camera according to the invention and it is distinguished from standard cameras because it has or is provided with a program which sends information to the system of the Louvre museum, communicates the location and stores the received information on suitable music in association with the image taken.
  • the program in the camera which performs these functions also embodies the invention.
  • a similar method and system may be useful in amusement parks, where the camera automatically makes contact with a system installed by the amusement park, when a picture or video is shot at a particular attraction or show, and one or more titles of a suitable song are associated with the photo or video.
  • the exact time of the photo or video may be used to advantage, i.e. the information of time and location of creating the file may be used to offer the possibility of coupling a song to the file substantially correctly with the song that was played at that moment and at that attraction or show and thus truly recaptures the spirit of the occasion.
  • Example 6 Example 6:
  • Photos are shot, e.g. with a mobile camera 5 during a summer holiday.
  • the camera automatically makes contact with the Internet site of a local radio station (with respect to the GPS location of the camera), and retrieves the hit chart (LHC) for this local radio station (LRS), the number-one local hit (LH) and/or the hit song that is played at or around the time of taking the picture (Simultaneous).
  • the camera uses the GPS data of the camera to find the location at which the photo or video was shot and uses the location to find the local radio station (e.g. within a radius of 10 to 50 kilometers within the same country or language area).
  • the local radio station sends a short, low-quality sample of the hit song and instructions on how to download and pay for the complete song.
  • the user listens to the sample and buys (or does not buy) the complete song. This is schematically indicated in Figure 2.
  • the data base is a local data base of a local radio station.
  • the song or a number of songs played (or most frequently played) at a local radio station at the time of taking a photo or video is associated with the photo or video at or around the time of creation.
  • This example solves a frequently encountered problem. Often photos are taken or videos are shot in a foreign beach resort. Months later, at home, the images are compiled to a slide show. The user would like to use the local summer hits to recapture the spirit, but often all he remembers vaguely is the tune and small fragments of lyrics in a language he does not understand. This makes it almost impossible for him to find the right song.
  • the method and camera according to the invention make it easy to find the right song and thereby recapture the holiday spirit when the slide show is presented.
  • This example of the method and camera of the invention solves this problem in a simple manner to the benefit of the user as well as the seller of the song.
  • the user makes a picture of the rock artist in concert.
  • the camera contacts a system set up by the organization of the concert.
  • This organization makes a live recording of the concert, possibly even at various points around the stage.
  • the camera communicates to this system the time and position of the camera (for instance, the performance stage at a festival or multi/stage rock concert) at the time of creating the picture.
  • the system sends back information which is tagged to the image file and enables the user to go to the Internet site of the organization after the concert, and a search in the data base of recordings will provide a matching song, namely the song performed at the time of taking the picture.
  • the user buys a live recording of the song that was sung when the image was taken and even (if various recordings are made) more or less at the position where he was standing.
  • As contact is made at the concert it is even possible for the organization to install safety measures so as to prevent those that did not attend the concert from buying the live recording or from further unauthorized copying of the live recording, while yet providing a valued service for those attending the concert.
  • the fact that it is possible to restrict access of the live recording to those attending the concert gives exclusivity to the live recording and provides added value.

Abstract

The geographical location at which one or more electronic images were taken and/or a time of creating the one or more electronic images are used to match the electronic image to an appropriate audio file. The camera comprises a communicator for communicating the metadata to an outside data base of audio files so as to instigate, in said outside data base, a search for matching audio files, and it comprises a receiver for receiving results of the instigated search and an associator for associating the results with the one or more image files. Alternatively, the camera comprises a data base memory and a means for searching the data base memory for audio files matching the metadata.

Description

Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file
FIELD OF THE INVENTION
The invention relates to a method of associating an audio file with an electronic image file.
The invention also relates to a system for associating an audio file with an electronic image file and to a camera for making an electronic image file.
In the context of the invention, an electronic image file may be a video file, an electronic image file, a slide show, an e-mail comprising one or more images, etc. Within the scope of the invention, an electronic image file may comprise several sub- files or it may be a collection of image files (such as e.g. a slide show, which can be considered to be a sequence of image files).
BACKGROUND OF THE INVENTION
A person may manually select an audio file which he knows and which is to be coupled with an image file. An example of a method and device for selecting an audio file to be associated with a slide show is e.g. known from United States patent application
2004/0001704. However, such a selection requires an active role of the user and often in- depth knowledge of suitable audio files and technical know-how on how to find suitable audio files and couple them to electronic image files, which know-how is not always available. Even so, if the technical know-how is available, the choice of suitable audio tracks may be simply too large or the effort to find and couple the audio tracks to the electronic file may be too much.
It is also known, for instance, from United States patent 6,084,169, to automatically compose background music so as to match the atmosphere and sequence of scenes within a video sequence. The article "Automatic background music generation based on actors' by Nakamura, J et al. in the Journal of Visualization and Computer Animation, 1994, vol. 5, part 4, pages 247-264, describes a method of automatically composing background music for an animated film based on the actors in the film.
The known techniques described in the preceding paragraph relate to composing suitable music related to the content of the electronic file. The result is a newly composed piece of music matching the content of the file. However, such a method first of all requires a sophisticated analysis of the content of the image, and, secondly, will often result in newly composed music. The reliving experience of an image is disturbed by newly composed music, because this music was unknown before creation of the image file. The emotional impact of newly composed music is unpredictable and often off the mark.
Providing a better choice of background sound to accompany an electronic image may augment the emotional or psychological impact of the presentation of the image file. The more recognizable, familiar, known or fitting background sounds are to the image file, the greater in general the emotional impact of the combination of the background sound and image is.
OBJECT AND SUMMARY OF THE INVENTION
It is an object of the invention to provide a method of associating audio, e.g. sound or music, with an electronic image file which is simpler and/or provides a better chance of finding the right audio. It is also an object of the invention to provide a system and a camera by which an audio file may be associated with a video file in a relatively simple manner.
To this end, the method of associating an audio file with an electronic image file is characterized in that, for an electronic image file having metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, the information of said metadata is made an input for an automatic search procedure wherein one or more electronic data bases of existing audio files are searched, wherein the information of said metadata of the electronic image file is compared with similar parameters of existing audio files so as to find one or more matching audio files, and an association between the one or more matching audio files and the electronic image file is established.
Within the scope of the invention, the wording "having metadata' is to be understood to mean that the metadata are available for the search. They may form a part of the electronic image file itself, or, in other words, the electronic image comprises said metadata as an integral part of the electronic image file, which is the case in preferred embodiments. However, they may alternatively be stored separately, e.g. in a volatile memory such as a DRAM when the metadata are generated just before they are used in an automatic search or on a server.
The information for the metadata is often easily obtainable, but content analysis can also be used to generate metadata, e.g. by recognizing objects (like the Eiffel Tower in Paris) in order to determine a geographical location. The metadata indicate the where (geographical location of creation) and/or when (time of creation) and/or some indication of cultural or ethnical background of the person who created the file (language of the file). In the context of the invention, time includes the date as well as the time (e.g. Monday June 20, 2005 , 7.24 AM) or one of these data, preferably both these data.
Information on the geographical location (the city and/or country, and/or countryside, height) is easily obtainable by GPS or other location-determining means, which is often available and becomes more and more standard in electronic equipment for creating electronic image files. The time of creation is routinely kept as metadata.
The language of the file may be made metadata for electronic image files by attaching information on the language setting of the camera to the image file. Attaching such information may be done permanently, or for the purpose of the method.
Mobile phones, for instance, which are nowadays routinely used for taking pictures, have all these metadata available and, consequently, no complex analysis of the content of the image file is required. Yet, surprisingly, in many cases, even such simple information allows an efficient automatic search procedure to be performed in data bases, rendering a suitable audio file or at least a manageable collection of suitable audio files, e.g. existing pieces of music. This is based on the insight that a large proportion of existing audio files, in particular existing pieces of music (especially, but not exclusively, popular songs) have a distinct peak of occurrence in time and/or place and cultural setting, or the subject of the audio file or song has a distinct relation to a geographical setting and/or time and/or cultural setting.
In a preferred embodiment, both geographical location and time of creation are input for the automatic search procedure.
In a most preferred embodiment, geographical location of creation, time of creation and language of the file are input for the automatic search procedure. In embodiments, the data bases may be public data bases, i.e. general access data bases or private data bases, i.e. data bases particular to the creator of the image file or with limited access.
A two-step automatic search procedure is preferably performed, wherein, in one step, public data bases are searched and, in another step, one or more private data bases are searched and a selection of the results of both searches is associated with the electronic image file. The search steps may be taken independently of each other, after which the selection is made, or one of the steps may follow the other, using the results of the first search step in the second search step. This allows a selection to be made of suitable audio files, such as popular songs, in general with the creator's own taste or cultural background, as exemplified in the creator's own data base. Making such a selection increases the possibility of finding the right audio file to be associated with the image file.
In preferred embodiments, the method is characterized in that one or more of the following data is a further input for the automatic search procedure: d. personal data of the creator of the file, namely one or more of the creator's nationality, the creator's native language, the creator's gender, the creator's age. If no access can be made to a private data base during the automatic search, the search may nevertheless be substantially improved by having personal data of the creator as an input. Cultural setting, gender and/or age of the creator of the file as an input of the automatic search will improve the end result.
In preferred embodiments, one or more of the following data is a further input for the automatic search procedure: e. the geographical location of a receiver the personal data of a receiver.
The object of sending an electronic image file to a receiver is often to convey or relay a message to the receiver. By adding data relating to the receiver as an input to the automatic search procedure, the result of the automatic search procedure will generally be better suited to make the most powerful impression.
The invention also relates to a system for associating an audio file with an electronic image file. The system is characterized in that it comprises an associator for associating an audio file with an electronic image file, said associator comprising a metadata unit for determining metadata of an electronic image file and a search unit having an input for the metadata of the image file, wherein said metadata indicate one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, wherein the search unit is arranged to search databases comprising audio files for audio files matching the information of the metadata, and the associator is arranged to establish an association between the result of the search and the image file.
Preferred embodiments of the system correspond to embodiments of the above system in accordance with the preferred embodiments of the associating method.
All of the above embodiments of the invention relate to electronic image files having metadata with which, after creation, a suitable audio file is to be associated.
In embodiments, the automatic search is instigated at the time of creation and at the time of creation of the metadata.
When taking a photo with a camera, which may be incorporated in a mobile phone, or PDA, the camera device immediately makes the metadata and contacts data bases outside the device, for instance, the Internet, searches a suitable piece of background music or sound or a selection of suitable pieces of music on the basis of the geographical location, and time, and/or language, and the user hears the music or may select the music or sound, and the data on the selected music is associated with the photo. The automatic search can thus be done substantially immediately before or after creation of the electronic image file. This gives the creator of the file a facility which was impossible before, namely finding a suitable piece of music while creating the file and associating it with the electronic image. When an image or video is shot, the person shooting the image or film would like to associate an appropriate audio file with the electronic image file at the time of shooting, e.g. an audio file which captures the mood of the occasion. It is often difficult, if not practically impossible to find the appropriate background sound to an image or video weeks or months after shooting the picture or film. Immediate association allows a better selection.
The data base outside the camera may be a local data base, i.e. a data base which is particular to or situated near the locality at which the electronic image file is created. According to the invention, the camera is characterized in that it comprises a metadata creator for creating metadata to an image file, the metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, and in that it comprises a communicator for communicating the metadata to an outside data base of audio files so as to instigate, in said outside data base, a search for matching audio files, and in that it comprises a receiver for receiving results of the instigated search and an associator for associating the results with the image file, and/or in that the camera comprises a data base memory and a means for searching the data base memory for audio files matching the metadata.
Preferred embodiments of the camera of the invention correspond to various embodiments of the method and system as described above.
Within the context of the invention, an 'associator', 'metadata creator', 'communicator' etc. are to be broadly understood and comprise e.g. any piece of hardware, any circuit or sub-circuit designed to establish an association, create metadata, communicate, etc. as described, as well as any piece of software (computer program or sub-program or set of computer programs, or program code or codes) designed or programmed to perform such a function in accordance with the invention, as well as any combination of pieces of hardware and software acting as such, alone or in combination, without being restricted to the given examples of embodiments. One program may combine several functions.
The invention is also embodied in any computer program comprising program code means for performing a method according to the invention when said program is run on a computer as well as in any computer program product comprising program code means stored on a computer-readable medium for performing a method according to the invention when said program is run on a computer, as well as any program product comprising program code means for use in a system according to the invention for performing the action or actions specific for the invention.
An association between the audio file or files and the electronic image file may be established in several manners, some of which are: directly coupling the audio file found to the image file. This can be done, for instance, when the audio file itself is already in the possession of the owner of the image file; making a direct link between the audio file and the image file, which can also be done when the audio file is already in the possession of the owner of the image file; tagging the image file with information on the audio file, enabling the user to find the audio file. When the owner is not in the possession of the audio file, but will have to buy the audio file, tagging the image file with the information enabling the user to download the audio file will be useful; coupling a short sample of the audio file to the image file, and tagging information to the image file, enabling the user to find and buy the complete audio file.
The invention also relates to a method of associating an audio file with an electronic image file, the method comprising the steps of: translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and associating the one or more image files having the desired metadata with the audio file.
The invention further relates to a system for associating an audio file with an electronic image file, the system comprising: a translator for translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, a search unit for searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and an associator for associating the one or more image files having the desired metadata with the audio file.
In this aspect of the invention, a user can be offered the possibility to select an audio file instead of one or more image files. A search for image files matching the audio file is performed instead of a search for an audio file matching the one or more image files. In this aspect of the invention, the desired metadata describe an electronic image and not an audio file.
The invention also relates to a method and system for associating an audio file with an electronic image file, which is characterized in that, for an electronic audio file having metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic audio file; b. the time of creating the electronic audio file; c. the language of the electronic audio file, the information of said metadata is made an input for an automatic search procedure wherein one or more electronic data bases of existing image files are searched, wherein the information of said metadata of the electronic audio file is compared with similar parameters of existing image files so as to find one or more matching image files of said data base and an association between the one or more matching image files and the electronic audio file is established.
This aspect of the invention, in which an audio file associated with an image file is found is the mirror image of finding an electronic image file associated with an audio file. An example is a sound recording of a jazz orchestra or opera performance or a recording of an interview. Knowing the time and place, it is possible for a user to compare this with similar video recordings or photos to find a matching image file.
BRIEF DESCRIPTION OF THE DRAWINGS
These and further aspects of the invention will be explained in greater detail by way of the following Figures and examples. In the accompanying drawings,
Fig. 1 schematically illustrates an example of a method and a system according to the invention.
Fig. 2 schematically illustrates a method and a camera according to the invention.
DESCRIPTION OF EMBODIMENTS Example IA (Fig 1). A user selects 150 pictures (1) taken in the French Riviera during the summer holiday of 2004. The user wants to make a slide show with sounds. In this example, the method and the system are adapted to select a song from the user's song collection that matches in some way. The algorithm below (with embedded variations) tries to find music related to
France or the French Riviera.
— algorithm description for this case — retrieve the GPS locations from each file metadata (MD) of the 150 pictures stored by the camera.
Reverse lookup of the nation or nations and/or localities (e.g. cities, towns) in which these coordinates are located. This will provide localities LOC. If multiple localities are provided, a locality may be selected and/or an encompassing locality (e.g. 'Paris' if "Eiffel Tower" and "Louvre" are provided) may be composed from the multiple localities. Pictures not having the selected or encompassing locality may optionally be removed from the slide show.
Analyze the song in the user's song collection (2). If any song in the user's collection has a title or a lyric with the name of this nation or any of these localities, it can be used. This will provide a suitable collection (C) . Further fine-tuning of the collection may be done by e.g. getting all performing artists of all songs in the user's collection; using an Internet database to retrieve each artist's nation of birth; selecting music from an artist born in the nation, preferably near the localities at which the pictures were taken; looking up the language spoken in that nation (simple table available on the Internet); for each song in the collection, retrieving the lyrics-text; analyze the lyrics of each song to evaluate the language of the song; - selecting a song in the language spoken in the nation where the pictures were taken.
The dotted lines in Fig.l schematically indicate a system 4 according to the invention. The collection C may be stored within the system itself, or outside the system which is schematically indicated by the dotted rectangle around collection C. Example IB.
A user (Maggie) selects 150 pictures taken in the French Riviera during the summer holiday of 2004. The user wants to make a slide show with audio. In this example, the method and the system are adapted to select a hit song that matches the pictures.
The algorithm below finds locally popular music titles ("hit songs") at creation time of the image, in this case localized towards France or the French Riviera.
— algorithm description for this case — - retrieve the GPS locations from each file meta data of the 150 pictures stored by Maggie's camera.
Reverse lookup of the nation or nations and/or localities (cities, towns) in which these coordinates are located, in this case resulting in the French Riviera area.
Retrieve the date markers of all pictures in the slide show, keep the first and last chronological date and denote them BEGIN and END, respectively.
Retrieve from historical hit lists (which are available on the Internet) hit songs in France and/or in French between BEGIN and END. Priority can be determined by occurrence and chart position.
Associate one or more of these songs with the images. In this example, the system according to the invention would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and for instigating a search in historical hit lists on or via e.g. the Internet, retrieving the information on suitable hit lists and associating (e.g. as metadata) the information on suitable songs with the images.
Example 2:
A video is shot in Paris early in the morning. Most cameras have a language setting, but each user has the tendency to use his mother tongue, or at least a language he is familiar with. In this example, the language setting of the camera is French.
— algorithm description for this case — retrieve the GPS locations from the video file meta data stored by Maggie's camera. Reverse lookup of the nation or nations and/or localities (cities, towns) in which these coordinates are located. Paris would be found.
Retrieve the time of creation (early in the morning).
Retrieve the language of the image (language setting of the camera, i.e. French).
Search a data base for a song matching these particulars, i.e. Paris, early in the morning, French. "Paris se reve" is one of the titles found.
In this example, the system according to the invention would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and language and for instigating a search in song lists (which form the data bases for the search) on or via e.g. the Internet, retrieving the information on suitable hit lists and associating (e.g. as metadata) the information on suitable songs with the images.
Example 3:
A user selects 25 pictures of a random roll for a slide show. All of these pictures were taken in April 2002.
The system selects music from the user's collection, based on what was considered a hit in April 2002, or based on what he listened to then, as follows:
— algorithm description for this example —
Retrieve the date markers of all pictures in the slide show, keep the first and last chronological date and denote them BEGIN and END, respectively.
Calculate which song or songs were most frequently played between BEGIN and END from the song playing log (like iTunes offers), and select them for the background of the slide show.
Consult the hit-charts history database (easily made available) for any hit charts dated between BEGIN and END.
If any song title or artist in the collection matches the retrieved chart, it is queued for playing. Priority can be determined by occurrence and chart position.
From the song collection, find any song bought (e.g. Apple's iTunes Online Music Store, MP3.Com) between BEGIN and END, and play it.
In this example, the system according to the invention would be e.g. a computer having programs or being provided with programs for reading the metadata of the images, which metadata comprise information on location (the GPS data) and time (the date markers), and for instigating a search in the user's song collection (forming the data base for the search), retrieving the information on suitable songs and associating (e.g. as metadata) the information on suitable songs with the images. Also in this case, the system may be a camera if the program and information on the song collection is loaded in the camera itself.
Example 4:
Walking through the Louvre museum, a user takes a picture of the Mona Lisa. A system is installed in the Louvre which comprises a data base of suitable audio tracks in various languages, and/or even directed to various ages. The camera contacts said system e.g. using Bluetooth, Near Field Communication (NFC), Wireless (WIFI) or wired communication (USB), communicates the location, and possibly the language and/or the age of the owner to the system of the Louvre museum and receives from said system one or more suitable songs or titles of songs which are added as metadata to the photo. The user may listen to the songs or short samples of the songs at any time and pay for and download the song he prefers. It is to be noted that, in such circumstances, the coupling of the metadata to geographical location and time of creation is implicit because the camera is there at that time.
In this example, the system according to the invention would be the combination of the camera and the system installed in the Louvre museum. The camera is a camera according to the invention and it is distinguished from standard cameras because it has or is provided with a program which sends information to the system of the Louvre museum, communicates the location and stores the received information on suitable music in association with the image taken. The program in the camera which performs these functions also embodies the invention.
Example 5:
A similar method and system may be useful in amusement parks, where the camera automatically makes contact with a system installed by the amusement park, when a picture or video is shot at a particular attraction or show, and one or more titles of a suitable song are associated with the photo or video. In this embodiment, even the exact time of the photo or video may be used to advantage, i.e. the information of time and location of creating the file may be used to offer the possibility of coupling a song to the file substantially correctly with the song that was played at that moment and at that attraction or show and thus truly recaptures the spirit of the occasion. Example 6:
Photos are shot, e.g. with a mobile camera 5 during a summer holiday. The camera automatically makes contact with the Internet site of a local radio station (with respect to the GPS location of the camera), and retrieves the hit chart (LHC) for this local radio station (LRS), the number-one local hit (LH) and/or the hit song that is played at or around the time of taking the picture (Simultaneous). The camera uses the GPS data of the camera to find the location at which the photo or video was shot and uses the location to find the local radio station (e.g. within a radius of 10 to 50 kilometers within the same country or language area). The local radio station sends a short, low-quality sample of the hit song and instructions on how to download and pay for the complete song. The user listens to the sample and buys (or does not buy) the complete song. This is schematically indicated in Figure 2.
In this example, the data base is a local data base of a local radio station. The song or a number of songs played (or most frequently played) at a local radio station at the time of taking a photo or video is associated with the photo or video at or around the time of creation. This example solves a frequently encountered problem. Often photos are taken or videos are shot in a foreign beach resort. Months later, at home, the images are compiled to a slide show. The user would like to use the local summer hits to recapture the spirit, but often all he remembers vaguely is the tune and small fragments of lyrics in a language he does not understand. This makes it almost impossible for him to find the right song. The method and camera according to the invention make it easy to find the right song and thereby recapture the holiday spirit when the slide show is presented. This example of the method and camera of the invention solves this problem in a simple manner to the benefit of the user as well as the seller of the song.
Example 7:
It is strictly forbidden to make audio recordings during rock concerts or jazz festivals, because the organizers fear that they lose money. Nevertheless, those attending the rock concert or festival would like to have a momentum which best captures the mood, and the organizers of the concert would like to please their audience.
This problem can be overcome as follows with an embodiment of the present invention. The user makes a picture of the rock artist in concert. The camera contacts a system set up by the organization of the concert. This organization makes a live recording of the concert, possibly even at various points around the stage. The camera communicates to this system the time and position of the camera (for instance, the performance stage at a festival or multi/stage rock concert) at the time of creating the picture. The system sends back information which is tagged to the image file and enables the user to go to the Internet site of the organization after the concert, and a search in the data base of recordings will provide a matching song, namely the song performed at the time of taking the picture. The user buys a live recording of the song that was sung when the image was taken and even (if various recordings are made) more or less at the position where he was standing. As contact is made at the concert, it is even possible for the organization to install safety measures so as to prevent those that did not attend the concert from buying the live recording or from further unauthorized copying of the live recording, while yet providing a valued service for those attending the concert. The fact that it is possible to restrict access of the live recording to those attending the concert gives exclusivity to the live recording and provides added value. The present invention has been described hereinbefore with reference to the accompanying examples. However, the invention may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; these embodiments are rather provided in order that this disclosure is thorough and complete and will fully convey the scope of the invention to those skilled in the art. Identical numerals and signs refer to the same elements throughout the description. The invention resides in each and every novel characteristic feature and each and every combination of characteristic features. Reference numerals in the claims do not limit their protective scope. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in the claims. Use of the article "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps.

Claims

CLAIMS:
1. A method of associating an audio file with an electronic image file, characterized in that, for an electronic image file having metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, the information of said metadata is made an input for an automatic search procedure wherein one or more electronic data bases of existing audio files are searched, wherein the information of said metadata of the electronic image file is compared with similar parameters of existing audio files so as to find one or more matching audio files of said data base, and an association between the one or more matching audio files and the electronic image file is established.
2. A method as claimed in claim 1 , wherein the information of said metadata on geographical location and time of creation is made an input for the automatic search procedure.
3. A method as claimed in claim 2, wherein the information of said metadata on language of the file is also made an input for the automatic search procedure.
4. A method as claimed in claim 1, wherein a two-step automatic search procedure is performed, wherein, in one step, public data bases are searched and, in another step, one or more private data bases are searched and a selection of the results of both searches is associated with the electronic image file.
5. A method as claimed in any one of the preceding claims, wherein one or more of the following data is a further input for the automatic search procedure: d. personal data of the creator of the file, namely one or more of the creator's nationality, the creator's (native) language, the creator's gender, the creator's age.
6. A method as claimed in any one of the preceding claims, wherein one or more of the following data is a further input for the automatic search procedure: e. the geographical location of a receiver the personal data of a receiver.
7. A method as claimed in any one of the preceding claims, wherein the metadata is created at creation of the electronic image file and the automatic search is done substantially immediately before or after creation of the electronic image file.
8. A method as claimed in claim 7, wherein the electronic image file is created by a device, said device making contact with a database outside the device to perform the automatic search.
9. A method as claimed in claim 8, wherein the database outside said device is a local database.
10. A method of associating an audio file with an electronic image file, the method comprising the steps of: translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, - searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and associating the one or more image files having the desired metadata with the audio file.
11. A computer program comprising program code means for performing a method according to the invention when said program is run on a computer.
12. A computer program product comprising program code means stored on a computer-readable medium for performing a method according to the invention when said program is run on a computer.
13. A system for associating an audio file with an electronic image file, the system comprising an associator for associating an audio file with an electronic image file, said associator comprising a metadata unit for determining metadata of an electronic image file and a search unit having an input for the metadata of the image file, wherein said metadata indicate one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, wherein the search unit is arranged to search databases comprising audio files for audio files matching the information of the metadata, and the associator is arranged to establish an association between the result of the search and the image file.
14. A camera for making an electronic image file, the camera comprising a metadata creator for creating metadata to an image file, the metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; c. the language of the electronic image file, and in that the camera comprises a communicator for communicating the metadata to an outside data base of audio files so as to instigate, in said outside data base, a search for matching audio files, and in that it comprises a receiver for receiving results of the instigated search and an associator for associating the results with the image file, and/or in that the camera comprises a data base memory and a means for searching the data base memory for audio files matching the metadata.
15. A system for associating an audio file with an electronic image file, the system comprising: a translator for translating metadata of an audio file into desired metadata of an electronic image file, the desired metadata indicating one or more of the following data: a. the geographical location or locations at the time of creating the electronic image file; b. the time of creating the electronic image file; and c. the language of the electronic image file, a search unit for searching one or more electronic databases of existing image files for one or more image files having the desired metadata; and an associator for associating the one or more image files having the desired metadata with the audio file.
PCT/IB2006/052156 2005-06-30 2006-06-28 Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file WO2007004139A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP05105891 2005-06-30
EP05105891.5 2005-06-30

Publications (2)

Publication Number Publication Date
WO2007004139A2 true WO2007004139A2 (en) 2007-01-11
WO2007004139A3 WO2007004139A3 (en) 2007-03-22

Family

ID=37459409

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/052156 WO2007004139A2 (en) 2005-06-30 2006-06-28 Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file

Country Status (1)

Country Link
WO (1) WO2007004139A2 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007057850A3 (en) * 2005-11-21 2007-11-01 Koninkl Philips Electronics Nv System and method for using content features and metadata of digital images to find related audio accompaniiment
WO2009001247A1 (en) * 2007-06-26 2008-12-31 Nokia Corporation Method, apparatus and computer program product for providing internationalization of content tagging
EP2051257A1 (en) * 2006-10-23 2009-04-22 Sony Corporation Reproduction device, reproduction method, and program
EP2704039A3 (en) * 2012-08-31 2014-08-27 LG Electronics, Inc. Mobile terminal
WO2015022006A1 (en) * 2013-08-12 2015-02-19 Telefonaktiebolaget Lm Ericsson (Publ) Real time combination of listened-to audio on a mobile user equipment with a simultaneous video recording
US11544314B2 (en) 2019-06-27 2023-01-03 Spotify Ab Providing media based on image analysis

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6084169A (en) * 1996-09-13 2000-07-04 Hitachi, Ltd. Automatically composing background music for an image by extracting a feature thereof
US20010037721A1 (en) * 2000-04-28 2001-11-08 Yamaha Corporation Apparatus and method for creating content comprising a combination of text data and music data
US20040001704A1 (en) * 2002-06-27 2004-01-01 Chan Ming Hong Slide show with audio
EP1422668A2 (en) * 2002-11-25 2004-05-26 Matsushita Electric Industrial Co., Ltd. Short film generation/reproduction apparatus and method thereof
EP1469456A1 (en) * 2002-01-23 2004-10-20 Konica Corporation Image delivery apparatus
EP1492342A1 (en) * 2002-04-11 2004-12-29 Konica Minolta Holdings, Inc. Information recording medium and manufacturing method thereof
EP1635326A2 (en) * 2004-09-14 2006-03-15 Sony Corporation Information processing device, method, and program
EP1655736A1 (en) * 2004-11-08 2006-05-10 Fujitsu Limited Data processing apparatus, information processing system, selection program and computer-readable recording medium recording the program

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6084169A (en) * 1996-09-13 2000-07-04 Hitachi, Ltd. Automatically composing background music for an image by extracting a feature thereof
US20010037721A1 (en) * 2000-04-28 2001-11-08 Yamaha Corporation Apparatus and method for creating content comprising a combination of text data and music data
EP1469456A1 (en) * 2002-01-23 2004-10-20 Konica Corporation Image delivery apparatus
EP1492342A1 (en) * 2002-04-11 2004-12-29 Konica Minolta Holdings, Inc. Information recording medium and manufacturing method thereof
US20040001704A1 (en) * 2002-06-27 2004-01-01 Chan Ming Hong Slide show with audio
EP1422668A2 (en) * 2002-11-25 2004-05-26 Matsushita Electric Industrial Co., Ltd. Short film generation/reproduction apparatus and method thereof
EP1635326A2 (en) * 2004-09-14 2006-03-15 Sony Corporation Information processing device, method, and program
EP1655736A1 (en) * 2004-11-08 2006-05-10 Fujitsu Limited Data processing apparatus, information processing system, selection program and computer-readable recording medium recording the program

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
NAKAMURA J-I ET AL: "AUTOMATIC BACKGROUND MUSIC GENERATION BASED ON ACTORS' MOOD AND MOTIONS" JOURNAL OF VISUALIZATION AND COMPUTER ANIMATION, XX, XX, vol. 5, 1994, pages 247-264, XP008061957 cited in the application *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007057850A3 (en) * 2005-11-21 2007-11-01 Koninkl Philips Electronics Nv System and method for using content features and metadata of digital images to find related audio accompaniiment
US8171016B2 (en) 2005-11-21 2012-05-01 Koninklijke Philips Electronics N.V. System and method for using content features and metadata of digital images to find related audio accompaniment
EP2051257A1 (en) * 2006-10-23 2009-04-22 Sony Corporation Reproduction device, reproduction method, and program
EP2051257A4 (en) * 2006-10-23 2009-10-21 Sony Corp Reproduction device, reproduction method, and program
US8049094B2 (en) 2006-10-23 2011-11-01 Sony Corporation Reproduction device, reproduction method, and program
WO2009001247A1 (en) * 2007-06-26 2008-12-31 Nokia Corporation Method, apparatus and computer program product for providing internationalization of content tagging
EP2704039A3 (en) * 2012-08-31 2014-08-27 LG Electronics, Inc. Mobile terminal
US9247144B2 (en) 2012-08-31 2016-01-26 Lg Electronics Inc. Mobile terminal generating a user diary based on extracted information
WO2015022006A1 (en) * 2013-08-12 2015-02-19 Telefonaktiebolaget Lm Ericsson (Publ) Real time combination of listened-to audio on a mobile user equipment with a simultaneous video recording
US20160182942A1 (en) * 2013-08-12 2016-06-23 Telefonaktiebolaget L M Ericsson (Publ) Real Time Combination of Listened-To Audio on a Mobile User Equipment With a Simultaneous Video Recording
US11544314B2 (en) 2019-06-27 2023-01-03 Spotify Ab Providing media based on image analysis

Also Published As

Publication number Publication date
WO2007004139A3 (en) 2007-03-22

Similar Documents

Publication Publication Date Title
JP4384671B2 (en) Lyrics providing system for digital audio files
US8868585B2 (en) Contents replay apparatus and contents replay method
US8082256B2 (en) User terminal and content searching and presentation method
US20220035858A1 (en) Generating playlists using calendar, location and event data
US10540699B1 (en) Methods and systems for scene driven content creation
US20170154109A1 (en) System and method for locating and notifying a user of the music or other audio metadata
US20080109404A1 (en) Location dependent music search
US9088662B2 (en) System and method for managing file catalogs on a wireless handheld device
WO2007004139A2 (en) Method of associating an audio file with an electronic image file, system for associating an audio file with an electronic image file, and camera for making an electronic image file
US8364396B2 (en) Organizing media data using a portable electronic device
CN108292411A (en) Video content item is generated using subject property
US20080109405A1 (en) Earmarking Media Documents
JP5779938B2 (en) Playlist creation device, playlist creation method, and playlist creation program
JP5146114B2 (en) Music player
CN111144076A (en) Social information publishing method and device
CN101925897B (en) Be proposed to be used in the method with the accompaniment melody of content data item reproduced in synchronization
JP2005327304A (en) Retrieval system, retrieving device, retrieving method and retrieving program, and communication device, communication method and communication program
JP2004152174A (en) Content reproducing device, content providing system, content retrieving method, and program
US20150032744A1 (en) Generation of personalized playlists for reproducing contents
JP2010191940A (en) Information processing apparatus, information processing method, and program
JP5573914B2 (en) Position correspondence list creation system
KR101243235B1 (en) System and method for providing sound source contents
CA2703504C (en) System and method for managing file catalogs on a wireless handheld device
KR101472034B1 (en) Radio broadcasting system, method of providing information about audio source in radio broadcasting system and method of purchasing audio source in radio broadcasting system
JP2008252921A (en) Retrieval system, communication method, retrieval apparatus, retrieval method and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase in:

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06765928

Country of ref document: EP

Kind code of ref document: A2