WO2006025797A1 - A search system - Google Patents

A search system Download PDF

Info

Publication number
WO2006025797A1
WO2006025797A1 PCT/SG2005/000289 SG2005000289W WO2006025797A1 WO 2006025797 A1 WO2006025797 A1 WO 2006025797A1 SG 2005000289 W SG2005000289 W SG 2005000289W WO 2006025797 A1 WO2006025797 A1 WO 2006025797A1
Authority
WO
WIPO (PCT)
Prior art keywords
search
digital media
audio
portable device
database
Prior art date
Application number
PCT/SG2005/000289
Other languages
French (fr)
Inventor
Siang Thia Goh
Yuen Khim Liow
Original Assignee
Creative Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US60624704P priority Critical
Priority to US60/606,247 priority
Application filed by Creative Technology Ltd filed Critical Creative Technology Ltd
Publication of WO2006025797A1 publication Critical patent/WO2006025797A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A research system for researching digital media information, the system comprising: a conversion module to convert the digital media information into textual data; an extraction module to extract search terms from the textual data to search for a record in at least one database; and a presentation module to present at least one matching record resulting from the search to the user via the portable device.

Description

Title
A Search System
Technical Field
The invention concerns a search system for searching digital media information recorded by a portable device.
Background of the Invention
Consumer products that enable users to capture digital information are increasingly popular. These products include camera-enabled mobile phones, camera enabled portable computers, digital camera and digital video recorders.
These consumer products are highly effective at capturing digital information. However, user friendly applications to exploit and process this information are not commonly available.
Summary of the Invention
In a first preferred aspect, there is provided a search system for searching digital media information. The system comprises a conversion module to convert the recorded digital media information into textual data; an extraction module to extract search terms from the textual data to search for a record in at least one database; and a presentation module to present at least one matching record resulting from the search to the user via the portable device. Each record in the database may be mapped to at least one search term or a collection of search terms. A user interface may be provided to enable the user to browse through the matching records, and navigate via links to other related records in the database. A contribution module may be provided to enable a user to contribute additional recorded digital media information to the database.
Digital media information may include photos, video clips, or audio clips and may be recorded by a portable device. The portable device may be a mobile phone or a mobile computing device. The mobile phone or a mobile computing device may have an integrated digital camera to capture images or video clips. The mobile phone or a mobile computing device may have an integrated microphone to record audio clips.
An optical character recognition engine may be used to convert text-based information captured in graphical form in the photos or video clips into textual data.
An image recognition engine may be used to convert image information captured in photos or video clips into textual data. Image information may include people's faces, animals, famous landmarks, vehicles or other objects. Image information may also include sign language used by deaf people.
A voice recognition engine may be used to convert spoken words in an audio clip into textual data.
The database may be locally stored on the portable device. For example, if a tourist is travelling to Europe, a database storing European tourist information may be downloaded on to the portable device.
Alternatively, the database may be remotely stored on a server. The server may be accessed via the Internet through wireless communication. The portable device may comprise a communications module to communicate via the Internet.
The results may be presented to the user via a display of the portable device. The results may be presented to the user according to a user-defined format and presentation style. The results may be presented to the user as an audio delivery. The audio delivery may be a computer generated voice or a pre-recorded audio clip associated with the matching record.
More than one item of recorded digital media information may be used together in order to increase the accuracy of the search. For example, a photo of a bird and an audio recording of the bird's call are used together to identify the species of the bird. Consequently, a record of the bird is presented to the user via their portable device that contains the biological data of the bird such as migratory patterns, life span and habitat.
In a second aspect, there is provided a method for researching digital media information. The method comprises converting the recorded digital media information into textual data; extracting search terms from the textual data to search for a record in at least one database; and presenting at least one matching record resulting from the search to the user via the portable device.
Each record in the database may be mapped to at least one search term or a collection of search terms.
The method may further comprise an initial step of recording digital media information.
The method may further comprise the step of translating the textual data into another language.
If the digital media information is a photo or a video clip, the method may further comprise the step of focusing on a specific area of the photo or frame of the video clip to limit the scope of the search.
If the digital media information is an audio clip, the method may further comprise the step of focusing on a specific portion of the audio clip to limit the scope of the search.
The concentrating on a specific area may be by at least one of: zooming, framing, and select drag and drop. Motion vectors are used for a video clip.
The method may further comprise concentrating on a specific portion of the digital media and converting the specific portion into textual data. When the digital media is an audio track and a fingerprint of the audio is generated, searching may be by use of the fingerprint. Alternatively, the concentration on a specific portion may be by selecting a start and end of a portion of an audio track to form an audio segment, and searching by use of the audio segment.
Brief Description of the Drawings
An example of the invention will now be described with reference to the accompanying drawings, in which:
Figure 1 is a block diagram of the system;
Figure 2 is a perspective view of an embodiment of a device for use in the system; Figure 3 is a block diagram of part of the device of Figures 1 and 2; and Figure 4 is a process flow diagram of researching using the system.
Detailed Description of the Drawings
Figure 1 and the following discussion are intended to provide a brief, general description of a suitable computing environment in which the present invention may be implemented. Although not required, the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by a personal computer. Generally, program modules include routines, programs, characters, components, data structures, ,that perform particular tasks or implement particular abstract data types. As those skilled in the art will appreciate, the invention may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Referring to Figure 1 , there is a provided a research system 10 for researching digital media information. The digital information may be on a personal computer, laptop computer, notebook computer or portable device. It may have been captured or recorded by a device 20 such as, for example, a computer, portable computer, or portable device.. Portable devices 20 include mobile phones, PDAs, tablet computers, notebook computers, digital camera or video recorders with a computer processor. Preferably, the portable device 20 has multimedia capabilities such as a high-resolution display and audio functionality. Digital media information includes photos, video clips, or audio clips. Input to a device such as computer 20 may be by scanner, camera, web cam, touch screen, and so forth.
As shown in Figures 1 to 3, the device 20 may have a microphone 60 for capturing audio. An analog/digital converter 61 converts the captured audio from analog to digital. A microprocessor and/or digital signal processor passes the digital audio to non-volatile memory 69. Memory 69 may be a hard disk, removable disk, or flash memory. The device 20 may also have a lens 63 for capturing images and/or video by use of an imaging system 64 or may have a separate image-capturing device such as, for example, a web cam. The captured images and/or video are also processed by microprocessor 62 and stored in memory 69. The lens may be fixed or may have a motor driver for optical zooming.
A keypad/keyboard 65 and/or a joystick 71 may be used for user input. A display
66 displays results or likely results, and a loudspeaker 68 may output audio results. An amplifier 67 is used to amplify the output audio after conversion in convertor 61.
Microprocessor 62 may be used to control other functions of device 20 (not shown), and is also used to control and operate various modules 21 and engines 50.
The modules 21 of system 10 comprise a conversion module 22, an extraction module 23 and a presentation module 24. These modules 22, 23, 24 are stored on the portable device 20 as software. Alternatively, the modules are hard-wired as a dedicated chip. Preferably, the modules are written in Java to facilitate portability and implementation onto other Java-enabled devices 20. The conversion module 22 converts the recorded digital media information into textual data. Depending on the recorded digital information, the conversion module 22 operates with an engine 50 that may include an optical character recognition engine 25, image recognition engine 26, voice recognition engine 27, face recognition engine 28, and a music engine 29.
The optical character recognition engine 25 converts text-based information captured in graphical form in photos or video clips into textual data. The voice recognition engine 27 converts spoken words in an audio clip into textual data. The image recognition engine 26 converts image information captured in photos or video clips into textual data. Image information includes animals, famous landmarks, vehicles or other objects. Image information also includes sign language used by deaf people. Image recognition is performed by known techniques. Face recognition engine 28 is for recognizing face using facial recognition software such as Facelt supplied by Visionics Corporation. The music engine 29 converts the captured music into a MIDI file and may be categorized by, for example, song title, performer and performance. An application for determining a melody from a melody sung, hummed or the like may be used. Furthermore, other audio fingerprinting techniques could be used to generate data that is representative of the recorded audio. For example, US 6,453,252, the disclosure of which is incorporated herein by reference, discloses a technique in which a fingerprint of an audio signal is generated based on the energy content in frequency sub-banks. The resulting fingerprints can then be used to help identify the recorded audio. Alternatively, an "A/B" button may be pressed at the start and finish of a desired audio segment and the search conducted on the basis of that audio segment.
When recognizing animal or insect sounds, these may be processed using one or more of: the actual sound, phonetics or mnemonics.
Next, the textual data is passed to the extraction module 23 for extracting search terms used to search for a record in a database 30. The database 30 is a centralised network database accessible via the Internet and mobile phone network. Preferably, the portable device 20 comprises a communications module (70) to communicate with the database 30 via the Internet. Alternatively, a light- version of the system 10 has a local database 30 stored on the portable device 20 which eliminates the need for a communications module. The database 30 in such case may be in memory 69. After at least one matching record is found, the presentation module 24 presents the matching record to the user via a user interface (not shown). The user interface enables the user to browse through the matching records, and navigate via links to other related records in the database 30. Each record in the database 30 is mapped to at least one search term or a collection of search terms. Records in the database 30 which are related to each other or in similar categories are linked with one another.
In one embodiment, the system provides a portable language translator. For example, in a foreign country, a road sign is encountered. Using the system, the foreign road sign is translated into a language understood by the user. A navigation application is intuitively provided to the user such as an interactive map.
In another embodiment, the system 10 is a portable tour guide or object recognizer. For example, at a museum, a camera-enabled portable device is captures an image of an object. The object is searched against the database 30. If a match is found, a translation is obtained in a language understood by the user and more detailed information about the object is retrieved. Another example is to identify an object or animal such as a bird. The system 10 is able to identify the species by its physical characteristics and play a sample of the call of the bird and/or name the bird.
In a further embodiment, the system 10 is a portable multimedia 'Internet Browser'. The system is able to recognise a face and retrieve the associated information regarding this face. For example, if the picture is of Bill Gates, the system 10 retrieves the personal biographical data of Bill Gates and lists his personal achievements, hobbies, favourite movies, and so forth, and displays them on display 66 and/or by audio output using loudspeaker 68.
In yet another embodiment, the system 10 is a portable video-based sign language translator. The system 10 is able to translate sign language captured in video format into words or voice and vice versa. The capturing device may be a video camera, still camera or mobile-phone camera. Alternatively, a wearable visor with a miniature camera can be used to explore of objects in a scene and their associated database 30.
Referring to Figure 4, in a typical scenario, the system 10 operates by firstly capturing 40 an image, audio or other sensory input and displaying any image or images forming at least a part of the input. A specific section or segment of the captured digital information is targeted 41. The targeting focuses on a particular object in the entire image or a fragment of the audio recording. This may be by zooming and/or framing and/or select, drag and drop using, for example, motion vectors for moving video. This may be in accordance with MPEG4. The object is then detected and recognised. This may be by use of MPEG7. After object recognition, the digital information is converted 42 into textual information. For music files, it is digitized to enable searching. The conversion 42 is performed by the conversion module 21 together with any of the engines in engine 40. For example, if the object is a photo of the Sydney Harbour Bridge, image recognition engine 26 converts the image into the phrase "Sydney Harbour Bridge". This phrase is extracted as a keyword phrase or as search terms for searching 43 in database 30. Since keywords are mapped in the database 30 to data records related to the objects, searching 43 is able to yield useful data. After the search is complete, the results of the search are presented 44 to the user.
The user is able to use a drop down menu box and/or a keypad to enter information to exclude certain classifications of groups if they are not relevant. For example, the user selects a group named "bridges" so that roads or other structures are filtered out of the search. This isolates and refines the search by user interaction.
In one embodiment, the user makes the final determination as to what the object is, based on recursive logical elimination of irrelevant results on a group by group basis. To assist in restricting the context of the search, when the digital information is transmitted to the database 30 for searching, the time of the recording and also the location of the recording are also transmitted. The time of the recording is a time stamp on the media file stamped by the portable device 20. The location of the recording assumes that transmission of the digital information is almost instantaneous from the time the user captures the photo, video or audio recording. The location is identifiable by the cell location of the mobile phone 20 or GPS co- ordinates of the portable device 20.
Using the previous example, if the user is in the physical vicinity of the "Sydney Harbour Bridge", the Golden Gate Bridge or the Brooklyn Bridge are not presented as likely matches. To increase customization capability, the user is able to define their user interface and how the results are to be presented to them.
To enhance the accuracy of searching, multiple input including an auxiliary input, may be used. That is, photos and audio information are used together for searching. For example, a photo of a bird and an audio recording of the bird's call may be used to identify the species of the bird. Auxiliary input may include, for example, location, temperature, humidity, light level, and so forth. These may be obtained automatically from appropriate functionality within level 20.
User input may be by use of a contribution module such as, for example, a keypad/keyboard 66, or use of voice recognition technology. If there are many objects in an image, zooming and/or framing and/or drag and drop may be used to identify the object being searched. This may be by use of known zooming and framing technologies, a joystick, and touch screen technologies. In sound, the audio to be searched can be extracted from the recorded audio by extracting the required audio and/or suppression of background or surrounding signals.
If the database cannot locate a correct match, the received data may be stored for the creation of a new entry. They may be stored in a separate database until sufficient data is received to provide conclusive information, whereupon it can be moved to the database 30. An editor may make decisions in this regard.
Furthermore, distributed corresponding may be used for communication from several devices 20 to database 30. The data may be first sent to a distribution server for controlling the distribution of data, and the search functionality. Ultra wide band may be used for connectivity.
The data may be sent from device 20 to data 30 by SMS or MMS.
It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the invention as shown in the specific embodiments without departing from the scope or spirit of the invention as broadly described. The present embodiments are, therefore, to be considered in all respects illustrative and not restrictive.

Claims

WE CLAIM:
1. A research system for researching digital media information, the system comprising: a conversion module to convert the digital media information into textual data; an extraction module to extract search terms from the textual data to search for a record in at least one database; and a presentation module to present at least one matching record resulting from the search to the user via the portable device.
2. The system according to claim 1 , wherein each record in the database is mapped to at least one search term or a collection of search terms.
3. The system according to claim 1, further comprising a user interface to enable the user to browse through the matching records, and navigate via links to other related records in the database, and to input additional information.
4. The system according to claim 1 , further comprising a contribution module to enable a user to contribute additional recorded digital media information to the database.
5. The system according to claim 1 , wherein digital media information includes photos, video clips, or audio clips.
6. The system according to claim 5, wherein the digital media information is recorded by a portable device, the portable device being at least one of a mobile telephone or a mobile computing device.
7. The system according to claim 6, wherein the portable device has at least one selected from the group consisting of: an integrated digital camera to capture images, an integrated digital camera to capture video clips, and an integrated microphone to record audio clips.
8. The system according to claim 7, further comprising at least one engine selected from the group consisting of: an optical character recognition engine to convert text-based information captured in graphical form in the photos or video clips into textual data, an image recognition engine to convert image information into textual data, a voice recognition engine to convert spoken words in an audio clip into textual data, a face recognition engine, and a music engine.
9. The system according to claim 8, wherein image information includes animals, famous landmarks, vehicles and sign language used by deaf people.
10. The system of claim 8, wherein an auxiliary input is provided to reduce search time, the auxiliary input including an input selected from the group consisting of: location, temperature, humidity, light level, times and an environmental analysis.
11. The system of claim 10, wherein location is determined by one of cell location, and GPS co-ordinates.
12. The system as claimed in claim 8, wherein the music engine is for converting music into a MIDI file or a fingerprint of an audio signal.
13. The system according to claim 6, wherein the database is locally stored on the portable device.
14. The system according to claim 1 , wherein the database is remotely stored on a server.
15. The system according to claim 6, further comprising a communications module to enable communication between the portable device and the database.
16. The system according to claim 6, wherein the results are presented to the user via a display of the portable device.
17. The system according to claim 1 , wherein the results are presented to the user as an audio delivery.
18. The system according to claim 1 , wherein more than one item of recorded digital media information is used to increase the accuracy of the search.
19. A method for researching digital media information, the method comprising: converting the digital media information into textual data; extracting search terms from the textual data to search for a record in at least one database; and presenting at least one matching record resulting from the search to the user via the portable device.
20. The method as claimed in claim 19, wherein each record in the database is mapped to at least one search term or a collection of search terms.
21. The method according to claim 19, further comprising an initial step of recording digital media information.
22. The method according to claim 19, further comprising the step of translating the textual data into another language.
23. The method according to claim 19, wherein when the digital media information is a photograph or a video clip, the method further comprises concentrating on a specific area of the photograph or a frame of the video clip to limit the scope of the search, concentration being by at least one of: framing, and zooming.
24. The method according to claim 19, wherein if the digital media information is an audio clip, the method further comprising the step of concentrating on a specific portion of the audio clip to limit the scope of the search.
25. The method according to claim 19, wherein if the digital media information is a music file, the music file is converted to at least one of: a MIDI file or a fingerprint of an audio signal.
26. The method according to claim 19, wherein searching is restricted based data from an auxiliary input including at least one of: the location of the portable device, and/or the time the digital information was recorded, temperature, humidity, light level, an environmental analysis.
27. The method according to claim 19, wherein the digital media information is recorded by a portable device, the user interacting with the portable device to eliminate irrelevant records resulting from the search.
28. The method according to claim 27, wherein the user interaction is selection or de-selection of items of a dynamic list to indicate whether the digital information is related to those items.
29. The method according to claim 26, wherein location is determined by one of: cell location and GPS.
30. The method of according to claim 24, wherein concentrating on the specific portion is by one or more of: extracting the specific portion, and suppressing surrounding and background audio.
31. The method according to claim 26, wherein the auxiliary input includes a user input, the user input being by use of an interface, and includes inputs of phonetics and mnemonics.
32. The method according to claim 23, wherein the concentrating on a specific area is by at least one selected from the group consisting of: zooming, framing, and select drag and drop.
33. The method according to claim 33, wherein motion vectors are used for the video clip.
34. The method according to claim 19, further comprising concentrating on a specific portion of the digital media and converting the specific portion into textual data.
35. The method according to claim 34, wherein the digital media is an audio track and a fingerprint of the audio is generated, searching being by use of the fingerprint.
36. The method according to claim 34, wherein the concentration on a specific portion is by selecting a start and end of a portion of an audio track to form an audio segment, and searching by use of the audio segment.
PCT/SG2005/000289 2004-09-01 2005-08-25 A search system WO2006025797A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US60624704P true 2004-09-01 2004-09-01
US60/606,247 2004-09-01

Publications (1)

Publication Number Publication Date
WO2006025797A1 true WO2006025797A1 (en) 2006-03-09

Family

ID=36000341

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2005/000289 WO2006025797A1 (en) 2004-09-01 2005-08-25 A search system

Country Status (3)

Country Link
CN (1) CN1783073A (en)
TW (1) TW200609775A (en)
WO (1) WO2006025797A1 (en)

Cited By (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007144705A1 (en) * 2006-06-09 2007-12-21 Sony Ericsson Mobile Communications Ab Media identification
EP2012510A1 (en) * 2007-06-24 2009-01-07 xSights Media Ltd. Method and apparatus for connecting a cellular telephone user to the internet
EP2057564A1 (en) * 2006-08-31 2009-05-13 Sony Ericsson Mobile Communications AB System and method for searching based on audio search criteria
EP2075669A1 (en) * 2007-12-31 2009-07-01 High Tech Computer Corp. (HTC) Method of generating a function output from a photographed image and related mobile computing device
WO2009112398A2 (en) * 2008-03-03 2009-09-17 Linguatec Sprachtechnologien Gmbh System and method for data correlation and mobile terminal therefor
WO2010000914A1 (en) * 2008-06-30 2010-01-07 Nokia Corporation Method and system for searching multiple data types
US7899564B2 (en) 2004-11-09 2011-03-01 Bang & Olufsen Procedure and apparatus for generating automatic replay of recordings
WO2011114293A1 (en) * 2010-03-15 2011-09-22 Nokia Corporation Image-based addressing of physical content for electronic communication
US8121830B2 (en) 2008-10-24 2012-02-21 The Nielsen Company (Us), Llc Methods and apparatus to extract data encoded in media content
US8239480B2 (en) 2006-08-31 2012-08-07 Sony Ericsson Mobile Communications Ab Methods of searching using captured portions of digital audio content and additional information separate therefrom and related systems and computer program products
US8295649B2 (en) * 2006-11-08 2012-10-23 Nextgenid, Inc. System and method for parallel processing of images from a large number of cameras
US8311823B2 (en) 2006-08-31 2012-11-13 Sony Mobile Communications Ab System and method for searching based on audio search criteria
US8359205B2 (en) 2008-10-24 2013-01-22 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US8508357B2 (en) 2008-11-26 2013-08-13 The Nielsen Company (Us), Llc Methods and apparatus to encode and decode audio for shopper location and advertisement presentation tracking
US8666528B2 (en) 2009-05-01 2014-03-04 The Nielsen Company (Us), Llc Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content
JP2015005069A (en) * 2013-06-19 2015-01-08 ヤフー株式会社 Distributing device, terminal device, distribution system, distribution method, and transmission program
US8959016B2 (en) 2002-09-27 2015-02-17 The Nielsen Company (Us), Llc Activating functions in processing devices using start codes embedded in audio
US8977293B2 (en) 2009-10-28 2015-03-10 Digimarc Corporation Intuitive computing methods and systems
US9100132B2 (en) 2002-07-26 2015-08-04 The Nielsen Company (Us), Llc Systems and methods for gathering audience measurement data
US9197421B2 (en) 2012-05-15 2015-11-24 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9210208B2 (en) 2011-06-21 2015-12-08 The Nielsen Company (Us), Llc Monitoring streaming media content
US9313544B2 (en) 2013-02-14 2016-04-12 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9336784B2 (en) 2013-07-31 2016-05-10 The Nielsen Company (Us), Llc Apparatus, system and method for merging code layers for audio encoding and decoding and error correction thereof
US9380356B2 (en) 2011-04-12 2016-06-28 The Nielsen Company (Us), Llc Methods and apparatus to generate a tag for media content
CN105898590A (en) * 2015-12-10 2016-08-24 乐视网信息技术(北京)股份有限公司 Media content play method, device and system
US9609034B2 (en) 2002-12-27 2017-03-28 The Nielsen Company (Us), Llc Methods and apparatus for transcoding metadata
US9667365B2 (en) 2008-10-24 2017-05-30 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US9711153B2 (en) 2002-09-27 2017-07-18 The Nielsen Company (Us), Llc Activating functions in processing devices using encoded audio and detecting audio signatures
US9711152B2 (en) 2013-07-31 2017-07-18 The Nielsen Company (Us), Llc Systems apparatus and methods for encoding/decoding persistent universal media codes to encoded audio
US9762965B2 (en) 2015-05-29 2017-09-12 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9940626B2 (en) 2006-06-23 2018-04-10 At&T Intellectual Property I, L.P. Methods and apparatus to provide an electronic agent
US11049094B2 (en) 2014-02-11 2021-06-29 Digimarc Corporation Methods and arrangements for device to device communication

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8620658B2 (en) * 2007-04-16 2013-12-31 Sony Corporation Voice chat system, information processing apparatus, speech recognition method, keyword data electrode detection method, and program for speech recognition
CN102270213A (en) * 2011-04-20 2011-12-07 深圳市凯立德科技股份有限公司 Searching method and device for interesting points of navigation system, and location service terminal
CN103501449A (en) * 2013-10-08 2014-01-08 十分(北京)信息科技有限公司 Method and device for recommending video source associated with television program
CN103747292A (en) * 2014-01-10 2014-04-23 北京酷云互动科技有限公司 Television program-associated application program recommending method and recommending device
CN104078044B (en) * 2014-07-02 2016-03-30 努比亚技术有限公司 The method and apparatus of mobile terminal and recording search thereof
CN106599059A (en) * 2016-11-15 2017-04-26 广州酷狗计算机科技有限公司 Method and device for adding songs
CN109767658B (en) * 2019-03-25 2021-05-04 重庆医药高等专科学校 English video example sentence sharing method and system

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002017090A1 (en) * 2000-08-24 2002-02-28 Friedman, Mark, M. A method and system for automatically connecting real-world entities directly to corresponding network-based data sources or services
US20020038597A1 (en) * 2000-09-29 2002-04-04 Jyri Huopaniemi Method and a system for recognizing a melody
WO2002027600A2 (en) * 2000-09-27 2002-04-04 Shazam Entertainment Ltd. Method and system for purchasing pre-recorded music
WO2003001435A1 (en) * 2001-06-22 2003-01-03 Emblaze Systems, Ltd Image based object identification
US20030087650A1 (en) * 1999-12-23 2003-05-08 Nokia Corporation Method and apparatus for providing precise location information through a communications network
US20030126126A1 (en) * 2001-12-29 2003-07-03 Lee Jin Soo Apparatus and method for searching multimedia object
US20030164819A1 (en) * 2002-03-04 2003-09-04 Alex Waibel Portable object identification and translation system
JP2004038367A (en) * 2002-07-01 2004-02-05 Nec Corp Internet web page connecting system and marketing information collecting system using cellular phone with camera, and method
DE10245900A1 (en) * 2002-09-30 2004-04-08 Neven jun., Hartmut, Prof.Dr. Image based query system for search engines or databases of mobile telephone, portable computer uses image recognition to access more information about objects in image
WO2004029885A1 (en) * 2002-09-24 2004-04-08 Koninklijke Philips Electronics N.V. Image recognition
WO2004029865A1 (en) * 2002-09-25 2004-04-08 Koninklijke Philips Electronics N.V. Capturing a text string
US20040095258A1 (en) * 2000-03-14 2004-05-20 Bosch Johannes Brits Detecting the presence of a vehicle with a particular vehicle registration number

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030087650A1 (en) * 1999-12-23 2003-05-08 Nokia Corporation Method and apparatus for providing precise location information through a communications network
US20040095258A1 (en) * 2000-03-14 2004-05-20 Bosch Johannes Brits Detecting the presence of a vehicle with a particular vehicle registration number
WO2002017090A1 (en) * 2000-08-24 2002-02-28 Friedman, Mark, M. A method and system for automatically connecting real-world entities directly to corresponding network-based data sources or services
WO2002027600A2 (en) * 2000-09-27 2002-04-04 Shazam Entertainment Ltd. Method and system for purchasing pre-recorded music
US20020038597A1 (en) * 2000-09-29 2002-04-04 Jyri Huopaniemi Method and a system for recognizing a melody
WO2003001435A1 (en) * 2001-06-22 2003-01-03 Emblaze Systems, Ltd Image based object identification
US20030126126A1 (en) * 2001-12-29 2003-07-03 Lee Jin Soo Apparatus and method for searching multimedia object
US20030164819A1 (en) * 2002-03-04 2003-09-04 Alex Waibel Portable object identification and translation system
JP2004038367A (en) * 2002-07-01 2004-02-05 Nec Corp Internet web page connecting system and marketing information collecting system using cellular phone with camera, and method
WO2004029885A1 (en) * 2002-09-24 2004-04-08 Koninklijke Philips Electronics N.V. Image recognition
WO2004029865A1 (en) * 2002-09-25 2004-04-08 Koninklijke Philips Electronics N.V. Capturing a text string
DE10245900A1 (en) * 2002-09-30 2004-04-08 Neven jun., Hartmut, Prof.Dr. Image based query system for search engines or databases of mobile telephone, portable computer uses image recognition to access more information about objects in image

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YEH T. ET AL.: "IDeixis - image-based deixis for finding location-based information", CHI EXTENDED ABSTRACTS ON HUMAN FACTORS IN COMPUTING SYSTEMS, April 2004 (2004-04-01), pages 781 - 782 *

Cited By (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9100132B2 (en) 2002-07-26 2015-08-04 The Nielsen Company (Us), Llc Systems and methods for gathering audience measurement data
US8959016B2 (en) 2002-09-27 2015-02-17 The Nielsen Company (Us), Llc Activating functions in processing devices using start codes embedded in audio
US9711153B2 (en) 2002-09-27 2017-07-18 The Nielsen Company (Us), Llc Activating functions in processing devices using encoded audio and detecting audio signatures
US9900652B2 (en) 2002-12-27 2018-02-20 The Nielsen Company (Us), Llc Methods and apparatus for transcoding metadata
US9609034B2 (en) 2002-12-27 2017-03-28 The Nielsen Company (Us), Llc Methods and apparatus for transcoding metadata
US7899564B2 (en) 2004-11-09 2011-03-01 Bang & Olufsen Procedure and apparatus for generating automatic replay of recordings
US7787697B2 (en) 2006-06-09 2010-08-31 Sony Ericsson Mobile Communications Ab Identification of an object in media and of related media objects
KR101010081B1 (en) 2006-06-09 2011-01-24 소니 에릭슨 모빌 커뮤니케이션즈 에이비 Media identification
WO2007144705A1 (en) * 2006-06-09 2007-12-21 Sony Ericsson Mobile Communications Ab Media identification
US8165409B2 (en) * 2006-06-09 2012-04-24 Sony Mobile Communications Ab Mobile device identification of media objects using audio and image recognition
US9940626B2 (en) 2006-06-23 2018-04-10 At&T Intellectual Property I, L.P. Methods and apparatus to provide an electronic agent
US10832259B2 (en) 2006-06-23 2020-11-10 At&T Intellectual Property I, L.P. Methods and apparatus to provide an electronic agent
EP2057564A1 (en) * 2006-08-31 2009-05-13 Sony Ericsson Mobile Communications AB System and method for searching based on audio search criteria
US8311823B2 (en) 2006-08-31 2012-11-13 Sony Mobile Communications Ab System and method for searching based on audio search criteria
US8239480B2 (en) 2006-08-31 2012-08-07 Sony Ericsson Mobile Communications Ab Methods of searching using captured portions of digital audio content and additional information separate therefrom and related systems and computer program products
US8295649B2 (en) * 2006-11-08 2012-10-23 Nextgenid, Inc. System and method for parallel processing of images from a large number of cameras
EP2012510A1 (en) * 2007-06-24 2009-01-07 xSights Media Ltd. Method and apparatus for connecting a cellular telephone user to the internet
EP2075669A1 (en) * 2007-12-31 2009-07-01 High Tech Computer Corp. (HTC) Method of generating a function output from a photographed image and related mobile computing device
US8190195B2 (en) 2008-03-03 2012-05-29 Linguatec Sprachtechnologien Gmbh System and method for data correlation and mobile terminal therefor
WO2009112398A3 (en) * 2008-03-03 2009-11-05 Linguatec Sprachtechnologien Gmbh System and method for data correlation and mobile terminal therefor
WO2009112398A2 (en) * 2008-03-03 2009-09-17 Linguatec Sprachtechnologien Gmbh System and method for data correlation and mobile terminal therefor
WO2010000914A1 (en) * 2008-06-30 2010-01-07 Nokia Corporation Method and system for searching multiple data types
US10134408B2 (en) 2008-10-24 2018-11-20 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US8121830B2 (en) 2008-10-24 2012-02-21 The Nielsen Company (Us), Llc Methods and apparatus to extract data encoded in media content
US8554545B2 (en) 2008-10-24 2013-10-08 The Nielsen Company (Us), Llc Methods and apparatus to extract data encoded in media content
US9667365B2 (en) 2008-10-24 2017-05-30 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US8359205B2 (en) 2008-10-24 2013-01-22 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US11256740B2 (en) 2008-10-24 2022-02-22 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US11386908B2 (en) 2008-10-24 2022-07-12 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US10467286B2 (en) 2008-10-24 2019-11-05 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US8508357B2 (en) 2008-11-26 2013-08-13 The Nielsen Company (Us), Llc Methods and apparatus to encode and decode audio for shopper location and advertisement presentation tracking
US10003846B2 (en) 2009-05-01 2018-06-19 The Nielsen Company (Us), Llc Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content
US10555048B2 (en) 2009-05-01 2020-02-04 The Nielsen Company (Us), Llc Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content
US8666528B2 (en) 2009-05-01 2014-03-04 The Nielsen Company (Us), Llc Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content
US11004456B2 (en) 2009-05-01 2021-05-11 The Nielsen Company (Us), Llc Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content
US8977293B2 (en) 2009-10-28 2015-03-10 Digimarc Corporation Intuitive computing methods and systems
US9444924B2 (en) 2009-10-28 2016-09-13 Digimarc Corporation Intuitive computing methods and systems
US9118771B2 (en) 2009-10-28 2015-08-25 Digimarc Corporation Intuitive computing methods and systems
US9253305B2 (en) 2010-03-15 2016-02-02 Nokia Corporation Image-based addressing of physical content for electronic communication
WO2011114293A1 (en) * 2010-03-15 2011-09-22 Nokia Corporation Image-based addressing of physical content for electronic communication
US9380356B2 (en) 2011-04-12 2016-06-28 The Nielsen Company (Us), Llc Methods and apparatus to generate a tag for media content
US11296962B2 (en) 2011-06-21 2022-04-05 The Nielsen Company (Us), Llc Monitoring streaming media content
US9515904B2 (en) 2011-06-21 2016-12-06 The Nielsen Company (Us), Llc Monitoring streaming media content
US9838281B2 (en) 2011-06-21 2017-12-05 The Nielsen Company (Us), Llc Monitoring streaming media content
US10791042B2 (en) 2011-06-21 2020-09-29 The Nielsen Company (Us), Llc Monitoring streaming media content
US11252062B2 (en) 2011-06-21 2022-02-15 The Nielsen Company (Us), Llc Monitoring streaming media content
US9210208B2 (en) 2011-06-21 2015-12-08 The Nielsen Company (Us), Llc Monitoring streaming media content
US9209978B2 (en) 2012-05-15 2015-12-08 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9197421B2 (en) 2012-05-15 2015-11-24 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9357261B2 (en) 2013-02-14 2016-05-31 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9313544B2 (en) 2013-02-14 2016-04-12 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
JP2015005069A (en) * 2013-06-19 2015-01-08 ヤフー株式会社 Distributing device, terminal device, distribution system, distribution method, and transmission program
US9336784B2 (en) 2013-07-31 2016-05-10 The Nielsen Company (Us), Llc Apparatus, system and method for merging code layers for audio encoding and decoding and error correction thereof
US9711152B2 (en) 2013-07-31 2017-07-18 The Nielsen Company (Us), Llc Systems apparatus and methods for encoding/decoding persistent universal media codes to encoded audio
US11049094B2 (en) 2014-02-11 2021-06-29 Digimarc Corporation Methods and arrangements for device to device communication
US9762965B2 (en) 2015-05-29 2017-09-12 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US11057680B2 (en) 2015-05-29 2021-07-06 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US10694254B2 (en) 2015-05-29 2020-06-23 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US10299002B2 (en) 2015-05-29 2019-05-21 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
CN105898590A (en) * 2015-12-10 2016-08-24 乐视网信息技术(北京)股份有限公司 Media content play method, device and system

Also Published As

Publication number Publication date
TW200609775A (en) 2006-03-16
CN1783073A (en) 2006-06-07

Similar Documents

Publication Publication Date Title
WO2006025797A1 (en) A search system
US9609117B2 (en) Methods and arrangements employing sensor-equipped smart phones
US7787697B2 (en) Identification of an object in media and of related media objects
JP5866728B2 (en) Knowledge information processing server system with image recognition system
CN102782751B (en) Digital media voice tags in social networks
KR100676863B1 (en) System and method for providing music search service
US20060173859A1 (en) Apparatus and method for extracting context and providing information based on context in multimedia communication system
US20090327272A1 (en) Method and System for Searching Multiple Data Types
KR20140114238A (en) Method for generating and displaying image coupled audio
JP2007507746A (en) Speech tagging, speech annotation, and speech recognition for portable devices with optional post-processing
US20140324858A1 (en) Information processing apparatus, keyword registration method, and program
US9525841B2 (en) Imaging device for associating image data with shooting condition information
US20050267749A1 (en) Information processing apparatus and information processing method
Li et al. Melog: mobile experience sharing through automatic multimedia blogging
CN109614482A (en) Processing method, device, electronic equipment and the storage medium of label
KR101087134B1 (en) Digital Data Tagging Apparatus, Tagging and Search Service Providing System and Method by Sensory and Environmental Information
US10043069B1 (en) Item recognition using context data
JP2010021638A (en) Device and method for adding tag information, and computer program
US20140078331A1 (en) Method and system for associating sound data with an image
KR20190066537A (en) Photograph sharing method, apparatus and system based on voice recognition
CN110309324A (en) A kind of searching method and relevant apparatus
KR102051323B1 (en) System and method for recommanding contents based on user context
AT&T untitled
KR100923942B1 (en) Method, system and computer-readable recording medium for extracting text from web page, converting same text into audio data file, and providing resultant audio data file
KR100783108B1 (en) The system and method for providing user preferred broadcasting service

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase