EP1574093B1 - Verfahren und vorrichtung zur organisation von benutzerseitiger information mit meta-information - Google Patents

Verfahren und vorrichtung zur organisation von benutzerseitiger information mit meta-information Download PDF

Info

Publication number
EP1574093B1
EP1574093B1 EP02788411A EP02788411A EP1574093B1 EP 1574093 B1 EP1574093 B1 EP 1574093B1 EP 02788411 A EP02788411 A EP 02788411A EP 02788411 A EP02788411 A EP 02788411A EP 1574093 B1 EP1574093 B1 EP 1574093B1
Authority
EP
European Patent Office
Prior art keywords
information
user provided
meta
audio
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP02788411A
Other languages
English (en)
French (fr)
Other versions
EP1574093A1 (de
Inventor
Wolfgang Theimer
Knut HABERLAND-SCHLÖSSER
Peter Weingart
Reza Serafat
Matthias Lück
Udo Görtz
Klaus Rateitschek
Jakke Mäkelä
Andreas Myka
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Oyj
Original Assignee
Nokia Oyj
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Oyj filed Critical Nokia Oyj
Publication of EP1574093A1 publication Critical patent/EP1574093A1/de
Application granted granted Critical
Publication of EP1574093B1 publication Critical patent/EP1574093B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72451—User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to schedules, e.g. using calendar applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/438—Presentation of query results
    • G06F16/4387—Presentation of query results by the use of playlists
    • G06F16/4393—Multimedia presentations, e.g. slide shows, multimedia albums
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72433—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for voice messaging, e.g. dictaphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/26—Devices for calling a subscriber
    • H04M1/27—Devices whereby a plurality of signals may be stored simultaneously
    • H04M1/274—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc
    • H04M1/2745—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc using static electronic memories, e.g. chips
    • H04M1/27453—Directories allowing storage of additional subscriber data, e.g. metadata
    • H04M1/2746—Sorting, e.g. according to history or frequency of use
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/56—Arrangements for indicating or recording the called number at the calling subscriber's set
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/57—Arrangements for indicating or recording the number of the calling subscriber at the called subscriber's set
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72412—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories using two-way short-range wireless interfaces
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/60—Details of telephonic subscriber devices logging of communication history, e.g. outgoing or incoming calls, missed calls, messages or URLs

Definitions

  • the present invention relates to a method and a device for organizing user provided information with meta-information.
  • the user provided information is user provided audio information.
  • the meta-information is received from context information being available in conjunction with the user provided information and related logically thereto.
  • the present invention is especially useful in the field of mobile terminal devices and mobile network devices which are more and more subjected for handling huge amounts data of different types and kinds which are related to some extent.
  • Document WO 0005914 A1 discloses a meta-service administrative management system wherein an improved interface is provided between an administrative system of a provider of telecommunications services and database network elements supporting communications service subscribed to by each subscriber of the telecommunications services.
  • the improved interface supports single command activation, management of a meta-service and identifies from the meta-service command one or more database network elements supporting individual telecommunications services implicated by the meta-service command.
  • An agent at the interface generates specifically formatted commands to be routed to the identified database network elements.
  • the possibility to access, manage, retrieve etc electronic information migrates to an increasing extent from classical desktop systems (e.g. personal computers) to mobile electronic devices like electronic organizers, personal digital assistants (PDA), mobile communication enabled terminals and the like such that the collecting of personal multimedia information relating to a user and events in the user's life become a predominant feature.
  • the personal multimedia information may be understood to comprise information acquired by audio recording, voice recording, taking pictures, taking videos, retrieving electronic information from any information serving service (document/information server), receiving electronic information via any transmitting service, transmitting bearer, etc.
  • An object of the present invention addresses a method for managing user provided information with meta-information.
  • the meta-information is extracted from information provided and available in the context of the user provided information.
  • the meta-information is employed for retrieval of user provided information.
  • the management enables to organize user provided information for establishing a retrieval system employing the associated meta-information.
  • a further object of the present invention is to provide a method for managing user provided audio information with meta-information, wherein the user provided audio information represents a subset of the user provided information.
  • the meta-information is provided simultaneously to the user provided audio information.
  • the objects of the invention are achieved with a method for organizing user provided information according to claim 1, with a method for retrieving user provided information according to the independent claim 27, with a computer program product for organizing user provided information according to the independent claim 30 and with a mobile terminal device being capable to organize user provided information with meta- information according to the independent claim 31.
  • Preferred embodiments of the invention are disclosed in the dependent claims.
  • the last calls function where a list of phone numbers consisting of a predefined number of put through telephone calls is presented to users through which the users are allowed to scroll to select one of the presented list entries.
  • the list of phone numbers is often associated with the personal telephone directory established and edited by the users themselves. Users tend to select one of those numbers quite often to initiate a new call.
  • the idea in the case of the phonebook is to store the last called numbers and offer them also for future use.
  • FIG. 1a illustrates the structural organization and interaction of a personal logbook according to an embodiment of the invention by means of a conceptional block diagram.
  • the block diagram depicts three example information streams the information of which is passed to the personal logbook 200 for being logged, combined, classified, labeled, indexed and provided to be retrieved.
  • the three example information streams include a user input information stream 101, a context information stream 100 and a transaction / communication details information stream 102.
  • the basic input parameters and properties of the personal logbook 200 are user input information, result information, e.g. resulting from the processing of the user input information and in conjunction with available and suitable context information.
  • a user input is based on the input means provided by an electronic device such as a PDA, an electronic organizer, a mobile communication terminal device and related electronic data processing devices.
  • User input means are operable by the user and the electronic device receives user input information corresponding to the operation of the input means.
  • one or several user inputs or input sequences operated by the following example input means are suitable for being logged:
  • the user input information stream 101 is limited to the aforementioned information being received in consequence on a user input.
  • the user input information stream 101 may be understood as user input information being obtained (received) in consequence on any user operation against the electronic device which also includes any information obtained (received) via any interface providing data exchange capabilities with any other counterpart device.
  • the context data/information is represented by additional- information and conditions under which the user input is carried out or is obtained from the current operation state in which the electronic device is currently active.
  • the context information may comprise:
  • the transaction / communication details information is the third information stream 102 to be fed to the personal logbook 200.
  • the transaction / communication details information may comprise the following information:
  • the transaction / communication details information may be regarded as being part of the context information.
  • the presented classification of available information into three different information streams is be understood as an example embodiment according to the concept of the present invention to enlighten a possible but not necessary classification of the information being logged by the personal logbook 200. It is to be noted, that the classification of information streams which appear in conjunction with the operation of an electronic device and which are capable to be logged into different information streams may be difficult, since certain information may be associated simultaneously with different information streams.
  • the presented concept is also operable in conjunction with a single information stream including one, several selected or all available information. Further, the single information parts assigned to one of the presented example information streams represent single example information parts according to an embodiment of the invention. The inventive concept shall not be understood as being limited to these mentioned single parts of information.
  • the electronic device being in control of these information streams 100, 101 and 102 supplies the information streams to the personal logbook 200.
  • the task of the personal logbook 200 may be split into three different subtasks:
  • the operation of the personal logbook 200 can be expressed in short by the following description.
  • the personal logbook 200 automatically obtains / receives and stores all user input, the application context, location context and time context where this happened and all communication-related information from outside.
  • the information is sorted in accordance with different (pre-defined/user-defined) criteria to facilitate a retrieving access to the stored information by the user. For example, the user is able to trace his or her input / transactions since follow-up activities are most likely based on this context.
  • the advantages with respect to the information overload dilemma can be recognized.
  • the technical pre-conditions to - overcome the information overload dilemma are provided by the logbook which allows to store all information which are encountered in daily life of a user, starting from documents, through audio recordings up to full-day videos of user's life. This information and especially multimedia information may be useless without the implementation of the person logbook 200 which allows to organize this information automatically or at least semi-automatically. Otherwise the user cannot concentrate on the relevant content and use it for the future.
  • Fig. 1b illustrates components allowing to constitute such a personal logbook-200 depicted in Fig. 1a according to an embodiment of the invention by means of an example embodiment of the personal logbook 200.
  • the personal logbook 200 may be implemented by the means of an information receiving interface or information obtaining component 201, a labeling/indexing component 202, a referencing component 203 and a logbook storage 210. These components 201, 202, 203 and 210 relate to the first and second subtasks which are mentioned above in view of the principle operation of the personal logbook.
  • the depicted logbook storage 210 may be supported by an external logbook storage 230 which may be connected to the personal logbook 200 and the logbook storage 210 by a storage management component 213, respectively, which allows to evacuate information thereto and to retrieve information therefrom. Moreover, the logbook storage 210 may be completely substituted by the external logbook storage 230 being operated as the only storage component for storing logbook related information.
  • the personal logbook 200 includes components which allow to retrieve information from the logbook storage 210 and/or, if present and employed for storing, from the external logbook storage 230.
  • the request interface 220 is able to receive one or more requests for information from e.g. an application which has generated an information retrieving request for example in consequence to a user input. Such a request includes one or more commands instructing to retrieve certain information e.g. in accordance with one or more retrieving, sorting and/or filtering criteria.
  • a logbook access component 221 is responsible for the access to the logbook storage 210 as well as to the possibly available external logbook storage 230.
  • a response containing information from the logbook storage 210 and external logbook storage 230, respectively, corresponding to the request for information is generated in cooperation with the logbook access component 221 and the response interface 222 to be transmitted to one or more applications which are capable to received the response or to be returned to the application which has initiated the request for information, respectively.
  • Fig. 1c illustrates a suitable device environment which comprises storage solutions / storage components and storage devices cooperating with the personal logbook according to an embodiment of the invention.
  • the aforementioned external logbook storage 230 may be realized in a large number of ways.
  • the depicted device environment shall illustrate a couple of embodiments of the external logbook storage 230 without limiting thereto.
  • the device environment comprises an electronic device 10 having implemented the personal logbook according to an embodiment of the invention, a detachable connected storage component 11, an external storage device 12 being coupled to the electronic device 10, a local external storage solution 13 and a networked external storage solution 17 each offering storage space/capacity for being used in conjunction with an embodiment of the personal logbook according to the invention.
  • the detachable connected storage component 11 can be realized technically in form of any memory cards (SD card, MultiMedia card, ...) being based on FLASH memory, magnetic memory, ferrite memory etc., which are today employed in mobile electric device products or in form of any once-writable or re-writable storage media, such as optical recording media (e.g. CD-R, CD-RW, a (once-/re-)writable DVD media according to any DVD standard), magnetic recording media and the like. Any known mass storage media are suitable for be used. Correspondingly, such a mass storage technology may be employed as an external storage device 12 being coupled to the electronic device 10 via a wired or a wireless connection.
  • a processing device 13 having implemented a mass storage component such as a desktop computer or a notebook may be employed in agree with the illustrated and described external logbook storage 230.
  • a wireless or wired communication connection provides communication with such a processing device 13.
  • a networked storage device 17 connected via a wireless or wired communication network 15 to the electronic device 10 may provide a dedicated storage space for being employed as external logbook storage 230.
  • any of the wired and/or wireless connection technologies may be used for realizing communication of the electronic device 10 with one of the illustrated external devices 12 and 13.
  • the universal serial bus, the firewire bus or any other serial but also parallel bus may be used for wired connection
  • Bluetooth, wireless local area network, wireless local loop, infrared communication and related wireless communication standards allow to realize a wireless communication 14.
  • Such a networked storage solution as illustrated in conjunction with the networked storage device 17 may be provided by a networked storage service which may be accessed via the internet 18, a wide area network, a local area network, a public land mobile network 16 (such as GSM, PCN, DCS, PCS, UMTS, etc.) or any combination thereof.
  • the networked storage device 17 has the advantage that the provided storage capacity may not be limited due to any storage capacity limitations as given in conjunction with detachable storage media and the provided storage capacity may be always available (on condition that a communication link to the networked storage device 17 is available).
  • Fig. 2a shows a flow chart depicting an operational sequence for operating and in particular for establishing logged information in a logbook component such as illustrated in Fig. 1b according to an embodiment of the invention, respectively.
  • operation S110, operation S120 and operation S130 user input information, context information and/or transaction information / communication details information is obtained and received, respectively.
  • the personal logbook 200 and in particular the information receiving interface 201 receives this set of information.
  • the personal logbook 200 In accordance with the current operation and/or the operational state of the electronic device having implemented the personal logbook 200 only a part of the aforementioned information or additional information may be available for logging.
  • the kind of information to be logged by the personal logbooks 200 may be pre-defined or user-defined and all available information in accordance with the (pre-, user-) defined logging regulations are logged.
  • the received information including at least a part of user input information, context information and/or transaction information / communication details information is combined to a set of information. That means, timely simultaneous received information, logically related and/or logically associated information and the like is taken into relationship and formed to a set of information resulting in an information record.
  • Received user input information is associated with received context information by considering time information or simultaneously receiving thereof, respectively.
  • Received communication details information and/or transaction information are associated with received context information by considering time information or simultaneously receiving thereof, respectively.
  • communication details information and/or transaction information may be received in a time interval and may be related by logical relationship.
  • the resulting information record is labeled and indexed, respectively, i.e. is provided with labeling information and indexing information, respectively.
  • the labeling information and indexing information is used for retrieving the information record or parts of the information record, e.g. subsequently in consequence on an information request.
  • the labeling and indexing information may be a set of individual meta-information, respectively, extracted from the information record and allowing to retrieve, select and/or evaluate the labeled and/or indexed information record in a useful and effective manner.
  • the labeling and indexing information themselves are obtained (extracted) from the information record by employing individual information from the user input information, context information and/or transaction information / communication details information.
  • the labeling information may refer to a certain device operation in conjunction to which the information record has been formed from the received information, i.e. the labeling information may indicate or designate the device application or device function in consequence to which the information has been received and to which the received information relates.
  • a number is dialed by the user of a mobile communication enabled terminal device having embedded the personal logbook 200 according to an embodiment of the invention.
  • the user input information may include either a sequence of numbers representing the dialed telephone number or a name entry from the personal telephone directory associated with a corresponding telephone number.
  • context information the moment of initiating and termination of the telephone call, the duration of the telephone call, the current date, the current geographical position and further available context information are associated with this user input information.
  • an information record may be formed from this information and a label information "telephone number" or "telephone call” may be assigned to this information record wherein that label information may be extracted from the context information providing information about the current focused application, herein the telephone call application.
  • indexing information parts of the available context information may be employed as indexing information.
  • a consecutive retrieving operation may instruct to deliver all telephone calls having been logged, or all telephone calls having been logged on a certain date, within a certain period of time, at a certain geographical position and the like such that the indexing information acts as information enabling a filtering of retrieved information or analogously a sorting of retrieved information, i.e. sorting by time/date, duration of time, geographical position, etc.
  • a user of a mobile communication enabled terminal device having embedded the personal logbook 200 accesses the internet by the means of a WAP (wireless application protocol) browser for paying a hotel bill by electronic payment employing the service of an e-commerce service provider.
  • the personal logbook receives user input information corresponding to the WAP-browser usage, e.g. amongst others a URL (universal resource locator) of the service provider and user input information of a form filled out by the user to order the payment such as address and account information about the hotel.
  • WAP wireless application protocol
  • the user input information are analogously accompanied by context information such as the aforementioned moment of initiating and termination of the use of the WAP-browser, the current date, the current geographical position and further available context information to be are associated with the user input information.
  • transaction information is also available and received by the logbook, where the transaction information may comprise a (unique) transaction ID (identification sequence) and individual information about the addressee, account, amount etc.
  • the resulting information record may be labeled as "transaction" and indexed with selected individual context information and/or individual transaction information.
  • the received information may be combined in two information records, one relating only to the WAP-browsing usage and the other relating to the transaction. Both information records are formed in combination with the aforementioned context information.
  • the first information record may be labeled "WAP browser” and the second information record may be labeled "transaction”.
  • Further transaction information may be received in consequence on the initiated transaction.
  • This transaction information are logged in conjunction with context information.
  • the aforementioned transaction ID may be comprised in all transaction information may be received in consequence on the initiated transaction such that a creating of an indexing information based on the transaction ID allows a user to retrieve all transaction information relating to one and the same transaction procedure even in case the transaction information are logged at different moment of times.
  • all transition information logged at different moment of times may be combined into on information set by a logical relationship being based on the unique transaction ID.
  • the information record is referenced.
  • the personal logbook 200 can be completely separated from the logbook storage where the actual information records are stored.
  • the personal logbook 200 stores only relationships between different instances of information records and points to the actual logbook storage in order to allow information record retrieval.
  • the referencing of the information record allows to determine linking information suitable for retrieving the information record from the actual logbook storage such as an external logbook storage 230. More precisely, the linking information, labeling information and/or the indexing information are stored in the personal logbook 200 itself, i.e.
  • logbook ID storage 212 which allow a useful retrieving, filtering, selecting and/or sorting of the stored information records whereas further information associated with the information stored in the logbook ID storage 212 of the logbook 200 is evacuated to the actual logbook storage, i.e. to the external logbook storage 230 and more precisely to the logbook information storage 232 outside of the personal logbook 200.
  • the above proposed hard separation of the personal logbook and the actual logbook information storage might elaborate, instead only larger portions of information such as audio and video clips might be moved to the external logbook storage 320 and more precisely in the external logbook information storage 232, respectively, but small portions of information such as textual information is kept in the personal logbook 200, the logbook storage 210 and more precisely in the logbook information storage 211, respectively.
  • the linking of information stored in the logbook storage 210 of the personal logbook 200 and stored in the external logbook storage 230 coupled to the personal logbook 200 may be established and realized by the aforementioned referencing information being stored in a dedicated storage area, i.e. in the logbook ID storage 212 of the logbook storage 210 included in the personal logbook 200 and the corresponding counterpart logbook ID storage 231 of the external logbook storage 230.
  • the storage management component 213 manages the information exchange between the logbook storage 210 embedded in the personal logbook 200 and the external logbook storage 230.
  • a user of a mobile communication enabled processing device such as a PDA or an electronic organizer with a communication adapter or a mobile communication terminal device
  • downloads a video presentation (slide show, pictures, large electronic document).
  • the downloaded information is logged by the personal logbook in conjunction with the corresponding context information, that means that the complete video presentation (slide show, pictures, large electronic document) is present in the logbook storage.
  • mobile processing device are commonly equipped with limited storage capacity due to their mobility (size, weight) and usability. The evacuation of such information to an externally coupled storage with high storage capacity may be necessary.
  • the information is stored.
  • the storing of the information record and labeling, indexing and/or referencing information is performed in accordance with the above described storage solution chosen for the personal logbook 200.
  • the labeling, indexing and/or referencing information as well as the information record may be organized in a data base which allows to access the labeling and indexing information in an efficient way for identifying one or several information records which match with one or more retrieval instructions.
  • the total storage capacity is always finite such that a policy / regulation what information to keep and what information to discard is required.
  • a life time can be associated with all information (records), meaning in practice that the oldest information are deleted first.
  • the deletion of timely expired information may be operated automatically at the moment of expiration or at the moment of requiring storage capacity which is occupied by expired information and which is required for storing newly information.
  • expired or older information can be moved to an archive medium which can be embodied analogously to the aforementioned external storage solution for clearing up the logbook storage capacity.
  • An archive medium may be a mass storage of a personal computer (PC), a removable storage medium or a network-based storage solution being connected detachably with the electronic device and the personal logbook for archiving purposes.
  • PC personal computer
  • the same policy / regulation can be applied to both, the logbook, the logbook storage and/or the external logbook storage.
  • the logging information with respect to a certain set of user input information, context information and/or transaction / communication details information is finished.
  • the logging operation embodied as the presented operational sequence being one possible embodiment according to the invention may be operated as a background process of the electronic device having implemented the personal logbook. As soon as a set of information to be logged is received by the personal logbook the aforementioned logging operation is performed by the electronic device and the personal logbook, respectively.
  • a personal logbook is most appealing to a user if the collected information can be utilized with a broad range of applications.
  • the critical operation of the above described logging process is the labeling / indexing operation S150 since only a useful labeling and indexing ensures the potential of the logbook information allowing for managing efficiently and usefully the huge amount of collected information.
  • Fig. 2b shows a flow chart depicting an operational sequence for operating and in particular for retrieving information from a logbook component such as illustrated in Fig. 1b according to an embodiment of the invention, respectively.
  • a request for retrieving information from the personal logbook is received. Such a request may be received by the aforementioned request interface 220 of the personal logbook 200.
  • the one or several information records are retrieved from the personal logbook. 200.
  • the request includes instructions which specifies the kind of information to be retrieved from the personal logbook 200.
  • the request specifies a certain designation or combination of designations in accordance with which previously stored information records may be selected.
  • the instructions command e.g. the logbook access component 221 to retrieve information records associated with a certain labeling information, associated with a certain indexing information and/or any combination of information thereof.
  • the retrieved information records matching with the instructions for retrieving which are included in the request may be further sorted in accordance with additional instructions comprised by the request.
  • the sorting of retrieved information records may analogously be based on a certain designation or combination of designations.
  • the instructions command e.g. the logbook access component 221 to sort the retrieved information records in accordance with a certain associated labeling information, a certain associated indexing information and/or any combination of associated information thereof.
  • the operation S230 and the operation S240 may be a combined operation such that the retrieving, filtering and/or sorting is operated in a combined operation.
  • the retrieved, filtered and/or sorted information records are transmitted to a certain application executed on the electronic device.
  • the addressed application may be coded in the request.
  • the retrieving of information in accordance with instructions comprised by the request may involve the retrieving of information from an external logbook storage acting as evacuation storage.
  • the selection of information records which are matching in accordance with the information request may be performed in conjunction with the logbook ID storage 212 including labeling information, indexing information and/or linking (referencing) information.
  • the actual information are retrieved by employing the corresponding linking (referencing) information from the evacuation storage, i.e. the external logbook storage 230 including a corresponding logbook ID storage 231 and a logbook information storage 232 including the information to be retrieved.
  • a request for retrieving information may be generated and transmitted to the personal logbook by user selection of a fixed device feature or application feature such as the command (operation menu item) for retrieving the last dialed telephone numbers, the last received telephone calls or the last missed telephone calls.
  • the generation of a corresponding request for retrieving information about the last dialed telephone numbers, the last received telephone calls or the last missed telephone calls from the personal logbook may be initiated by selecting a certain application function or by operating a certain dedicated key of the mobile communication terminal device. Since the logbook logs all user input information, context information and communication details information, the corresponding respective information to be retrieved is present in the personal logbook and are presented to the user correspondingly. Similarly, a history list of last transactions and the like may be requested by the user and finally presented correspondingly to the user.
  • a request may comprise a user inputted text string (e.g. a name, an address etc) in accordance to which information records are to be retrieved which include a matching text information.
  • a user inputted text string e.g. a name, an address etc
  • This example can result in a response including a plurality of different information records, such as an information record relating to a dialed telephone number, an information record relating to a calendar entry, an information record relating to an editing of the personal telephone directory and further information records containing the inputted text string.
  • the inputted text string may be compared with the labeling and/or indexing information associated to the information records or may be compared with the whole information content of the information records.
  • a request may comprise information being associated to an application context. For example a user currently scrolls through the personal telephone directory and discovers a name to which he wants to retrieve further information.
  • a request for retrieving information from the personal logbook is generated, which comprises for example the designation of the discovered entry of the personal telephone directory.
  • the personal logbook serves all information logged and stored therein which match with the provided designation extracted from the current applicational context. The generation of logbook information request in accordance with certain applicational context is also applicable to other applications and/or device functions having an analogous context.
  • the retrieved and finally presented logbook information in consequence on a request may be filtered or sorted in a pre-defined or user-defined way.
  • History lists are usually sorted chronologically (backward) and limited to a certain number of entries (e.g. ten entries).
  • the sorting of the retrieved information records may be based on a usage frequency information, that means, that information records associated with a low usage frequency are sorted at the end of a resulting sequence of information records or are filtered out. Further sorting and filtering procedures are possible since the context information offers a huge number of individual information allowing an effective and useful sorting and filtering.
  • the personal logbook may be based on a data base technology for organizing the described storing structure.
  • the coding of the labeling / indexing information may be based on a device independent markup language coding.
  • Suitable coding technology for the data being managed by personal logbook may be a coding technology in accordance with the extended markup language (XML) or binary encoded XML in conjunction with a corresponding document type description (DTD) and related markup language representations.
  • XML extended markup language
  • DTD document type description
  • the content and presentation / organization of the logbook data may be structured separately and the content can be referenced by an addressing information (referencing information) like provided with a uniform resource locator (URL), uniform resource indicator (URI) and comparable resource addressing schemes:
  • a first detailed description will relate to labeling and/or indexing user provided information with information obtained from an electronic calendar and from calendar entries, thereof.
  • a second detailed description will relate to labeling and/or indexing user provided audio information with information being obtained from the user actions being operated during the audio recording and provided by user actions during the audio recording.
  • a third detailed description will relate to a method for embedding information of any type into a user provided audio information. Especially, the method defines a new type of markup language encoding being based on a language encoding system and alphabet for embed characters (symbols) of any mark-up language around or within the audio clip.
  • multimedia information that is considered as being relevant with regard to a specific person is restricted to those information objects that directly contain this person, i.e., its face, its voice in case of referring to the automatic recognition.
  • the present invention relates further to a method for labeling information and more particular multimedia information with calendar data resulting in both highly reliable and cost-efficient attributes applicable for retrieval thereof.
  • the inventive concept makes use of calendar entries in order to label / index information.
  • This kind of labeling / indexing is supposed to be reliable because the context information obtained therefrom has been contributed by the user manually itself and the employing of this manually entered context information is cost-efficient since this tapping of user knowledge is only targeting information that has already been stored by user within its personal calendar.
  • calendar entries allow to define alerts triggered by time and/or date. Further, calendar entries may be inter-linked and intra-linked such that information may be passed from or to other applications such as contact information included in a calendar entry and being inter-linked with a personal telephone directory for selecting fast and easily a certain telephone directory entry and vice versa.
  • calendar information for labeling / indexing multimedia information requires certain perquisites relating to the semantic structuring of the calendar information as well as context information relating the multimedia information to be labeled / indexed.
  • the presented semantic structure of calendar entries is just an example.
  • the inventive concept primarily requires date and time information of calendar entries which are available independently from the specific implemented semantic structure due to the concept of the electronic calendar itself.
  • the semantic structure of calendar entries relates to the labeling / indexing of the multimedia information.
  • the labeling and/or indexing information finally associated with the multimedia information is extracted from the calendar entries such that the proposed semantic structure ensures that the extracted labeling and/or indexing information are reliable and sensible.
  • the proposed semantic structure of calendar entries may be varied, modified or adapted to certain conditions without departing the scope of the inventive concept.
  • Fig. 3a depicts two schematically structured calendar entries #i and #j.
  • the calendar entries include the above purposed semantic fields for structuring reliably the information comprised by the calendar entries.
  • the first example calendar entry #i specifies a meeting (field "type") near the north pole (field “where”) with Santa Claus and Rudy “Rednose” Reindeer (field “who") a Friday, December 6 th , 2002 (field “date”) starting at 6.00 pm and ending at 7.00 pm (field "time”).
  • the calendar entry #i also includes a free text note (field "notes").
  • the second example calendar entry #j specifies a telephone conference (field "type") with Jane (field “who") a Friday, December 6 th , 2002 (field “date”) starting at 7.10 pm and ending at 7.25 pm (field “time”).
  • the calendar entry #j also includes a free text note (field "notes”).
  • calendar entries #i and #j it can be recognized that the purposed semantic structuring of calendar entries is very useful since the semantic structuring ensures that information, extracted from calendar entries and be used for labeling / indexing multimedia information represent reliable information guaranteeing high retrieval precision in case that a retrieval is based on such information.
  • multimedia information may be labeled and/or indexed with name information which can be extracted reliably from the "who" field or multimedia information may be labeled and/or indexed with place information which can be extracted reliably from the "where" field.
  • a point of time or a period of time has to be allocable to the multimedia information to be labeled / indexed.
  • a personal collection of multimedia information are provided and available that reflects the personal recordings of reality of a user.
  • This collection may includes continuous multimedia recordings, automatically taken multimedia snapshots or manually triggered multimedia snapshots where snapshot is used as a general term for performing recordings of any type of multimedia information comprising images, pictures, picture sequences, video sequences, legacy documents and the like.
  • Timestamps are attachable to each entry of the personal collection of multimedia information where the timestamps may specify a moment of time or a period of time at which the recording has been performed.
  • the inventive concept is being based on the main idea to associate calendar entries and multimedia information by matching time information provided by the calendar entries with the time information (timestamps) associated to the multimedia information.
  • a matching of the time information obtained therefrom allows to associate one or more calendar entries to one or more multimedia information sequences and particular to assign information extracted from the matching one or more calendar entries as labeling / indexing information of one or more multimedia information sequences.
  • the electronic calendar is used to label / index multimedia information in a retrospective way.
  • the multimedia information is not only labeled with a specific labeling information where the labeling information is clearly assignable to the multimedia information, but also information that is extractable from the one or more matching calendar entries and that is related in a general sense.
  • Fig. 3b shows a plot diagram illustrating example membership functions used for associating a membership grade to certain events according to an embodiment of the invention.
  • the abscissa comprises time values whereas the ordinate comprises membership grades.
  • a scheduled meeting begins at the moment of time “start” and ends at the moment of time “stop”
  • the period of time between “start” and “stop” is the timeframe of this meeting. Both the moment "start” and “stop” are indicated in the plot diagram shown in Fig. 3b .
  • the membership function 1 and the membership function 2 both start rising from a membership grad value zero at a first distinct moment of time before the beginning of the timeframe of the calendar entry. Consecutively, the membership functions rise with different gradients up to the maximum membership grade value one before being valid within the timeframe for certain different periods of time. Subsequently, the membership functions drop again with different gradients and become zero at a second distinct moment of time after the ending of the timeframe of the calendar entry.
  • the rising of the membership function before the actual beginning of the timeframe and the dropping of the membership function after the actual ending of the timeframe ensures that multimedia information are associated with the respective calendar entry to which the timeframe belongs in case that the timestamp of the multimedia information is within the timeframe and is in the projecting periods of time defined by the exceeding definition area of the membership function in respect to the timeframe of the respective calendar entry. Examples will be given below to clarify this.
  • both membership functions 1 and 2 are chosen identical and the membership functions are defined as sectionalized linear curves.
  • the rising and reaching point as well as the complete shape of the membership function is not only limited to the depicted example ones but any shape including to points of onset and termination is possible.
  • the implemented possibility distribution should be based on the habits of the user who is the owner of both the multimedia information and the electronic calendar. However, the employed distribution has to be adjusted manually only once. Afterwards this adjusted distribution is valid for all multimedia information having been recorded in the past and will be recorded in the future and calendar entries, respectively.
  • Fig. 3c depicts an operational sequence for generating one or more labeling / indexing information of multimedia information according to an embodiment of the invention.
  • the conceptional idea can be reduced to a selection of operations of the operations depicted in Fig. 3c .
  • the object to be reached is to label and/or index multimedia information with one or more calendar entries and information extracted from the one or more calendar entries, respectively.
  • a timestamp is determined which is associated with the multimedia information to be processed. This timestamp is compared with the available calendar entries and the matching ones are selected thereof. Membership grades are obtained from the aforementioned membership function assigned to the timeframes of the matching calendar entries, where each membership grade relates to each matching calendar entry. Finally the multimedia information is labeled /indexed with the one or more matching calendar entries and information extracted from the one or more matching calendar entries are used for labeling / indexing the multimedia information, respectively.
  • an operation S300 the method for labeling a multimedia information is started.
  • a moment of time or a period of time is obtained which relates to a recording event of the multimedia information, respectively.
  • the moment of time or the period of time associated with the multimedia information may be extracted from the multimedia information or may be retrieved from information being provided with or being assigned to this multimedia information, respectively.
  • an operation S320 it is detected whether a moment of time or a period of time is associated with the multimedia information. In case of a moment of time being determined to be associated, the operational sequence is continued with an operation S325 whereas otherwise, i.e. in case of a period of time being determined to be associated, the operational sequence is continued with an operation S355.
  • the total number of calendar entries available is searched to find matching calendar entries. That is, the obtained moment of time is compared with the timeframes and the extended timeframes of the calendar entries, respectively, where an extended timeframe is defined by the period of time in which the used membership function assigned to a timeframe of a calendar entry is not equal to zero.
  • a calendar entry matches in case the moment of time is within the timeframe and the extended timeframe, respectively.
  • a membership grade is determined from the membership function assigned to the timeframe of the current selected matching calendar entry in accordance with the moment of time being determined as being associated to the multimedia information.
  • the membership grade is employed as a reliability value which indicates to which extent the currently selected calendar entry relates in any kind to the multimedia information.
  • the multimedia information is labeled and indexed, respectively.
  • the labeling / indexing is based on information being available from the current selected calendar entry, i.e. the whole calendar entry itself or the whole information contained in the calendar entry, selected information extracted from the calendar entry and the like, wherein the extracted information may be used in case that the calendar entry includes semantically structured information (fields).
  • an operation S345 it is determined whether all matching calendar entries have been processed. In case there exists one or more remaining calendar entries which have not be processed up to now, the operational sequence returns to operation S330, whereas otherwise the operational sequence is continued with an operation S350.
  • the total number of calendar entries available is searched to find matching calendar entries. That is, the obtained period of time is compared with the timeframes and the extended timeframes of the calendar entries, respectively, where an extended timeframe is defined by the period of time in addition with an overhead at the beginning and ending of the timeframe in accordance with the corresponding used membership function.
  • a calendar entry matches in case the period of time overlaps with timeframe and the extended timeframe, the total number of calendar entries available is searched to find matching calendar entries. That is, the obtained moment of time is compared with the timeframes and the extended timeframes of the calendar entries, respectively, where an extended timeframe is defined by the period of time in which the used membership function assigned to a timeframe of a calendar entry is not equal to zero.
  • a calendar entry matches in case the moment of time is within the timeframe and the extended timeframe, respectively, either completely or at least partially.
  • an operation S365 it is determined whether the period of time being identified to be associated with the multimedia information matches partially with the timeframe and the extended timeframe of the current selected calendar entry, respectively. In case the period of time and the (extended) time frame partially overlap the operational sequence is continued with operation S367. In case the period of time and the (extended) timeframe completely overlap and the period of time is within the (extended) timeframe, respectively, the operational sequence is continued with operation S370.
  • the multimedia information is divided into one or more sections of multimedia information having corresponding sectionalized periods of time.
  • One of the section with a corresponding associated sectionalized period of time matches with the timeframe and the extended timeframe of the current selected calendar entry, respectively.
  • the remaining sections having corresponding associated sectionalized periods of time do neither match nor overlap with the into the timeframe and the extended timeframe of the current selected calendar entry, respectively.
  • the multimedia information is partitioned into one or more parts of multimedia information having corresponding partitioned periods of time.
  • the partitioned periods of time may have substantially the same period value, e.g. 5 minutes.
  • the value of the partitioned period of time of the last part obtained from the multimedia information by partitioning may differ from the desired period value.
  • the period value of the partitioned periods of time may be pre-defined and/or user-defined.
  • a membership grade is determined from the membership function assigned to the timeframe of the current selected matching calendar entry in accordance with the partitioned period of time.
  • the membership grade is employed as a reliability value which indicates to which extent the currently selected calendar entry relates in any kind to that current selected part of the multimedia information.
  • the multimedia information is labeled and indexed, respectively.
  • the labeling / indexing is based on information being available from the current selected calendar entry, i.e. the whole calendar entry itself or the whole information contained in the calendar entry, selected information extracted from the calendar entry and the like, wherein the extracted information may be used in case that the calendar entry includes semantically structured information (fields).
  • partitioning of the multimedia information such a partitioned multimedia information is labeled and indexed section-wise for a certain period of time, increasing the value of the assigned membership grade as a quantity for evaluating the retrieval reliability.
  • the partitioning allows to establish a fine reliability value grid applied to the multimedia information.
  • an operation S390 it is determined whether all parts of the partitioned calendar entries have been processed. In case there exists one or more remaining parts which have not be processed up to now, the operational sequence returns to operation S375, whereas otherwise the operational sequence is continued with an operation S395.
  • an operation S395 it is determined whether all matching calendar entries have been-processed. In case there exists one ore more remaining calendar entries which have not be processed up to now, the operational sequence returns to operation S360, whereas otherwise the operational sequence is continued with an operation S350.
  • a following retrieval of information is preferably based on the typed labeling / indexing information being determined and associated to the multimedia information, however, free text search is possible as well.
  • Fig. 3d illustrates example membership function plots corresponding to the calendar entries depicted in Fig. 3a and several timestamps of different individual multimedia data according to an embodiment of the invention.
  • Fig. 3e shows a depiction of context information of a selection of individual multimedia data including indexing information in accordance with the calendar entries depicted in Fig. 3a and obtained in accordance with an embodiment of the present invention.
  • the Fig. 3d and Fig. 3e relates to the same individual multimedia data and are described below.
  • the plot diagram referred to in Fig. 3d contains membership functions for the example calendar entry #i and the example calendar entry #j which are both extended by certain different overheads in time.
  • the period of time defined in calendar entry #i extends from 6:00 pm to 7:00 pm, whereas the corresponding associated extended timeframe extends from 5:58 pm to 7:06 pm.
  • the period of time defined in calendar entry #j extends from 7:10 pm to 7:25 pm, whereas the corresponding associated extended timeframe extends from 7:04 pm to 7:28 pm.
  • Both the membership function associated to calendar entry #i and the membership function associated to calendar entry #j are sectionalized linear membership functions.
  • the membership function #i rises from 5:58 pm to 6:02 pm (from membership grade zero to membership grade one) and drops from 6:46 pm to 7:06 pm (from membership grade one to membership grade zero), respectively, whereas the period of time between is constant (membership grade one).
  • the membership function #j rises from 7:04 pm to 7:08 pm (from membership grade zero to membership grade one) and drops from 7:20 pm to 7:28 pm (from membership grade one to membership grade zero), respectively, whereas the period of time between is constant (membership grade one).
  • the membership function #i and membership function #j differs in their shape concerning the dropping period. In this example, the shape of the membership functions may depend on the length of the timeframe.
  • the periods of time relating to the periods of time of recording an audio multimedia data #a, an audio multimedia data #b and an audio multimedia data #c are schematically depicted above the membership functions as gray colored horizontal bars.
  • the moments of time relating to the moments of time of taking an image multimedia data #a and an image multimedia data #b is schematically depicted above the membership functions as gray colored dots.
  • the respective exact periods of time of the audio multimedia data #a, the audio multimedia data #b and the audio multimedia data #c as well as the moments of time of the image multimedia data #a and the image multimedia data #b can be seen in the respective context information illustrated in Fig. 3e .
  • the audio multimedia data #a is recorded within the period of time extending from 6:14 pm to 6:22 pm.
  • the period of time of the audio multimedia data #a matches with the (extended) timeframe of calendar entry #i and the audio multimedia data #a is partitioned into two parts in accordance with a partitioning time value of 6 minutes, the first part extending from 6:14 pm to 6:20 pm whereas the second part extending from 6:20 pm to 6:22 pm.
  • the first as well as the second part are within the extended timeframe of calendar entry #i such that both parts of the audio multimedia data #a are labeled / indexed with the calendar entry #i (depicted herein) or information obtained from the calendar entry #i (not depicted herein), respectively.
  • the parts (indicated by indices 1 and 2) are illustrated additionally in the plot diagram shown in Fig. 3d as black colored horizontal bars in accordance with their respective membership grades and in correspondence with their partitioned periods of time.
  • the audio multimedia data #b is recorded within the period of time extending from 6:46 pm to 7:10 pm.
  • the period of time of the audio multimedia data #b matches with the (extended) timeframe of calendar entry #i and with the (extended) timeframe of calendar entry #j.
  • labeling / indexing is performed with respect to both the calendar entry #i as well as the calendar entry #j.
  • the period of time of the audio multimedia data #b matches with but do not fit to the (extended) timeframe of calendar entry #i such that audio multimedia data #b is sectioned and the fitting audio multimedia data #b section extending from 6:46 pm to 7:06 pm will be further processed.
  • the fitting audio multimedia data #b section is partitioned into four parts in accordance with a partitioning time value of 6 minutes, the first part extending from 6:46 pm to 6:52 pm, the second part extending from 6:52 pm to 6:58 pm, the third part extending from 6:58 pm to 7:04 pm whereas the fourth part extending from 7:04 pm to 7:06 pm.
  • the first to fourth parts are within the extended timeframe of calendar entry #i such that these parts of the audio multimedia data #b section are labeled / indexed with the calendar entry #i (depicted herein) or information obtained from the calendar entry #i (not depicted herein), respectively.
  • the parts (indicated by indices 1, 2, 3 and 4) are illustrated additionally in the plot diagram shown in Fig. 3d as black colored horizontal bars in accordance with their respective membership grades and in correspondence with their partitioned periods of time.
  • the section/part is within the extended timeframe of calendar entry #j such that this section/part of the audio multimedia data #b is labeled / indexed with the calendar entry #j (depicted herein) or information obtained from the calendar entry #j (not depicted herein), respectively.
  • the part (indicated by index 5) is illustrated additionally in the plot diagram shown in Fig. 3d as black colored horizontal bars in accordance with its respective membership grade and in correspondence with its period of time.
  • the membership grade within a certain period of time may be obtained from the membership function in different way.
  • the above presented way relates to an averaging process of membership grade values within that certain period of time.
  • the membership grade may be obtained by extracting a maximum or a minimal membership grade value existing within that certain period of time, respectively. Further mathematical method for obtaining sensitive membership grades are possible and applicable.
  • the audio multimedia data #c is recorded within the period of time extending from 7:12 pm to 7:28 pm.
  • the period of time of the audio multimedia data #c matches with the (extended) timeframe of calendar entry #j and the audio multimedia data #c is partitioned into three parts in accordance with the partitioning time value, the first part extending from 7:12 pm to 7:18 pm, the second part extending from 7:18 pm to 7:24 pm whereas the third part extending from 7:24 pm to 7:28 pm.
  • the first, second as well as the third part are within the extended timeframe of calendar entry #j such that the three parts of the audio multimedia data #c are labeled / indexed with the calendar entry #j (depicted herein) or information obtained from the calendar entry #j (not depicted herein), respectively.
  • the parts (indicated by indices 1, 2 and 3) are illustrated additionally in the plot diagram shown in Fig. 3d as black colored horizontal bars in accordance with their respective membership grades and in correspondence with their partitioned periods of time.
  • the image multimedia data #a and image multimedia data #b are taken at 6:42 pm and 18:43 pm where both moments of time are within the (extended) timeframe of calendar entry #i.
  • the image multimedia data #a and image multimedia data #b are labeled / indexed with the calendar entry #i (depicted herein) or information obtained from the calendar entry #i (not depicted herein), respectively.
  • the image multimedia data #a and the image multimedia data #b are illustrated additionally in the plot diagram shown in Fig. 3d as black colored dots in accordance with their respective membership grades and in correspondence with their moments of time.
  • the above described method as been embodied to attribute multimedia information with calendar entry information being based on a timestamp comparison operation. Further context and sensor information may be included in the labeling operation of multimedia information.
  • One possible additional information of interest for attributing multimedia information is a location information in form of geographical position information or in form of mapped location information.
  • Existing devices like GPS devices or mobile communication terminal device implementing positioning services, provide for data on location information of the devices. Based on the timestamps both of the location/position information and the multimedia information location/position information can be attributed to multimedia information.
  • Location/position information as provided for e.g. by GPS components may not be sufficient in some cases; in contrast the resolution of positioning information as provided for by a calendar entry information is only depending on the indications of the user (street number, floor number, room number, etc.). In case both kinds of information, i.e. location/position information such as GPS data and location as part of a calendar entry, are available, the one should be selected that provides for the better precision. At the same time, the other data can be used in order to validate the used location/position information.
  • the location/position information obtain for example from a GPS component could even be used in order to validate a complete calendar entry and/or to contribute to the membership grade determined in conjunction with the calendar entry.
  • the labeling of personal multimedia information by means of personal calendar entries provides for a highly reliable approach towards the labeling / indexing and subsequent retrieval of multimedia information.
  • this inventive approach provides for a better recall and also for a better precision. Further, this gain in retrieval quality is reached with almost no additional costs including both any kind of required efforts from the side of the user and computational costs.
  • the presented method for extracting meta-information being used for retrieval of information associates a huge amount of meta-information with any user provided information (multimedia information).
  • Each user provided information overlapping in time with each calendar entry is provided with meta-information for retrieval provided by the information comprised in the each calendar entry. That means for example relating to a name field comprising information about persons being members of a meeting, user provided information, e.g. recorded dictation, images, downloaded information such as documents and the like, is associated with each name of the meeting members such that a retrieval in accordance with one of the name of the meeting members results in a list of retrieval results comprising at least those information parts which have been provided by the user of the electronic device during the meeting.
  • a membership grade value which serves as a measure for evaluating a retrieval.
  • Such a measure is useful in view of a huge amount of retrieval results for sorting and/or filtering the retrieval results providing a reliable processing of the retrieval results.
  • a pre-defmed or user-defined membership grade threshold may be defined and only retrieval results are presented to a user on a retrieval instruction in case the membership grade values defined for each retrieval results exceed the defined membership grade threshold which enables to find the intended retrieval result easier since the number of retrieval results is limited therewith.
  • the membership grade values are suitable for sorting multiple retrieval results for example in a value dropping sequence.
  • the user may start a manually selecting of relevant retrieval results beginning with those retrieval results which have the highest membership grade values.
  • the context information is to be understood as any information selected and indicated by a user to be associated to the audio information.
  • dictation machines take account of this natural human behavior but conventionally, dictation machines are limited to a kind of serial operation mode only allowing to record speech/voice information.
  • audio information are also subjected to the problem that information retrieval systems may not have enough interpretable information about the content of the audio information to allow reliable retrieval and provide a useful results thereof.
  • serial audio recording of human ideas and views is indeed a most natural process therefor but it is advantageous in view of the expressing of ideas and view as well as in regard to ideas and views to associate accompanying context information to the audio information, which may be any sensor information, any information allowing a reliable labeling / indexing of the audio information, any information amplifying or completing the audio information such as images, sketches, video sequences and the like.
  • Fig. 4a illustrates an operational sequence for recording audio information and associating additional information of any type to the recorded audio information according to an embodiment of the invention.
  • the operational sequence for recording audio information may be operated by an adequate apparatus or device at least being equipped with an audio sensor such as a microphone for sensing audio signals, an audio recording component for recording either analog audio signals or a digitized audio information being obtained by an analog-to-digital converter which is supplied with the analog audio signals, a recording storage again either suitable to receive and store analog audio signals or digitized audio information, and means for controlling the recording operation such as specific keys, a keypad or a keyboard. More particularly, the apparatus or device may comprise additionally an audio reproducer such as a speaker for reproducing the recorded audio information.
  • an audio sensor such as a microphone for sensing audio signals
  • an audio recording component for recording either analog audio signals or a digitized audio information being obtained by an analog-to-digital converter which is supplied with the analog audio signals
  • a recording storage again either suitable to receive and store analog audio signals or digitized audio information
  • means for controlling the recording operation such as specific keys, a keypad or a keyboard.
  • the apparatus or device may comprise additionally an audio reproducer such
  • the audio recording component may further include an audio compressing component for compressing digital audio information of the analog-to-digital converter and for decompressing the compressed digital audio information to be supplied to a digital-to-analog converter for being reproduced.
  • the recording storage may be any storage device and/or medium capable to store and provide digital information. Conventionally, memory, magnetic disk storage, removable storage medium devices are mass storage systems connected via a wired or wireless communication network are used for storing digital information.
  • the apparatus or device implementing audio recording capability should further have the capability to generate, retrieve, receive, process or the like the additional information.
  • the operational sequence is operated by a mobile terminal and more particular by a mobile communication terminal having audio recording and reproducing capability and being able to handle information such as multimedia information. More precisely, a personal digital assistant (PDA) or an enhanced mobile communication device having any multimedia capability.
  • PDA personal digital assistant
  • a user input is awaited for and on a trigger signal caused by a user input for example by user operating of any dedicated keys a corresponding key selection signal is received for being evaluated.
  • the audio recording is started.
  • the sensed audio signals are converted into an audio (information) stream representing the digital audio information.
  • the digital audio information may represent compressed digital audio information in accordance resulting from applying any audio compression codec of state of the art.
  • the audio stream obtained during the recording of audio signals is continuously stored in a digital storage, e.g. in a memory or in a mass storage such as a removable magnetic or optical medium storage.
  • a storage employing removable magnetic or optical medium may be recommended for storing the audio stream.
  • the audio signal recording having been started by the user with operating the "start" key may be operated continuously until a dedicated stopping or pausing of the audio signal recording is instructed by the user.
  • keywords are recorded.
  • the recording of keywords may be operated in parallel to the audio signal recording. That is, the resulting audio stream from the audio signal recording may be either split into two different audio (information) streams, one being the aforementioned audio (information) stream and the other being an audio keyword (information) stream such that it is possible to store both audio streams separately, for example one on a removable storage medium being coupled to the device having audio recording capability the and the other in the memory of the device having audio recording capability.
  • the aforementioned audio (information) stream relating to the continuous audio recording may be paused such that the sensed audio signals are only converted into the audio keyword (information) stream for being stored separately from the aforementioned audio (information) stream.
  • the attaching command a user to associate additional information of any type to the audio information being currently under recording where the audio signal recording may be continuously operated or may be interrupted.
  • the user may obtain any additional information of any type to be associated to the recorded audio information.
  • the obtaining of that further information may be performed by any procedure provided for by the used device.
  • the recording device may have implemented a digital camera which allows to take images or video sequences. These taken images and/or video sequences may amplifying the audio information with regard to its intellectual content.
  • information to be associated to the recorded audio information may be obtained via a wired or wireless communication interface from an information servicing entity or may be retrieved from the memory of the device or from a storage medium coupled to the device.
  • the recording of audio signals is paused.
  • the audio signal recording may be resumed on instruction of the user.
  • the recording of audio signals is continued.
  • the continuing of the audio signal recording allows a user to resume the audio signal recording after having instructed to pause the audio signal recording (compare with operation S450 and operation S455).
  • the audio signal recording is stop completely.
  • the audio information resulting from the continuous audio stream is complete.
  • context information during the audio signal recording are accumulated in an associated audio context information. Additionally to the audio information each activity having been performed during the audio signal recording by the user has been logged such that a complete protocol / description can be generated therefrom.
  • context information relating directly to the audio signal recording are included into the protocol, that is, e.g. timestamp information relating to starting / stopping of the audio recording, keyword recording operation, pausing / resuming operation, linking information relating to address information where to retrieve the audio information resulting from the audio (information) stream, the audio information resulting from the keyword audio (information) stream and/or additional information of any type being instructed by the user to be associated with the audio information or audio recording event.
  • more common context information can be added to the protocol / description like identification of the recording device, of the speaker, of the employed storage (e.g. identification of the storage medium), location/position information relating to the place of recording and more.
  • the operation of the audio signal recording has been described in view of dedicated keys operable with a user for instruction the described operations / commands.
  • Such commands for controlling the audio signal recording may be also generated by operating keys having multiple assignments or by a textual or graphical menu offering items corresponding to audio signal recording functions for being selected by the user.
  • Fig. 4b shows a sequence of operations in time illustrating an example usage sequence according to an embodiment of the invention.
  • the operations in time may be seen as a dictation of a user summarizing the results of a meeting.
  • the user starts the audio signal recording for dictating the summary for later usage or for later remembering.
  • the mobile terminal On indication of the user to start audio signal recording (the user may select the key "start") the mobile terminal having audio recording capability starts recording to receiving audio stream to a removable storage medium (magnetic disk, optical disk, memory card etc) coupled to the mobile terminal device via a respective storage medium interface (drive, interface etc).
  • a designation of the may be defined manually by the user or automatically by the recording component to be assigned to the audio information for later retrieval.
  • the resulting audio information is provided with the designation "AUDIO1".
  • the starting time (timestamp) of the audio recording is stored for example in the memory of the device or in a memory assigned to the audio recording component which is responsible for the audio signal recording.
  • the user dictates text to the mobile terminal which is sensed by an audio sensor, converted into a digital audio stream and stored accordingly.
  • the user indicates that keywords should be recorded and correspondingly, pushes and holds the "keyword" key.
  • the audio signal recording of one or more keywords is initiated.
  • the audio signal recording of keywords is stored separately to the audio information dictated before and the audio recording operation initiated before is either continued or paused during the recording of the one or more keywords.
  • the starting time of the keyword recording is stored.
  • the stopping time of the keyword may be stored
  • the user instructs subsequently to pause the audio recording by selecting a dedicated "pause” key.
  • the recording of the audio signals is paused accordingly.
  • the starting time of the initiated pause is stored.
  • the user may user other function or other applications provided for by the mobile terminal.
  • the user takes an image with a digital camera coupled in any way to the mobile terminal for image data transmission or built in the mobile terminal.
  • the user indicates that this image is to be associated with (attached to) the recorded audio information for example by operating a dedicated "attach" key.
  • the picture may be taken from a sketch board to amplify the dictation.
  • the taken image is stored (here in the memory or any other available storage area) and a manually or automatically defined designation (herein "IMAGE1”) is assigned to the image which enable to retrieve the image later.
  • IMAGE1 manually or automatically defined designation
  • the user resumes the audio signal recording by operating the "continue" key.
  • the resuming time is stored.
  • the user carries on dictating information.
  • the audio signal recording is finished on user instruction or on operation of the "stop" key by the user, respectively.
  • the stopping time of the audio recording is stored.
  • the mobile terminal may evacuate information stored in a non-permanent storage up to now to a permanent storage if available.
  • the protocol / description is generated in accordance with the above described user and terminal operations. A corresponding example description which may be obtained is shown the following Fig. 4c .
  • Fig. 4c shows a possible example description in view of the example usage sequence presented in Fig. 4b and according to an embodiment of the invention.
  • the description / protocol of the audio recording may be coded as an extended markup language (XML) coded description / protocol.
  • XML extended markup language
  • DTD document type description
  • the example file "DESCRIPTION1" shown in Fig. 4c comprises a header section (lines 1 to 3) which includes information about the storage medium / storage device where at least the recorded audio information "AUDIO1" is stored, information about the recording device and information about the time at which the audio recording as be taken place or at which the audio information has been finally stored. Further information may be included in the header section such as an identification information about the user how has dictated the audio information and the like.
  • the protocol information is coded in a body section herein extending from line 6 to line 29.
  • a start time and a stop time are defined.
  • the start time is defined to be TIME1 and the stop time is defined to be TIME5.
  • this time information has been stored at starting and at stopping of the audio signal recording.
  • an address information to the recorded audio information is coded.
  • the address information may be coded in accordance to the employed data storage system such as a directory/file address information or more commonly a uniform resource locator (URL) or uniform resource indicator (URI).
  • URL uniform resource locator
  • URI uniform resource indicator
  • the audio information may be embedded directly into the subsection.
  • a next subsection of the body includes information relating to the keywords.
  • a moment of time relating to the recording of the keywords is coded as TIME2.
  • an addressing information to the keyword audio information is coded.
  • a pause has been initiate at the moment of time TIME3 and the recording has been resumed at the moment of time TIME4.
  • a further subsection of the body includes information about associated information, herein designated as embedded objects.
  • the moment of time TIME3 is coded in the subsection at which the additional information has been associated to the recorded audio information and address information about the information (object) is coded in the subsection in order to allow access of the user to this associated information.
  • location/position information and other information relating to additional sensors are coded such as sensor information obtained from and ambient light sensor, acceleration sensor(s), background audio signals and background audio signal analysis results and the like.
  • the detailed XML-coded description / protocol with reference to Fig. 4c is one possibility to perform a logging of context information during the audio signal recording. Another possibility of logging context information is described with respect to the personal logbook mentioned above.
  • the recorded audio information as well as user commands concerning the operation of the audio signal recording may be logged as user input information. Further information relating to the associating of information to the recorded dictation may be provided to the personal logbook as context information.
  • the labeling and/or indexing of the audio information results in the same interdependency structure such as described in view of the following audio information description / protocol referred to in Fig. 4c and with regard to the following graphical depiction referred to in Fig. 4d and Fig. 4e .
  • Speech / voice recognition systems/applications are improving and will gain significance in the nearer future since the capability to convert information provided by speech / voice input into text based information is time saving. Besides the pure text based information obtained from information provided by speech / voice input the original audio recording will be preserved to check the speech recognition (translation) at any time.
  • a speech / voice recognition may be employed for the recorded audio information where the recognition may be operated in the recording device or by a speech recognition service provided by any external entity accessible via a communication network, in particular via a mobile communication network in case of mobile recording terminal.
  • the resulting text based information resulting from a speech / voice recognition can be included directly into the description / protocol information of the audio recording or address information for retrieving the text based information may be included.
  • the speech / voice recognition may be employed to the dictation and/or to the keyword dictation.
  • a retrieval system allowing a user to retrieve certain audio information in accordance with request information may employ the audio keyword recording by matching request information inputted by audio recording with the previously recorded audio keywords being recorded during the dictation of the audio information.
  • the keywords may be available as text based keywords.
  • the retrieval may be also based on text comparison of text information included the retrieval request. Further information being included in the description / protocol of the recording may be employed for retrieval by using that information for labeling / indexing the audio recording as aforementioned.
  • the protocol / description information may be employed to create a graphical depiction of the logged information for providing a flexible mind-map supporting recognition and retrieval of information.
  • the following Fig. 4d and Fig. 4e both illustrate example mind-maps, where the mind-map depicted in Fig. 4d relates to a graphical user interface and the mind-map depicted in Fig. 4e relates to textual oriented user interface.
  • Fig. 4d illustrates graphically a depiction of a protocol / description information according to an embodiment of the invention.
  • the graphical depiction may comprise only an extraction of information available with the audio information.
  • the graphical depiction shows a first graphical representation of a recorded audio information object 402 being denoted as (audio) clip #002. Further the graphical depiction of the audio information object 402 includes information about the recording date / time and the place of recording.
  • a plurality of further graphical representations are arranged around the depicted audio information object 402. Lines linking the graphical representations shall represent interdependencies such as associations.
  • a set of graphical representations 403 relates to keywords, keyword object #4, keyword object #5 and keyword object #6.
  • the linking lines connecting the graphical representations of each of the keyword objects #4, #5 and #6 with the audio information object 402 indicates the belonging of those keyword objects #4 to #6 to this audio information object 402.
  • a set of graphical representations 404 is depicted and each of the graphical representations is linked by a connecting line to the audio information object 402.
  • the linking lines indicate that the image object #1, the image object #2 and the text object #1 is associated with the audio information object 402 and audio clip #002, respectively.
  • the graphical depiction shows a second graphical representation of a recorded audio information object 400 being denoted as (audio) clip #001 and the audio information object 400 has associated keyword objects #1, #2 and #3 shown as a set of graphical representations 401 each of which is connected via a line with the audio information object 400.
  • the graphical depiction may not only be used for supporting the mind but the graphical depiction may be operated as a graphical user interface.
  • the user may select one of the graphical representations in order to retrieve the respective indicated information. That is, in case the user selects a keyword object the mobile terminal reproduces the recorded digital audio sequence. Analogously, in case the user selects audio clip #002 the corresponding dictation is reproduced, in case the user selects one of the graphical representations of the image objects an image viewer application shows the corresponding image to the user and in case the user selects the graphical representation of the text object a text viewer application shows the corresponding text to the user.
  • Fig. 4e illustrates a textual oriented depiction of a protocol / description information according to an embodiment of the invention.
  • This textual oriented depiction is an alternative depiction showing the same properties and information to the user as described with reference to Fig. 4d .
  • the logged attributes of the recorded audio information objects audio clip #001 and audio clip #002 are presented in a table depiction.
  • the table depiction includes the elements clip designation, date and time of recording, place of recording, and address information of the associated information, associated text objects, associated keyword objects and associated image objects.
  • the tabular depiction allows a clear overview of the information / objects which have been associated by user instructions to the respective audio information.
  • a selecting of an address information entry in the depicted table result in reproducing / illustrating of the respective selected information in accordance with the type of the information (object).
  • the aforementioned methods relating to a reliable retrieval and to sensitive retrieval results require additional information containing labeling and/or indexing information. That is, the single information in reference to which the retrieval, labeling and/or indexing by accompanying meta-information is described is of limited usefulness. Therefore, it is advantageous to offer a method for embedding or appending at least a limited amount of meta-information directly into/to information and more particularly directly into audio information.
  • the inventive concept for appending and/or embedding information to/into an audio information sequence provides several advantages besides the mentioned possibility to complete the audio information sequence with labeling and/or indexing information.
  • the inventive concept further relates to a method for embedding information of any type into an audio sequence which ensures that the embedded information is prevented independently from converting and transcoding of the audio sequence. That is, the embedded information is preserved in the audio sequence although the audio sequence is converted from an analog encoded audio sequence to a digital encoded audio sequence and vice versa, from a digital encoded audio sequence to an analog encoded audio sequence. Further, the embedded information is preserved in the audio sequence although the audio sequence is transcoded from an audio sequence being encoded in a first digital audio format comprising any audio compression formats, into an audio sequence being encoded in a second digital audio format.
  • the capability to embed information according to an embodiment of the invention may be employed to embed authentication and/or watermarking information which may be also encrypted.
  • the inventive concept to addend and/or to embed information to/into audio information sequences is based on an octave-based coding method which allows coding of characters representing the appended or embedded information.
  • a well known character encoding is the ASCII 0 encoding assigning uniquely a sequence of 7 or 8 bits to one character/symbol resulting in code tables assigning uniquely each of the bit sequences with a length of either 7 bits or 8 bits to one characters/symbols, respectively.
  • the total number of characters/symbols which are required may differ such that related coding tables may be employed.
  • bit sequence of the length 6 bits will be employed to described to inventive method, where this bit sequence allows to code 64 different characters/symbols.
  • Such a code table comprising 64 different elements should be suitable to code HTML (hypertext markup language) tags, URL (uniform resource locator) sequences, URI (uniform resource indicator) sequences and the like, for the coding of which the numbers 0-9, characters A-Z (26 letters) and a few specific symbols 0/: ⁇ .#,@_ are sufficient.
  • Each character/symbol is coded by a unique combination of tones which is repeated over more than one order (octave). For example relating to the musical scale comprising the tones C,D,E,F,G,A and H such a coding could be expressed by
  • Fig. 5a illustrates a frequency range of an audio recording / reproducing system including several complete octaves.
  • An index n is shows in the frequency range illustrating for enumerating the depicted octaves ranging from 1 to 7 corresponding to the seven octaves subsequent in frequency.
  • corresponding frequencies in each other octave (1, 2, 3, 4, 5 and 7) are indicated which relate to the originally indicated frequencies b1, b2 and b3.
  • the code basis is orthogonal i.e. the code bases of the characters/symbols have no correlation.
  • the code bases are partly overlapping, i.e. are partly correlated, which takes account of the fact that audio signals should be able to withstand high noise.
  • Fig. 5c depicts a signal flow chart illustrating a processing of an audio signal s 0 (t) for embedding information into that according to an embodiment of the invention.
  • the audio signal s 0 (t) 530 is assumed to be the original audio signal having no embedded information.
  • the combining of both the audio signal s 0 (t) 530 and the code formulation ⁇ b j (x) ⁇ 540 is performed in the frequency domain of the signal and the code formulation.
  • the audio signal s 0 (t) 530 is transformed into the frequency domain resulting in an audio Fourier transformation A( ⁇ ) 535 of the audio signal s 0 (t) 530.
  • K is a pre-defined attenuation factor (K ⁇ 1).
  • the aforementioned given mathematical definition for expressing the modulation signal M 0 ( ⁇ ) 545 represents an ideal mathematical expression.
  • delta function as presented and employed are not available such that the employed delta function is to be replaced with a suitable approximation function such as a sinc-function, a Gaussian function etc as known to those skilled in the art.
  • the audio Fourier transformation A( ⁇ ) 535 as well as the modulation signal M 0 ( ⁇ ) 545, are to be combined in the frequency domain.
  • the Fourier transformation A( ⁇ ) 535 and the modulation signal M 0 ( ⁇ ) 545 may be added or may be multiplied in order to obtain a suitable modulation signal M( ⁇ ), respectively.
  • Fig. 5b shows a set of plot diagram depicting the audio Fourier transformation A( ⁇ ) and modulation signals M( ⁇ ) resulting from the combining of the Fourier transformation A( ⁇ ) 535 and the modulation signal M 0 ( ⁇ ) 545 according to embodiment of the invention.
  • the first plot diagram illustrates a curve 500 representing the original audio signal s 0 (t) in the frequency domain, i.e. the audio Fourier transformation A( ⁇ ) 535. Additionally, indications above and below the abscissa of the plot diagram being the frequency axis represent both the frequency sections (octaves) in which the tones of the code are repeated and the tones of the code, respectively.
  • the frequency sections (octaves) are depicted by indication below the abscissa whereas the tones are depicted by indication above the abscissa.
  • a second plot diagram illustrates a curve 510 representing modulation signal M( ⁇ ) obtained from the audio Fourier transformation A( ⁇ ) and the modulation signal M 0 ( ⁇ ) by adding in the frequency domain.
  • a third plot diagram illustrates a curve 520 representing modulation signal M( ⁇ ) obtained from the audio Fourier transformation A( ⁇ ) and the modulation signal M 0 ( ⁇ ) by multiplying in the frequency domain.
  • Both plot diagrams further include the original audio Fourier transformation A( ⁇ ) as dotted curves 515 and 525, respectively, in order to indicate the difference produced by adding and multiplying for modulating with the modulation signal M 0 ( ⁇ ).
  • both modulation operations are applicable for deriving a suitable modulation signal M( ⁇ ).
  • the obtaining of the modulation signal M( ⁇ ) may be used since as depicted deviations in the frequency curve caused by the modulation signal M 0 ( ⁇ ) may be smaller such that a resulting interference in the final audio signal due to these deviations is less obvious to hear by a user.
  • the attenuation factor K is pre-determined in such a way that code embedded in the audio signal is decodable later thereof but the interference is low.
  • the audio Fourier transformation A( ⁇ ) 535 obtained by Fourier transformation (e.g. performed by a Fourier transformer) of the original signal s 0 (t) 530 and the modulation signal M 0 ( ⁇ ) 545 are multiplied 550 (e.g. by a multiplying component, multiplier) which results in the modulated signal M( ⁇ ).
  • the multiplying of the audio Fourier transformation A( ⁇ ) and the modulation signal M 0 ( ⁇ ) 545 may be operated point-wise.
  • the resulting modulated signal M( ⁇ ) is transformed back into a modulated audio signal m(t) in the time domain e.g. by employing inverse Fourier transformation which can be operated by an inverse Fourier transformer.
  • inverse Fourier transformation which can be operated by an inverse Fourier transformer.
  • the modulated signal M( ⁇ ) in the frequency domain is a discrete signal fast Fourier transformation may be applied for transforming into the time domain.
  • the modulated audio signal m(t) is combined with the original audio signal s 0 (t). Since the above described processing operations applied to the original audio signal s 0 (t) for obtaining the audio signal m(t) the audio signal m(t) and the original audio signal so(t) may be spaced in time for a certain period - ⁇ t. A combining of the modulated audio signal m(t) and the original audio signal s 0 (t) without taking account of the time difference - ⁇ t may cause interference in the combined signals.
  • Fig. 5d refers to the extracting of an embedded character/symbol from an audio signal according to an embodiment of the invention.
  • Fig. 5d shows a block diagram depicting a recovering of information being embedded in an audio signal according to an embodiment of the invention.
  • the decoding of an unknown character/symbol and an unknown character/symbol sequence requires the input of an audio signal s(t) embedding the an unknown character/symbol and an unknown character/symbol, a correlation basis ⁇ B i ⁇ and the code bases ⁇ b j (x) ⁇ , ⁇ b j (y) ⁇ , ... of the total set of characters/symbols x, y, ... being possibly coded in the audio signal s(t).
  • the audio signal s(t) 600 is transformed into the frequency domain resulting in a Fourier signal Z( ⁇ ) 605, for example by Fourier transformation
  • the correlation basis ⁇ B i ⁇ 610 can be defined directly in the frequency domain and is represented by a set of all frequencies being used to code the characters/symbols.
  • the modulation C( ⁇ ) 615 defining all frequencies employed for embedding characters/symbols in the audio signal s(t) allows to delete all audio signal information from the audio signal s(t) which does not related in any way to coded characters/symbols.
  • the Fourier signal Z( ⁇ ) 605 is convoluted with the modulation C( ⁇ ) 615 resulting in a convolution signal Y( ⁇ ) representing frequency magnitude signals of the audio Fourier signal Z( ⁇ ) 605 at the frequencies being comprised in the modulation C( ⁇ ) 615 and correlation basis ⁇ B i ⁇ 610, respectively.
  • the convolution may be operated by a convoluting component.
  • the frequency magnitude signals of the audio Fourier signal Z( ⁇ ) 605 at frequencies being not comprised in the modulation C( ⁇ ) 615 and correlation basis ⁇ B i ⁇ 610, respectively, are at least approximately zero.
  • the overall convolution signal Y( ⁇ ) is evaluated at the frequencies which are associated with a certain code basis. That means, the convolution signal Y( ⁇ ) is evaluated at the frequencies defined by the code basis ⁇ b j (x) ⁇ 625, at the frequencies defined by the code basis ⁇ b j (y) ⁇ 630 and so on.
  • the evaluation results in magnitude values M(x,n) 635, M(y,n) 640, ... each representing an evaluation value corresponding to the magnitudes of the frequencies which are assigned to the code basis ⁇ b j (x) ⁇ , to the code basis (b j (y)), ...
  • the magnitude values M(x,n), M(y,n), ... may be obtained by adding the frequency magnitudes at frequencies defined by the each code basis.
  • the convolution signal Y( ⁇ ) may be performed by one summarizing component or summarizer receiving the convolution signal Y( ⁇ ) and subsequently the frequencies of the code bases b j (x) ⁇ , b j (y) ⁇ , ... to provide correspondingly the magnitude values M(x,n), M(y,n), ... in sequence or may be performed by a plurality of summarizing components or summarizers each the convolution signal Y( ⁇ ) and one of the sets of frequencies corresponding to one of the code bases b j (x) ⁇ , b j (y) ⁇ , ... to provide correspondingly the magnitude values M(x,n), M(y,n), ... in parallel.
  • a final evaluation of the magnitude values M(x,n), M(y,n), ... is obtained by comparing each f with another.
  • the comparisons of each of the magnitude values M(x,n), M(y,n), ... with each other may be defined as by a quotient of magnitude values of each character/symbol pair such as (x,y).
  • each ratio R(x,y,n) 645 may be calculated by a respective divider or dividing component or alternatively, each ratio R(x,y,n) 645 may be calculated and provided by one divider or dividing component subsequently.
  • the ratio R( ⁇ ,x,n) corresponding to the unknown character/symbol ⁇ is significantly greater as the ratios R(x,y,n), ... which do not correspond to the unknown character/symbol ⁇ .
  • the unknown character/symbol ⁇ . is extracted and determined from the audio signal s(t).
  • the identification of the unknown character/symbol ⁇ by evaluation the ratios R(x,y,n), ... may be uncertain. In such a case a probability evaluation of the ratios R(x,y,n), ..., may result in a more reliable and certain result.
  • the evaluation of a kind of maximum probability can be employed therefor.
  • the frequencies of the code bases may be coded as frequencies having a certain length, in particular of constant time e.g. approximately 0,1 seconds, wherein the coding of frequencies belonging to different code bases are separated by a certain duration of time, in particular of a constant time such as approximately 0,05 seconds.
  • the coding of constant-time frequency signals and constant-time separation makes it particularly easy to decode the signals.
  • the coded information may be completed by error correction information, check ode information, redundancy information such as known from signal processing technology to enhance the robustness of the coded information due to interference.
  • a mean value ⁇ R> 650 of all the ratios R(x,y,n) is derived.
  • the mean value ⁇ R> allows to normalize the ratios R(x,y,n), ... resulting in a corresponding set of normalized ratios F(x,y,n), ...
  • the normalization of the ratios R(x,y,n), ... takes into account that the frequency magnitudes evaluated incorporates magnitude contributions of the original audio signal s 0 (t).
  • the normalization maps (centers) the ratios R(x,y,n), ... around 1 which makes the evaluation of the obtained ratios R(x,y,n) or normalized ratios F(x,y,n), ... much more easier.
  • the mean value ⁇ R> 650 may be obtained by a component for calculating a mean value from a plurality of individual values.
  • the arithmetic mean, the geometric mean, the square mean are known mean values. Further different means, especially weighted or weighting means may be employed for normalizing.
  • Example prepending/appending tags and/or meta-information to an audio sequence
  • the method for embedding information of any type into an audio sequence can be also employed to append information at the beginning of the audio sequence, in the following designated as prepending, and to append information at the end of the audio sequence, in the following designated as appending.
  • an audio sequence coding silence is predended and/or appended to the original audio sequence.
  • the respective audio sequence coding silence is used to embed information as described in combination with the aforementioned method for embedding information into an audio sequence which forms audio signals predended and/or appended to the audio sequence.
  • a following constructed audio sequence exhibits prepended and/appended information arranged to an original audio sequence.
  • the constructed audio sequence may be established as following:
  • the preprending information is coded by an initial (BEGIN_TAG) tone, which is a signal with a certain frequency or with certain multiple frequencies, one or more TAG tones for coding information, and a final (END_TAG) tone, which is again a signal with a certain frequency or with certain multiple frequencies.
  • the initial one or more TAG tones shall enable to identify the audio sequence uniquely, therefore, the initial one or more TAG tones include for example time information about the recording of the audio sequence (for example UTC-encoded), identification information about the terminal/device which has been used for recording (for example an IMEI code of a cellular terminal) and further identification information. But the combination of the above presented time and identification information provided for unique identification since one only one audio sequence (voice message/mail) can be recorded by one given terminal/device at one time.
  • the one or more TAG tones may be used to coded numbers for 0 to 9, which have a simple implementation as musical tones.
  • the TAG for such a clip for example created on January 15, 2002, at 9:45:17 UTC on terminal 357149/19/415292/7 can be coded as the following number sequence: 2 ⁇ 0 ⁇ 0 ⁇ 2 ⁇ 0 ⁇ 1 ⁇ 1 ⁇ 1 ⁇ 5 ⁇ 0 ⁇ 9 ⁇ 4 ⁇ 5 ⁇ 1 ⁇ 7 ⁇ 3 ⁇ 5 ⁇ 7 ⁇ 1 ⁇ 4 ⁇ 9 ⁇ 1 ⁇ 9 ⁇ 4 ⁇ 1 ⁇ 5 ⁇ 2 ⁇ 9 ⁇ 2 ⁇ 7.
  • the appending information is again coded by an initial (BEGIN_TAG) tone, which is a signal with a certain frequency or with certain multiple frequencies, one or more TAG tones for coding information, and a final (END_TAG) tone, which is again a signal with a certain frequency or with certain multiple frequencies.
  • initial (BEGIN_TAG) tone which is a signal with a certain frequency or with certain multiple frequencies
  • END_TAG final
  • the additional appending information is coded by an initial (START_METADATA) tone, which is a signal with a certain frequency or with certain multiple frequencies, one or more METADATA tones for coding meta-information, and a final (END_METADATA) tone, which is again a signal with a certain frequency or with certain multiple frequencies.
  • the meta-information can be used to code information being associated with the original audio sequence, such as address information (link information, URL information and the like), but also content information which may be coded itself for example by using a markup language such as hypertext markup language (HTML), wireless markup language (WML), extended markup language (XML) and the like.
  • HTML hypertext markup language
  • WML wireless markup language
  • XML extended markup language
  • the tone system mentioned above can be used to create a basis for e.g. the ASCII table or a subset of it, plus a few additional tones shown above.
  • the METADATA tones are implemented directly as code bases corresponding to ASCII character/symbol table.
  • a code bases allowing to create a reliable 7-or 8-bit coding scheme in accordance with the aforementioned method is possible.
  • An alternative coding scheme to coding the ASCII character/symbol table refers to the use of the three-digit representation for the ASCII character/symbols in question. For example the character "A" is represented by the number sequence 065.
  • ASCII representation for the html tag " ⁇ HEAD>" for example would be 060-072-069-065-068-062, giving the tones 0 ⁇ 6 ⁇ 0 ⁇ 0 ⁇ 7 ⁇ 2 ⁇ 0 ⁇ 6 ⁇ 9 ⁇ 0 ⁇ 6 ⁇ 5 ⁇ 0 ⁇ 6 ⁇ 8 ⁇ 0 ⁇ 6 ⁇ 2.
  • This coding creates many redundant 0's, and some other coding scheme (for example omitting the lowercase letters from the ASCII table) would enable just two digits to be used. However, this requires standardization.
  • the advantage of the pure ASCII table representation is that it is global and well-standardized.
  • Corresponding code bases according to an embodiment of the invention to encode the number 0 to 9 required for embedding a three-digit encoded ACSII character/symbol table into an audio sequence may have the following form each comprising two different music tones being based on the known description for representing music tones C, D, E, F, G, A and H: "0”: C D “1”: C F “2”: D A “3”: D F “4": E F "5": E H “6”: F H "7”: C G “8” G A “9”: E H
  • the presented code bases represent advantageous code bases since all pairs of code bases have at most one tone in common, and the two tones for each code are mostly separated by at least two frequencies which decreases inter-frequency interference. With this choice, it is possible to make the codes nearly orthogonal (having a small correlation) and also separated enough from each other that small glitches in the original audio signal (playback/recording) are not strongly affected.
  • the advantageous of the code bases each comprising two music tones can be recognized by assuming that the number which is coded is the number "7" comprising the music tones "C” and "G". Further assuming this number is to be decoded from an audio sequence.
  • the magnitude (amplitude) of the signal equal to 7,4 in case that the symbol '7' is embedded
  • the magnitude (amplitude) value of the signal is equal to 7,0 - 7,2 in case one of the other symbols ('0' to '6', '8', '9') is embedded.
  • the system must be capable of distinguishing magnitude (amplitude) values from roughly 5% signal differences.
  • meta-information may be embedded directly into such an audio information.
  • the advantage of a direct embedding is that the meta-information is always present and retrievable without using any retrieval functionality as described in context of the personal logbook.
  • the meta-information can be used to code information being associated with the original audio sequence, such as address information (link information, URL information and the like), but also content information which may be coded itself for example by using a markup language such as hypertext markup language (HTML), wireless markup language (WML), extended markup language (XML) and the like.
  • HTML hypertext markup language
  • WML wireless markup language
  • XML extended markup language
  • the language encoding system and alphabet is given above allows to code any clear text information independent from its meaning and purpose.
  • the alphabet described there can be used to embed the letters of any mark-up language around or within the audio clip itself; in particular, the markup codes could be embedded in the METADATA field described therein. These letters can then later be retrieved from the audio clip by the aforementioned method and converted to a machine-readable format of the markup language.
  • Watermarking is a technique being used for example to code information about for example origin of electronic data and the producer of the electronic data in a manner which ensures that the information is present and decodable from the electronic data even in case the electronic data have been manipulated. Basically, watermarking was introduced to save copyright information of digitally provided images. Such a technique is also of interest for being provided for digital audio information.
  • the simplest watermark is just the aforementioned TAG information of the audio sequence.
  • a constant-time tone with a fairly long time e.g. several tenths of a second or more, may be employed, since it is then easier to identify the code statistically. More complex redundancy and error correction schemes may also be added if so desired.
  • the TAG information and meta-information directly into the audio sequence by embedding the information according to an embodiment of the invention.
  • the embedded information replaces an appending/prepending of the TAG information which may interfere the audio reproduction in systems which do not recognize the prepending/appending information as information being coded.
  • the aforementioned method for embedding information of any type into an audio sequence can be used to embed a hidden authentication watermark into the audio sequence.
  • the authentication watermark may be relevant in e.g. some legal situations, and allows to verify whether modifications of the audio sequences have been made.
  • the authentication watermark relies on known digital signature algorithms.
  • the aforementioned prepending information i.e. the prepending TAG
  • the prepending information is encrypted resulting in an encrypted ENCTAG, for example by employing a strong asymmetric encryption procedure, such as a public key encryption employing a public and a private (secret) key.
  • the encrypted ENCTAG is embedded into the original audio sequence.
  • the authentication of the embedded encrypted ENCTAG can be verified by the receiver of the corresponding audio sequence, but only the sender (producer) is actual able to decrypt the embedded encrypted ENCTAG.
  • the aforementioned method for embedding an authentication watermark can be improved by implementing obtaining a kind of checksum from the original audio signal to create a signature which can not be manipulated or exchanged resulting in a more reliable authentication watermark.
  • the audio Fourier transform A( ⁇ ) is partitioned into several discrete segments of for example a constant length such as a length of 0.1 seconds and the frequencies are discretized into a certain number of frequency sections such as 8 frequency sections.
  • Discretized amplitude values A'( ⁇ ) are obtained from the audio Fourier transform A( ⁇ ) and an amplitude mean value ⁇ A>, i.e. an average amplitude, of the audio Fourier transform A( ⁇ ).
  • the corresponding discretized amplitude value A'( ⁇ ) is set equal to zero, whereas in case the actual amplitude value of the audio Fourier transform A( ⁇ ) is greater than the amplitude mean value ⁇ A>, the corresponding discretized amplitude value A'( ⁇ ) is set equal to one.
  • the resulting discretized amplitude values A'( ⁇ ) comprises a sequence consisting of "0" and "1" which can be interpreted as a bit sequence. Moreover, this sequence may be partitioned into parts of 8 discretized amplitude values which are interpreted to form a sequence of 8-bit values of the total time of the audio sequence.
  • the sequence of 8-bit values can be is treated as aforementioned for embedding information into an audio sequence, in particular an ASCII coding employing 8 bits.
  • the sequence of 8-bit values may be encrypted using any known encryption method, such as an asymmetric encryption method like the public key encryption.
  • the encrypted sequence is finally embedded into the audio sequence and allows a receiver of the audio sequence to separate the encrypted sequence from the audio sequence for verifying the authentication of the audio sequence.
  • the embedding of an encrypted information may be hidden.
  • the hiding of information embedded in an audio sequence according to an embodiment of the invention can be achieved by employing code bases unique to the user and varying in time. Additionally, normal code bases may be employed to embed further information in a transparent (this information is apparent to a receiver reproducing the audio sequence) way to the audio sequence.
  • An improved implementation of an embodiment according to the invention allows to establish a file allocation table analogously to the file allocation table technique known from file handling and managing techniques in processing devices.
  • a FAT comprises information about an identification sequence, an associated name, a length, a location on the memory medium or the storage device (such as a start time in serial media) and the like.
  • map ASCII coding into a frequency based audio coding allows to code that information correspondingly.
  • the retrieving of audio FAT information embedded in an audio sequence requires a complete processing of the audio sequence to extract that information therefrom.
  • the extracting process may be time-intensive such that a separate storing (mirroring) of the FAT information independently from the audio sequence is advantageous to preserve processing capabilities and processing time.
  • a finalizing operation may ensure that the FAT information being (mirrored) provided separately and the FAT information being embedding in the audio sequence are identical.
  • the finalizing operation may be operated as a synchronization operation, i.e. at a certain time interval or at certain event at which the synchronizing of the FAT information is useful to prevent any unsynchronized situations.
  • the audio FAT information may be stored at a fixed position in the audio sequence for example pre-defined by a pointer referring to a certain moment of time in the audio sequence. Moreover, the duration of time required for storing the audio FAT information may be allocated permanently and unambiguously for the audio FAT information.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Nitrogen And Oxygen Or Sulfur-Condensed Heterocyclic Ring Systems (AREA)

Claims (34)

  1. Verfahren zum Organisieren von einem Benutzer bereitgestellter Information, die in einer mobilen Endgerätevorrichtung verfügbar ist, mit Meta-Information, um das Abrufen der vom Benutzer bereitgestellten Information zu gestatten;
    wobei das Verfahren umfasst:
    - Erhalten der vom Benutzer bereitgestellten Information, wobei die vom Benutzer bereitgestellte Information infolge einer Bedienung der mobilen Endgerätevorrichtung durch den Benutzer erhalten wird;
    - Erhalten von Kontextinformation, die der vom Benutzer bereitgestellten Information zugeordnet ist;
    - Zuweisen von Meta-Information, die aus der Kontextinformation erhalten wird, zu der vom Benutzer bereitgestellten Information; und
    - Speichern der vom Benutzer bereitgestellten Information und der Meta-Information in einem Verlaufsspeicher, um eine Informationsverlaufs-Funktionalität zu erstellen;
    wobei die Meta-Information zum Abrufen der vom Benutzer bereitgestellten Information durch Abgleichen von Anfrageinformation eingesetzt wird, die mit einer Abrufanfrage mit der Meta-Information bereitgestellt wird, um eine vom Benutzer bereitgestellte Information auszuwählen, die der passenden Meta-Information zugewiesen ist.
  2. Verfahren gemäß Anspruch 1, wobei das Zuweisen umfasst:
    - Entnehmen von Meta-Information aus der Kontextinformation; und
    - Zuweisen der Meta-Information zu der vom Benutzer bereitgestellten Information;
  3. Verfahren gemäß Anspruch 1, wobei das Erhalten der vom Benutzer bereitgestellten Information umfasst:
    - Empfangen von Benutzer-Eingabeinformation, die durch Bedienung eines der Eingabemittel durch den Benutzer erzeugt wird; und/oder
    - Empfangen von Vorgangsinformation und/oder Kommunikationsinformation, die über eine Kommunikationsschnittstelle empfangen wird,
    wobei die Benutzer-Eingabeinformation und/oder Vorgangsinformation und/oder Kommunikationsinformation die vom Benutzer bereitgestellte Information darstellen.
  4. Verfahren gemäß Anspruch 1, wobei das Zuweisen der Meta-Information zu der vom Benutzer bereitgestellten Information umfasst:
    - Entnehmen von Kennzeichnungsinformation und/oder Indexinformation aus der Kontextinformation;
    - Zuweisen der Kennzeichnungsinformation und/oder Indexinformation der vom Benutzer bereitgestellten Information; und
    - Speichern der Kennzeichnungsinformation und/oder Indexinformation und der vom Benutzer bereitgestellten Information;
    wobei die Kennzeichnungsinformation und/oder Indexinformation als die Meta-Information eingesetzt wird, um die Abruf-Funktionalität des Eingabeverlaufs durch den Benutzer zu erstellen.
  5. Verfahren gemäß Anspruch 1, wobei das Zuordnen umfasst:
    - Erzeugen von Referenzinformation für mindestens einen Teil der vom Benutzer bereitgestellten Information;
    - Speichern der Referenzinformation in einem ersten Speicherbereich; und
    - Speichern des Teils der vom Benutzer bereitgestellten Information in einem zweiten Speicherbereich;
    wobei die Referenzinformation Adressinformation umfasst, die es gestattet, den Teil vom Benutzer bereitgestellter Information aus dem zweiten Speicherbereich abzurufen.
  6. Verfahren gemäß Anspruch 1,
    wobei das Erhalten von Kontextinformation umfasst:
    - Erhalten von Kalenderinformation aus einem elektronischen Kalender, der in der mobilen Endgerätevorrichtung implementiert ist, wobei die Kalenderinformation Kontextinformation darstellt;
    - Abgleichen der Kalenderinformation und der vom Benutzer bereitgestellten Information; und
    - Erhalten von Meta-Information aus der Kalenderinformation.
  7. Verfahren gemäß Anspruch 6, wobei das Erhalten von Meta-Information aus der Kalenderinformation umfasst:
    - Erhalten einer ersten Zeitinformation gemäß der vom Benutzer bereitgestellten Information;
    - Erhalten einer Vielzahl von Kalendereinträgen, die in der Kalenderinformation eingeschlossen sind, wobei jeder Kalendereintrag eine zweite Zeitinformation umfasst;
    - Abgleichen der ersten Zeitinformation und jeder der zweiten Zeitinformationen für:
    - Erhalten von Meta-Information aus jedem passenden Kalendereintrag der Vielzahl von Kalendereinträgen; und
    - Speichern der vom Benutzer bereitgestellten Information und der Meta-Information in dem Verlaufsspeicher, um die Informationsverlauf-Funktionalität zu erstellen.
  8. Verfahren gemäß Anspruch 7, wobei das Erhalten von Meta-Information aus jedem passenden Kalendereintrag umfasst:
    - Zuweisen einer Zugehörigkeitsfunktion zu der zweiten Zeitinformation;
    - Herleiten eines Wertes des Zugehörigkeitsgrades aus der Zugehörigkeitsfunktion gemäß der ersten Zeitinformation; und
    - Zuweisen des Wertes des Zugehörigkeitsgrades zu der vom Benutzer bereitgestellten Information;
    wobei der Wert des Zugehörigkeitsgrades ein Maß definiert, das es gestattet, eine Zuverlässigkeit für einen Abruf abzuschätzen.
  9. Verfahren gemäß Anspruch 7, wobei im Fall, dass die erste Zeitinformation eine erste Zeitspanne betrifft:
    - Aufteilen der vom Benutzer bereitgestellten Information in eine Vielzahl von Informationsteilen, wobei jeder Informationsteil eine maximale vordefinierte Zeitspanne betrifft, wobei jeder Informationsteil separat abgeglichen wird.
  10. Verfahren gemäß Anspruch 7, wobei im Fall, dass die erste Zeitinformation eine erste Zeitspanne betrifft und die zweite Zeitinformation eine zweite Zeitspanne betrifft; und im Fall, dass die erste Zeitspanne die zweite Zeitspanne überschreitet:
    - Unterteilen der vom Benutzer bereitgestellten Information in mindestens zwei Informationsabschnitte, wobei einer der mindestens zwei Informationsabschnitte der zweiten Zeitspanne entspricht, wobei der eine entsprechende Informationsabschnitt abgeglichen wird.
  11. Verfahren gemäß Anspruch 7, wobei die zweite Zeitspanne bestimmte vordefinierte Overhead-Zeitspannen einschließt, die zu dem Anfang der zweiten Zeitspanne und zu dem Ende der zweiten Zeitspanne angehängt werden.
  12. Verfahren gemäß Anspruch 7, wobei jeder der Vielzahl von Kalendereinträgen, die in der Kalenderinformation eingeschlossen sind, semantisch strukturiert wird und das Erhalten von Meta-Information aus jedem passenden Kalendereintrag umfasst:
    - Erhalten von Meta-Information aus jedem der passenden semantisch strukturierten Kalendereinträge.
  13. Verfahren gemäß Anspruch 1,
    wobei das Erhalten der vom Benutzer bereitgestellten Information und Zuweisen der Meta-Information einschließt:
    - Erhalten vom Benutzer bereitgestellter Audioinformation zusätzlich zu der vom Benutzer bereitgestellten Information, wobei die vom Benutzer bereitgestellte Audioinformation aufgenommen und gespeichert wird;
    - Erhalten der Kontextinformation parallel zu der vom Benutzer bereitgestellten Audioinformation, wobei die Kontextinformation Benutzer-Eingabeinformation umfasst, die infolge einer Handlung an der mobilen Endgerätevorrichtung durch den Benutzer in Verbindung mit dem Aufnehmen und Speichern der vom Benutzer bereitgestellten Audioinformation erzeugt wird; und
    - Erzeugen von Meta-Information, die Information über die vom Benutzer bereitgestellte Information und die Kontextinformation umfasst.
  14. Verfahren gemäß Anspruch 13, wobei die Benutzer-Eingabeinformation Kontrollsignale umfasst, die das Aufnehmen der vom Benutzer bereitgestellten Audioinformation betreffen:
    - Nach dem Empfangen eines Startsignals: Einleiten des Aufnehmens und Speicherns der vom Benutzer bereitgestellten Audioinformation;
    - Nach dem Empfangen eines Schlüsselwort-Signals: Einleiten eines Aufnehmens einer vom Benutzer bereitgestellten Audio-Schlüsselwort-Information, die Schlüsselwörter umfasst, die die vom Benutzer bereitgestellten Audioinformation betreffen;
    - Nach dem Empfangen eines Anhängesignals: Zuordnen zusätzlicher Information zu der vom Benutzer bereitgestellten Audioinformation;
    - Nach dem Empfangen eines Pause-Signals: Pausieren des Aufnehmens und Speicherns der vom Benutzer bereitgestellten Audioinformation;
    - Nach dem Empfangen eines Fortsetzungssignals: Weiterführen des Aufnehmens und Speicherns der vom Benutzer bereitgestellten Audioinformation nach dem Pausieren des Aufnehmens und Speicherns; und
    - Nach dem Empfangen eines Stoppsignals: Stoppen des Aufnehmens und Speicherns der vom Benutzer bereitgestellten Audioinformation und Erzeugen einer Kontextinformation des Aufnehmens.
  15. Verfahren gemäß Anspruch 14, wobei das Aufnehmen von Kontextinformation mindestens einschließt:
    - Information und Zeitinformation über die vom Benutzer bereitgestellte Audioinformation,
    - Zeitinformation über die Benutzer-Eingabeinformation; und
    - Information über die zusätzliche vom Benutzer bereitgestellte Information, die der vom Benutzer bereitgestellten Audioinformation zugeordnet wird.
  16. Verfahren gemäß Anspruch 14, wobei das Erzeugen des Aufnehmens von Kontextinformation umfasst:
    - Codieren der Meta-Information wie zum Beispiel ein Dokument, der gemäß einer Auszeichnungssprache codiert wird.
  17. Verfahren gemäß Anspruch 13, wobei die Meta-Information durch Zuweisen grafischer Elementen zu jedem in der Meta-Information eingeschlossenen Informationseintrag ausgelegt wird und das Berechnen der grafischen Elemente, die den Inhalt der Meta-Information veranschaulichen und die Zuordnungen zeigen, die in der Meta-Information definiert wird.
  18. Verfahren gemäß Anspruch 13,
    wobei das Speichern der vom Benutzer bereitgestellten Information und der Meta-Information ein Einbinden der Meta-Information in die vom Benutzer bereitgestellte Audioinformation umfasst, wobei das Einbinden einschließt:
    - Bereitstellen eines Satzes von Codegrundlagen, die eine Vielzahl der Codierungssymbole darstellen, wobei jede Codegrundlage des Satzes von Codegrundlagen eine vordefinierte Anzahl der vordefinierten Frequenzen umfasst;
    - Abbilden der Meta-Information auf die eine Vielzahl der Codierungssymbole; und
    - Verknüpfen der vom Benutzer bereitgestellten Information und der abgebildeten Meta-Information durch Einbinden der abgebildeten Meta-Information in die vom Benutzer bereitgestellte Audioinformation.
  19. Verfahren gemäß Anspruch 18, wobei das Verknüpfen umfasst:
    - Erhalten eines Modulationssignals aus der abgebildeten Meta-Information;
    - Erhalten eines modulierten Signals durch das Verknüpfen der vom Benutzer bereitgestellten Audioinformation und des Modulationssignals in einem Frequenzbereich; und
    - Erhalten der vom Benutzer bereitgestellten Audioinformation, die die Meta-Information eingebunden hat, durch Verknüpfen des modulierten Signals mit der vom Benutzer bereitgestellten Audioinformation in einem Zeitbereich.
  20. Verfahren gemäß Anspruch 18, wobei das Bereitstellen des Satzes von Codegrundlagen umfasst:
    - Bereitstellen eines Satzes von Codegrundlagen in einem ersten Frequenzintervall, wobei das erste Frequenzintervall ein Frequenzintervall einer Vielzahl der Frequenzintervalle ist;
    - Abbilden des Satzes von Codegrundlagen auf jedes Frequenzintervall einer Vielzahl der Frequenzintervalle, wobei die Vielzahl der Frequenzintervalle ein Gesamtfrequenzintervall formt, das auf die vom Benutzer bereitgestellte Audioinformation anwendbar ist.
  21. Verfahren gemäß Anspruch 18, wobei das Erhalten des modulierten Signals umfasst:
    - Erhalten des modulierten Signals durch Multiplizieren der vom Benutzer bereitgestellten Audioinformation und des Modulationssignals in dem Frequenzbereich.
  22. Verfahren gemäß Anspruch 20, wobei das Erhalten der vom Benutzer bereitgestellten Audioinformation, die die Meta-Information eingebunden hat, umfasst:
    - Erhalten der vom Benutzer bereitgestellten Audioinformation, die die Meta-Information durch Addieren des Modulationssignals und des ursprünglichen Audiosignals in dem Zeitbereich eingebunden hat.
  23. Verfahren gemäß Anspruch 18,
    wobei das Verfahren ferner ein Entnehmen der Meta-Information umfasst, die in der vom Benutzer bereitgestellten Audioinformation eingebunden ist, wobei das Entnehmen einschließt:
    - Bereitstellen einer Korrelationsgrundlage, die jede Frequenz umfasst, die in einem Satz von Codegrundlagen eingeschlossen ist, die eine Vielzahl der Codierungszeichen darstellen, wobei die Codierungszeichen zum Codieren der Meta-Information eingesetzt werden; und
    - Anwenden der Korrelationsgrundlage auf die Audioinformation, die die Meta-Information eingebunden hat, um die Meta-Information zu entnehmen;
    wobei die entnommene Meta-Information zum Abrufen verfügbar ist.
  24. Verfahren gemäß Anspruch 23, wobei das Anwenden umfasst:
    - Erhalten eines Modulationssignals aus der Korrelationsgrundlage;
    - Erhalten eines Korrelationssignals aus dem Audiosignal durch Faltung des Modulationssignals und der Audioinformation, die die Meta-Information in einem Frequenzbereich eingebunden hat;
    - Entnehmen von Größenordnung der Signalwerte aus dem Korrelationssignal,
    wobei die Größenordnung der Signalwerte jede Codegrundlage des Satzes von Codegrundlagen betrifft; und
    - Auswerten der Größenordnung der Signalwerte, um die Meta-Information aus der Audioinformation abzurufen.
  25. Verfahren gemäß Anspruch 24, wobei das Auswerten der Größenordnung der Signale umfasst:
    - Erhalten der Größenordnung der Verhältniswerte jedes Paares der Größenordnung der Signalwerte; und
    - Normieren der Größenordnung der Verhältniswerte; und
    - Auswerten der normierten Größenordnung der Verhältniswerte der Größenordnung der Signalwerte, um die Meta-Information aus der Audioinformation zu entnehmen.
  26. Verfahren gemäß Anspruch 24, wobei die Korrelationsgrundlage in einem ersten Frequenzintervall definiert wird, wobei das Erhalten des Korrelationssignals umfasst:
    - Bereitstellen einer Korrelationsgrundlage, die alle Frequenzen des Satzes von Codegrundlagen in einem ersten Frequenzintervall einer Vielzahl der Frequenzintervalle umfasst;
    - Abbilden der Korrelationsgrundlage auf jedes Frequenzintervall einer Vielzahl der Frequenzintervalle, wobei die Vielzahl der Frequenzintervalle ein Gesamtfrequenzintervall formt; das auf die Audioinformation anwendbar ist, um das Modulationssignal zu erhalten; und
    - Erhalten des Korrelationssignals aus dem Audiosignal durch Faltung des Modulationssignals und der Audioinformation in dem Frequenzbereich.
  27. Verfahren zum Abrufen vom Benutzer bereitgestellter Information, wobei die vom Benutzer bereitgestellte Information auf der Basis von Meta-Information gemäß dem Verfahren eines der Ansprüche 1 bis 26 organisiert ist, wobei das Verfahren umfasst:
    - Abrufen einer Anfrage zum Abrufen, wobei die Anfrage Anfrageinformation zum Anweisen umfasst, bestimmte vom Benutzer bereitgestellte Information abzurufen;
    - Vergleichen der Anfrageinformation mit der Meta-Information, die der vom Benutzer bereitgestellten Information zugewiesen wird, die durch die Informationsverlaufs-Funktionalität bereitgestellt wird;
    - Abrufen der vom Benutzer bereitgestellten Information, die der Meta-Information zugewiesen wurde, die passt;
    - Erzeugen einer Rückmeldung, die die abgerufene vom Benutzer bereitgestellte Information umfasst; und
    - Senden der Rückmeldung.
  28. Verfahren gemäß Anspruch 27, wobei das Abrufen der vom Benutzer bereitgestellten Information umfasst:
    - Abrufen von Referenzinformation, die der vom Benutzer bereitgestellten Information zugeordnet ist, die abzurufen ist, wobei die Referenzinformation Adresseninformation umfasst, die den Teil vom Benutzer bereitgestellter Information adressiert, die in einem zweiten Speicherbereich gespeichert wurde; und
    - Abrufen des Teils der vom Benutzer bereitgestellten Information aus dem zweiten Speicherbereich.
  29. Verfahren gemäß Anspruch 27, wobei das Abrufen der vom Benutzer bereitgestellten Information umfasst:
    - Auswerten der vom Benutzer bereitgestellten Information, die auf der Basis der Werte des Zugehörigkeitsgrades abgerufen wird, die aus den Kalendereinträgen erhalten werden;
    wobei der Wert des Zugehörigkeitsgrades ein Maß definiert, das es gestattet, eine Zuverlässigkeit für einen Abruf abzuschätzen.
  30. Computerprogrammprodukt zum Organisieren vom Benutzer bereitgestellter Information mit einer Meta-Information, einschließlich der Ausführungsbefehle zum Durchrühren der Bedienungen eines der Ansprüche 1 bis 26, wenn die Ausführungsbefehle auf einer Verarbeitungsvorrichtung ausgeführt werden, die einer mobilen Endgerätevorrichtung zugeordnet ist.
  31. Mobile Endgerätevorrichtung, die fähig ist, vom Benutzer bereitgestellte Information mit Meta-Information zu organisieren, umfassend:
    - eine Komponente zum Erhalten der vom Benutzer bereitgestellten Information und der Kontextinformation, die der vom Benutzer bereitgestellten Information zugeordnet ist, wobei die vom Benutzer bereitgestellte Information infolge einer Bedienung an der mobilen Endgerätevorrichtung durch den Benutzer erhalten wird;
    - eine Komponente zum Zuweisen von Meta-Information, die aus der Kontextinformation erhalten wird, einschließend:
    - eine Speicherkomponente für ein Speichern der vom Benutzer bereitgestellten Information und der Meta-Information;
    wobei die Meta-Information zum Abrufen der vom Benutzer bereitgestellten Information durch Abgleichen von Anfrageinformation eingesetzt wird, die mit einer Abrufanfrage mit der Meta-Information bereitgestellt wird, um eine vom Benutzer bereitgestellte Information auszuwählen, die der passenden Meta-Information zugewiesen ist.
  32. Mobile Endgerätevorrichtung gemäß Anspruch 31, ferner umfassend:
    - eine Komponente zum Erhalten einer Kalenderinformation aus einem elektronischen Kalender, der in der mobilen Endgerätevorrichtung implementiert ist, wobei die Kalenderinformation Kontextinformation darstellt;
    - eine Komponente zum Abgleichen der Kalenderinformation und der vom Benutzer bereitgestellten Information; und
    - eine Komponente zum Erhalten einer Meta-Information aus der passenden Kalenderinformation.
  33. Mobile Endgerätevorrichtung gemäß Anspruch 31, ferner umfassend:
    - eine Komponente zum Erhalten der vom Benutzer bereitgestellten Audioinformation zusätzlich zu der vom Benutzer bereitgestellten Information;
    - eine Komponente zum Aufnehmen der vom Benutzer bereitgestellten Audioinformation;
    - eine Speicherkomponente zum Speichern der vom Benutzer bereitgestellten Audioinformation;
    - eine Komponente zum Erhalten der Kontextinformation parallel zu der vom Benutzer bereitgestellten Audioinformation, wobei die Kontextinformation Benutzer-Eingabeinformation umfasst, die infolge einer Handlung an der mobilen Endgerätevorrichtung durch den Benutzer in Verbindung mit dem Aufnehmen und Speichern der vom Benutzer bereitgestellten Audioinformation erzeugt wird; und
    - eine Komponente zum Erzeugen von Meta-Information, die Information über die vom Benutzer bereitgestellte Information und die Kontextinformation umfasst.
  34. Mobile Endgerätevorrichtung gemäß Anspruch 33, ferner umfassend:
    - einen Satz von Codegrundlagen, die eine Vielzahl der Codierungssymbole darstellen, wobei jede Codegrundlage des Satzes von Codegrundlagen eine vordefinierte Anzahl von vordefinierten Frequenzen umfasst;
    - eine Komponente zum Abbilden der Meta-Information auf die eine Vielzahl der Codierungssymbole; und
    - eine Komponente zum Erhalten der vom Benutzer bereitgestellten Audioinformation, die Meta-Information eingebunden hat, durch Einbinden der abgebildeten Meta-Information in die vom Benutzer bereitgestellte Audioinformation.
EP02788411A 2002-12-20 2002-12-20 Verfahren und vorrichtung zur organisation von benutzerseitiger information mit meta-information Expired - Lifetime EP1574093B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2002/005563 WO2004057892A1 (en) 2002-12-20 2002-12-20 Method and device for organizing user provided information with meta-information

Publications (2)

Publication Number Publication Date
EP1574093A1 EP1574093A1 (de) 2005-09-14
EP1574093B1 true EP1574093B1 (de) 2009-04-01

Family

ID=32676692

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02788411A Expired - Lifetime EP1574093B1 (de) 2002-12-20 2002-12-20 Verfahren und vorrichtung zur organisation von benutzerseitiger information mit meta-information

Country Status (7)

Country Link
US (2) US7797331B2 (de)
EP (1) EP1574093B1 (de)
CN (1) CN1714584B (de)
AT (1) ATE427623T1 (de)
AU (1) AU2002353389A1 (de)
DE (1) DE60231844D1 (de)
WO (1) WO2004057892A1 (de)

Families Citing this family (234)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8645137B2 (en) 2000-03-16 2014-02-04 Apple Inc. Fast, language-independent method for user authentication by voice
CN1714584B (zh) 2002-12-20 2010-05-05 诺基亚有限公司 采用元信息来组织用户提供信息的方法及装置
US20040125958A1 (en) * 2002-12-26 2004-07-01 Brewster Bruce F. Headphone for wireless use with authentication/encryption
US20050125497A1 (en) * 2003-12-08 2005-06-09 Hevener Dennis L. Shareable log book system for amateur radio
US8706686B2 (en) * 2003-12-24 2014-04-22 Split-Vision Kennis B.V. Method, computer system, computer program and computer program product for storage and retrieval of data files in a data storage means
US20060044398A1 (en) * 2004-08-31 2006-03-02 Foong Annie P Digital image classification system
KR100647953B1 (ko) * 2004-11-04 2006-11-23 엘지전자 주식회사 이미지 메타정보를 제공하는 광학줌 카메라형이동통신단말기
KR20060063013A (ko) * 2004-12-06 2006-06-12 삼성전자주식회사 메타데이터를 이용한 사용자 정합 방법
US8583139B2 (en) 2004-12-31 2013-11-12 Nokia Corporation Context diary application for a mobile terminal
US20080270888A1 (en) * 2005-02-07 2008-10-30 Electronics And Telecommunications Research Institute Method For Generating Metadata For Symbolic Music of Traditional Music, and Apparatus For Coding/Decoding Multimedia Data Using the Same
KR100673653B1 (ko) * 2005-08-22 2007-01-24 주식회사 텔레칩스 메타 데이터 검색 방법
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
JP4657944B2 (ja) * 2006-02-20 2011-03-23 ソニー・エリクソン・モバイルコミュニケーションズ株式会社 情報処理装置、情報処理方法、情報処理プログラム、及び携帯端末装置
FR2897743B1 (fr) * 2006-02-22 2008-04-04 Alcatel Sa Terminal de communication mobile bidirectionnelle comprenant une fonction d'acquisition de contexte
US20070198271A1 (en) * 2006-02-23 2007-08-23 Dana Abramson Method for training a user of speech recognition software
US20070211871A1 (en) * 2006-03-08 2007-09-13 David Sjolander Method and system for organizing incident records in an electronic equipment
KR100678091B1 (ko) * 2006-04-19 2007-02-02 삼성전자주식회사 휴대단말기의 데이터 관리 방법 및 장치
US7707222B2 (en) * 2006-07-06 2010-04-27 The United States Of America As Represented By The Secretary Of The Air Force Method and apparatus for providing access to information systems via e-mail
US9798859B2 (en) * 2006-07-07 2017-10-24 Roche Diabetes Care, Inc Fluid delivery device and methods of its operation
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
TW200824408A (en) * 2006-11-27 2008-06-01 Htc Corp Methods and systems for information retrieval during communication, and machine readable medium thereof
US9135797B2 (en) * 2006-12-28 2015-09-15 International Business Machines Corporation Audio detection using distributed mobile computing
US8345858B2 (en) * 2007-03-21 2013-01-01 Avaya Inc. Adaptive, context-driven telephone number dialing
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8788485B2 (en) * 2007-05-11 2014-07-22 International Business Machines Corporation Method and system for remote binding of meta-content to a uniform resource identifier
US7460149B1 (en) * 2007-05-28 2008-12-02 Kd Secure, Llc Video data storage, search, and retrieval using meta-data and attribute data in a video surveillance system
JP2009037320A (ja) * 2007-07-31 2009-02-19 Toshiba Corp 情報処理装置、情報処理装置の制御方法
US20090041428A1 (en) * 2007-08-07 2009-02-12 Jacoby Keith A Recording audio metadata for captured images
US20090156186A1 (en) * 2007-12-12 2009-06-18 Lyle Ruthie D Method and system for managing contacts in a mobile communications device
US10002189B2 (en) 2007-12-20 2018-06-19 Apple Inc. Method and apparatus for searching using an active ontology
US9128178B2 (en) * 2007-12-27 2015-09-08 Seiko Epson Corporation Correlation calculation process execution method, control circuit, signal processing circuit, and position calculation device
US20090172546A1 (en) * 2007-12-31 2009-07-02 Motorola, Inc. Search-based dynamic voice activation
US9330720B2 (en) * 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US8996376B2 (en) 2008-04-05 2015-03-31 Apple Inc. Intelligent text-to-speech conversion
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
PL3300076T3 (pl) 2008-07-11 2019-11-29 Fraunhofer Ges Forschung Koder audio i dekoder audio
US20100030549A1 (en) 2008-07-31 2010-02-04 Lee Michael M Mobile device having human language translation capability with positional feedback
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US9491573B2 (en) 2008-11-06 2016-11-08 Texas Instruments Incorporated Communication device for providing value-added information based upon content and/or context information
US9904681B2 (en) * 2009-01-12 2018-02-27 Sri International Method and apparatus for assembling a set of documents related to a triggering item
US20100199176A1 (en) * 2009-02-02 2010-08-05 Chronqvist Fredrik A Electronic device with text prediction function and method
US9424578B2 (en) 2009-02-24 2016-08-23 Ebay Inc. System and method to provide gesture functions at a device
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US20120309363A1 (en) 2011-06-03 2012-12-06 Apple Inc. Triggering notifications associated with tasks items that represent tasks to perform
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US9431006B2 (en) 2009-07-02 2016-08-30 Apple Inc. Methods and apparatuses for automatic speech recognition
KR101679744B1 (ko) 2009-11-10 2016-12-06 삼성전자주식회사 단말기에서 데이터 처리 방법 및 그를 수행하는 장치
US11416214B2 (en) 2009-12-23 2022-08-16 Google Llc Multi-modal input on an electronic device
EP3091535B1 (de) 2009-12-23 2023-10-11 Google LLC Multimodale eingabe in eine elektronische vorrichtung
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
WO2011089450A2 (en) 2010-01-25 2011-07-28 Andrew Peter Nelson Jerram Apparatuses, methods and systems for a digital conversation management platform
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US9112989B2 (en) * 2010-04-08 2015-08-18 Qualcomm Incorporated System and method of smart audio logging for mobile devices
BE1019349A5 (fr) * 2010-05-25 2012-06-05 H O P Procede et dispositif pour stocker et/ou reproduire du son et des images.
US8780741B2 (en) 2010-06-21 2014-07-15 International Business Machines Corporation On-demand information retrieval using wireless communication devices
US9241014B2 (en) 2010-10-12 2016-01-19 United Services Automobile Association (Usaa) Methods, systems, and computer-readable media for transferring or recovering a communication between a number of participants
MX2013005358A (es) * 2010-11-13 2014-02-28 Airpatrol Corp Sistema de politica dinamica a base de contexto para dispositivos moviles e infraestructura de red de apoyo.
US8358843B2 (en) * 2011-01-31 2013-01-22 Yahoo! Inc. Techniques including URL recognition and applications
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
KR20140024434A (ko) * 2011-06-08 2014-02-28 샤잠 엔터테인먼트 리미티드 수신된 데이터의 비교를 수행하고 비교에 기초하여 후속 서비스를 제공하는 방법 및 시스템
US20120321062A1 (en) * 2011-06-17 2012-12-20 Fitzsimmons Jeffrey E Telephonic Conference Access System
US8994660B2 (en) 2011-08-29 2015-03-31 Apple Inc. Text correction processing
US8918773B2 (en) * 2011-09-22 2014-12-23 Verizon Patent And Licensing Inc. Partially upgradable application and performing database operations with the application
US9111317B2 (en) 2011-12-21 2015-08-18 Facebook, Inc. Tagging posted content in a social networking system with media information
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9008688B2 (en) * 2012-05-07 2015-04-14 Qualcomm Incorporated Calendar matching of inferred contexts and label propagation
US9280610B2 (en) 2012-05-14 2016-03-08 Apple Inc. Crowd sourcing information to fulfill user requests
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
US9721563B2 (en) 2012-06-08 2017-08-01 Apple Inc. Name recognition system
US20130331971A1 (en) * 2012-06-10 2013-12-12 Eran Bida Watermarking and using same for audience measurement
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
CN102768616A (zh) * 2012-06-29 2012-11-07 鸿富锦精密工业（深圳）有限公司 电子设备及其备忘添加方法
US8892480B2 (en) 2012-07-25 2014-11-18 Aro, Inc. Contextual information provider
CN103680549B (zh) * 2012-09-10 2016-06-01 联想(北京)有限公司 一种音频处理方法及电子设备
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US9547647B2 (en) 2012-09-19 2017-01-17 Apple Inc. Voice-based media searching
KR101413987B1 (ko) * 2012-10-02 2014-07-01 (주)이스트소프트 마인드맵 사용자 인터페이스를 포함하는 전자기기, 및 이를 이용한 마인드맵 작성방법
EP2720222A1 (de) * 2012-10-10 2014-04-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur wirksamen Synthese von Sinosoiden und Sweeps durch Verwendung spektraler Muster
EP3809407A1 (de) 2013-02-07 2021-04-21 Apple Inc. Sprachauslöser für digitalen assistent
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
WO2014144579A1 (en) 2013-03-15 2014-09-18 Apple Inc. System and method for updating an adaptive speech recognition model
AU2014233517B2 (en) 2013-03-15 2017-05-25 Apple Inc. Training an at least partial voice command system
CN103258544B (zh) * 2013-04-15 2016-05-25 深圳市海云天科技股份有限公司 一种考试系统
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
WO2014197334A2 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
WO2014197336A1 (en) 2013-06-07 2014-12-11 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
KR101922663B1 (ko) 2013-06-09 2018-11-28 애플 인크. 디지털 어시스턴트의 둘 이상의 인스턴스들에 걸친 대화 지속성을 가능하게 하기 위한 디바이스, 방법 및 그래픽 사용자 인터페이스
JP2016521948A (ja) 2013-06-13 2016-07-25 アップル インコーポレイテッド 音声コマンドによって開始される緊急電話のためのシステム及び方法
ITRM20130383A1 (it) * 2013-06-28 2014-12-29 Simone Giacco Metodo di elaborazione di un messaggio audio
US9503532B2 (en) 2013-09-03 2016-11-22 Western Digital Technologies, Inc. Rediscovery of past data
US10049413B2 (en) 2013-09-20 2018-08-14 Vulcan Technologies Llc Automatically creating a hierarchical storyline from mobile device data
US9612802B1 (en) * 2013-09-27 2017-04-04 Emc Corporation Contextual interactive program feedback
US10331305B2 (en) 2013-11-12 2019-06-25 Microsoft Technology Licensing, Llc Aggregating and presenting event information
US9760756B2 (en) 2013-12-05 2017-09-12 Aro, Inc. Venue identification using sensor fingerprints
US9541652B2 (en) 2013-12-06 2017-01-10 Aro, Inc. Accurate mobile context detection at low sensor cost
US10296160B2 (en) 2013-12-06 2019-05-21 Apple Inc. Method for extracting salient dialog usage from live data
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US20160381108A1 (en) * 2015-06-26 2016-12-29 Bart Van Coppenolle Co-device to a Mobile Device for Integrated Use and Experience of Mobile Applications on Another Device and Display
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10277703B2 (en) 2015-07-22 2019-04-30 International Business Machines Corporation Optimizing bandwidth usage and improving performance for web page caching
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US9743139B2 (en) * 2015-12-23 2017-08-22 Rovi Guides, Inc. Methods and systems for detecting overlaps between calendar appointments and media asset transmission times
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
DK179309B1 (en) 2016-06-09 2018-04-23 Apple Inc Intelligent automated assistant in a home environment
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
DK179049B1 (en) 2016-06-11 2017-09-18 Apple Inc Data driven natural language event detection and classification
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179343B1 (en) 2016-06-11 2018-05-14 Apple Inc Intelligent task discovery
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
US10055481B2 (en) * 2016-07-20 2018-08-21 LogsHero Ltd. Method and system for automatic event classification
US10474753B2 (en) 2016-09-07 2019-11-12 Apple Inc. Language identification using recurrent neural networks
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
JP6852337B2 (ja) * 2016-09-29 2021-03-31 富士通株式会社 情報処理装置、情報処理プログラム、情報処理システム及び情報処理方法
US11281993B2 (en) 2016-12-05 2022-03-22 Apple Inc. Model and ensemble compression for metric learning
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
US10417266B2 (en) 2017-05-09 2019-09-17 Apple Inc. Context-aware ranking of intelligent response suggestions
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. USER INTERFACE FOR CORRECTING RECOGNITION ERRORS
US10395654B2 (en) 2017-05-11 2019-08-27 Apple Inc. Text normalization based on a data-driven learning network
DK201770439A1 (en) 2017-05-11 2018-12-13 Apple Inc. Offline personal assistant
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
US11301477B2 (en) 2017-05-12 2022-04-12 Apple Inc. Feedback analysis of a digital assistant
DK201770429A1 (en) 2017-05-12 2018-12-14 Apple Inc. LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK201770432A1 (en) 2017-05-15 2018-12-21 Apple Inc. Hierarchical belief states for digital assistants
DK201770431A1 (en) 2017-05-15 2018-12-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US10311144B2 (en) 2017-05-16 2019-06-04 Apple Inc. Emoji word sense disambiguation
US10303715B2 (en) 2017-05-16 2019-05-28 Apple Inc. Intelligent automated assistant for media exploration
DK179549B1 (en) 2017-05-16 2019-02-12 Apple Inc. FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES
US10403278B2 (en) 2017-05-16 2019-09-03 Apple Inc. Methods and systems for phonetic matching in digital assistant services
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
US10657328B2 (en) 2017-06-02 2020-05-19 Apple Inc. Multi-task recurrent neural network architecture for efficient morphology handling in neural language modeling
US10445429B2 (en) 2017-09-21 2019-10-15 Apple Inc. Natural language understanding using vocabularies with compressed serialized tries
US10755051B2 (en) 2017-09-29 2020-08-25 Apple Inc. Rule-based natural language processing
US10636424B2 (en) 2017-11-30 2020-04-28 Apple Inc. Multi-turn canned dialog
US10733982B2 (en) 2018-01-08 2020-08-04 Apple Inc. Multi-directional dialog
US10733375B2 (en) 2018-01-31 2020-08-04 Apple Inc. Knowledge-based framework for improving natural language understanding
US10789959B2 (en) 2018-03-02 2020-09-29 Apple Inc. Training speaker recognition models for digital assistants
US10592604B2 (en) 2018-03-12 2020-03-17 Apple Inc. Inverse text normalization for automatic speech recognition
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US10909331B2 (en) 2018-03-30 2021-02-02 Apple Inc. Implicit identification of translation payload with neural machine translation
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
US10984780B2 (en) 2018-05-21 2021-04-20 Apple Inc. Global semantic word embeddings using bi-directional recurrent neural networks
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. VIRTUAL ASSISTANT OPERATION IN MULTI-DEVICE ENVIRONMENTS
DK179822B1 (da) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
US11386266B2 (en) 2018-06-01 2022-07-12 Apple Inc. Text correction
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US10496705B1 (en) 2018-06-03 2019-12-03 Apple Inc. Accelerated task performance
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. USER ACTIVITY SHORTCUT SUGGESTIONS
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
DK201970511A1 (en) 2019-05-31 2021-02-15 Apple Inc Voice identification in digital assistant systems
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US11652656B2 (en) * 2019-06-26 2023-05-16 International Business Machines Corporation Web conference replay association upon meeting completion
CN110751591B (zh) * 2019-09-16 2023-04-07 北卡科技有限公司 一种基于模糊推理系统的自适应彩色图像隐写方法
WO2021056255A1 (en) 2019-09-25 2021-04-01 Apple Inc. Text detection using global geometry estimators

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4882765A (en) * 1987-05-22 1989-11-21 Maxwell Ray F Data transmission system
JPH04195338A (ja) * 1990-11-28 1992-07-15 Hitachi Ltd ファジィ推論方式
US5537467A (en) * 1994-08-23 1996-07-16 Bell Communications Research, Inc. Method for forwarding a call to a temporarily utilized portable telephone
AU703123B2 (en) 1996-02-23 1999-03-18 Bell Communications Research, Inc. Distributed service management system and method for personal communication services
US5974300A (en) 1996-07-30 1999-10-26 Lucent Technologies Inc. Two-way wireless cellular messaging system
US6212199B1 (en) * 1997-03-18 2001-04-03 Apple Computer, Inc. Apparatus and method for interpretation and translation of serial digital audio transmission formats
US6075755A (en) * 1997-05-12 2000-06-13 Recall Services, Inc. Medical reminder system and messaging watch
JPH10313284A (ja) * 1997-05-12 1998-11-24 Sony Corp 復調装置及び復調方法
US6236768B1 (en) 1997-10-14 2001-05-22 Massachusetts Institute Of Technology Method and apparatus for automated, context-dependent retrieval of information
US6094660A (en) 1997-11-21 2000-07-25 Telefonaktiebolaget Lm Ericsson Customer administrative system management of redundant database network elements in a telecommunications system
US6199066B1 (en) * 1998-07-20 2001-03-06 Telefonaktiebolaget L M Ericsson Meta-service activating interface between a customer administrative system and database network elements of a communications network
US7051275B2 (en) * 1998-09-15 2006-05-23 Microsoft Corporation Annotations for multiple versions of media content
EP1087306A3 (de) 1999-09-24 2004-11-10 Xerox Corporation Meta-Dokumente und entsprechendes Verwaltungsverfahren
US6721741B1 (en) * 2000-01-24 2004-04-13 Friskit, Inc. Streaming media search system
US7415662B2 (en) * 2000-01-31 2008-08-19 Adobe Systems Incorporated Digital media management apparatus and methods
JP2001306581A (ja) * 2000-04-18 2001-11-02 Sony Corp ミドルウェアおよびミドルウェアを用いたメディアデータ視聴機器
DE60039262D1 (de) * 2000-05-17 2008-07-31 Symstream Technology Holdings Octave Pulse Data Kodierung- und Dekodierungsverfahren und Vorrichtung
US7346776B2 (en) * 2000-09-11 2008-03-18 Digimarc Corporation Authenticating media signals by adjusting frequency characteristics to reference values
US7076503B2 (en) * 2001-03-09 2006-07-11 Microsoft Corporation Managing media objects in a database
US6741996B1 (en) * 2001-04-18 2004-05-25 Microsoft Corporation Managing user clips
US6804684B2 (en) * 2001-05-07 2004-10-12 Eastman Kodak Company Method for associating semantic information with multiple images in an image database environment
US20020194246A1 (en) * 2001-06-14 2002-12-19 International Business Machines Corporation Context dependent calendar
US6678613B2 (en) 2001-09-21 2004-01-13 International Business Machines Corporation Method and apparatus for notifying a user of an appointment
EP1696355A1 (de) 2001-09-25 2006-08-30 Sony Deutschland GmbH Automatische Erzeugung von Metadaten durch das Erfassen des Kontexts
US7020494B2 (en) * 2002-02-07 2006-03-28 Sap Aktiengesellschaft Integrating contextual information into mobile enterprise applications
US7441194B2 (en) * 2002-02-20 2008-10-21 Microsoft Corporation Calendar-based user interface system
US7009643B2 (en) * 2002-03-15 2006-03-07 Canon Kabushiki Kaisha Automatic determination of image storage location
US20030187820A1 (en) * 2002-03-29 2003-10-02 Michael Kohut Media management system and process
US8516114B2 (en) * 2002-03-29 2013-08-20 International Business Machines Corporation Method and apparatus for content pre-fetching and preparation
US20040034655A1 (en) * 2002-07-17 2004-02-19 Tecu Kirk Steven Multimedia system and method
US7624143B2 (en) 2002-12-12 2009-11-24 Xerox Corporation Methods, apparatus, and program products for utilizing contextual property metadata in networked computing environments
CN1714584B (zh) 2002-12-20 2010-05-05 诺基亚有限公司 采用元信息来组织用户提供信息的方法及装置

Also Published As

Publication number Publication date
AU2002353389A1 (en) 2004-07-14
US7797331B2 (en) 2010-09-14
EP1574093A1 (de) 2005-09-14
US8612473B2 (en) 2013-12-17
ATE427623T1 (de) 2009-04-15
US20110060754A1 (en) 2011-03-10
CN1714584B (zh) 2010-05-05
DE60231844D1 (de) 2009-05-14
WO2004057892A1 (en) 2004-07-08
US20060217990A1 (en) 2006-09-28
CN1714584A (zh) 2005-12-28

Similar Documents

Publication Publication Date Title
EP1574093B1 (de) Verfahren und vorrichtung zur organisation von benutzerseitiger information mit meta-information
US8370314B2 (en) Replacing a master media file
US7753260B2 (en) Information processing system, information processing method, program, and recording system
CN1272733C (zh) 提供附加信息的方法
EP1900207B1 (de) Erzeugen standardisierter playlists und aufrechterhaltung von kohärenz
US7877408B2 (en) Digital audio track set recognition system
US20040132432A1 (en) Voice recordal methods and systems
CA2610002C (en) Device specific content indexing for optimized device operation
EP1770559A2 (de) Vorrichtung zur Informationsverarbeitung, Wiedergabevorrichtung, Kommunikationsverfahren, Wiedergabeverfahren und Computerprogramme
US20040038670A1 (en) Mail server, program and mobile terminal for realizing the mail server
US20070011142A1 (en) Method and apparatus for non-redundant search results
WO2010000914A1 (en) Method and system for searching multiple data types
EP2336898A1 (de) Verfahren und system zur verarbeitung elektronischer daten
US20100153393A1 (en) Constructing album data using discrete track data from multiple sources
CN112866088A (zh) 一种即时通讯应用中的用户画像方法和系统
KR100737998B1 (ko) 사용자가 제공한 정보를 메타 정보와 함께 체계화시키는방법 및 장치
WO2007005607A2 (en) Contact information managing apparatus and method
WO2007073097A1 (en) Method and system for sorting/searching file and record media therefor
Anerousis et al. Making voice knowledge pervasive
US8204192B1 (en) Documented call log
JP2007148740A (ja) ファイル制御装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050428

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO

DAX Request for extension of the european patent (deleted)
RIN1 Information on inventor provided before grant (corrected)

Inventor name: MYKA, ANDREAS

Inventor name: THEIMER, WOLFGANG

Inventor name: LUECK, MATTHIAS

Inventor name: GOERTZ, UDO

Inventor name: RATEITSCHEK, KLAUS

Inventor name: WEINGART, PETER

Inventor name: SERAFAT, REZA

Inventor name: MAEKELAE, JAKKE

Inventor name: HABERLAND-SCHLOESSER, KNUT

17Q First examination report despatched

Effective date: 20070705

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

RIN1 Information on inventor provided before grant (corrected)

Inventor name: MYKA, ANDREAS

Inventor name: MAEKELAE, JAKKE

Inventor name: GOERTZ, UDO

Inventor name: WEINGART, PETER

Inventor name: THEIMER, WOLFGANG

Inventor name: SERAFAT, REZA

Inventor name: LUECK, MATTHIAS

Inventor name: HABERLAND-SCHLOESSER, KNUT

Inventor name: RATEITSCHEK, KLAUS

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60231844

Country of ref document: DE

Date of ref document: 20090514

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090902

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090712

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090701

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

26N No opposition filed

Effective date: 20100105

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100701

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091231

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091231

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090702

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090401

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20150910 AND 20150916

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 60231844

Country of ref document: DE

Owner name: NOKIA TECHNOLOGIES OY, FI

Free format text: FORMER OWNER: NOKIA CORP., 02610 ESPOO, FI

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: NOKIA TECHNOLOGIES OY, FI

Effective date: 20170109

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20201209

Year of fee payment: 19

Ref country code: FR

Payment date: 20201112

Year of fee payment: 19

Ref country code: DE

Payment date: 20201208

Year of fee payment: 19

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60231844

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20211220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211220

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220701

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211231