EP1973103B1 - System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück - Google Patents

System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück Download PDF

Info

Publication number
EP1973103B1
EP1973103B1 EP07006014.0A EP07006014A EP1973103B1 EP 1973103 B1 EP1973103 B1 EP 1973103B1 EP 07006014 A EP07006014 A EP 07006014A EP 1973103 B1 EP1973103 B1 EP 1973103B1
Authority
EP
European Patent Office
Prior art keywords
audio
additional information
piece
signal
playback
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP07006014.0A
Other languages
English (en)
French (fr)
Other versions
EP1973103A1 (de
Inventor
Jeroen De Weerd
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vodafone Holding GmbH
Original Assignee
Vodafone Holding GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vodafone Holding GmbH filed Critical Vodafone Holding GmbH
Priority to EP07006014.0A priority Critical patent/EP1973103B1/de
Publication of EP1973103A1 publication Critical patent/EP1973103A1/de
Application granted granted Critical
Publication of EP1973103B1 publication Critical patent/EP1973103B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/018—Audio watermarking, i.e. embedding inaudible data in the audio signal

Definitions

  • the invention relates to a playback system and to a method for playing back an audio piece at a playback site and for providing additional information related to the audio piece.
  • An audio watermark is a data signal within an audio signal that is essentially inaudible due to the psycho-acoustic properties of the human hearing. Audio watermarks are generated by modifications of the audio signal that are essentially imperceptible to the human ear, but can be detected and evaluated electronically. Through the use of watermark technology, additional information can be embedded into an audio signal and this information can be used, for example, to prove copyright of an audio piece or else the audio watermark can contain data identifying the audio piece.
  • German patent application DE 100 00 933 describes a method wherein identification information comprising, for example, the title and the performer of an audio piece, is embedded into the audio piece as an audio watermark.
  • identification information comprising, for example, the title and the performer of an audio piece
  • the audio watermark in the radio signal is detected and the contained identification information about the audio piece is extracted.
  • the audio watermark can be decoded from the acoustic signal of the audio piece using a mobile receiving device for decoding the additional information and displaying it to the user of the mobile receiving device.
  • a user hears an audio piece, for example, on the radio or somewhere in the street, he can obtain information about the title and performer using the receiving device. On the basis of this information he can, for example, purchase the audio piece if desired.
  • the identification information contained in the audio watermark can comprise ordering information allowing the user to order the audio piece using the receiving device.
  • the audio data of the audio piece is provided together with the embedded audio watermark at the playback site of the audio piece.
  • compressed audio files such as, for example audio files in the mp3 format
  • these files owing to their reduced data volume, can be simply acquired electronically.
  • audio watermarks are often not robust to compression of audio data, so that the audio watermarks can no longer be read out error-free.
  • other manipulations of the audio data such as for example format changes can at least partially destroy the audio watermark, so that the additional information contained in the audio watermarks can be at least partially unusable.
  • EP 1 739 618 A1 discloses a playback device for presenting video and audio data (in a cinema) into which a watermarking device can be included and every time data is presented a watermark is embedded, which can later be used to identify the copy of the data.
  • EP 1 672 638 A1 relates to a signal output apparatus, which reads an audio and video signal from an optical disk and retrieves optional audio and video content from a server. By means of signal processors playback signals are generated from the data from both sources. Then, the signals are multiplexed to develop a composite signal, which is output through an output port.
  • DE 100 00 933 A1 refers to a device for identifying broadcasted audio pieces.
  • the broadcast signal comprises a data signal including identification information.
  • the data signal is embedded into the audio signal as a watermark and the receiving device, receiving the audio signal extracts the data signal from the audio signal and stores the identification information upon a command from the user.
  • the data signal is extracted in the device receiving the broadcast signal, and in another embodiment, the data signal is extracted in the device receiving the acoustic signal of the audio piece.
  • the publication L. de C.T. Gomes et al., "Audio Watermarking and Fingerprinting: For Which applications?" describes principles of audio watermarking and audio fingerprinting and applications of both techniques.
  • One application comprises the provision content information about an audio excerpt.
  • the publication also describes a system using audio watermarks to embed additional information into an audio signal.
  • DE 43 13 107 C1 describes an arrangement for identifying broadcast contributions using an identification code.
  • the identification code is allocated to the contribution by a central authority and broadcasted in parallel to the contribution.
  • the receiver of the broadcast signal has a filter, which extracts the identification code.
  • the audio signal included in the broadcast signal is played back and the identification is stored in the receiver.
  • this object is achieved by a system comprising the features of claim 1 and by a method comprising the features of claim 8.
  • the invention proposes a system comprising playback system for playing back an audio piece at a playback site and for providing additional information related to the audio piece and a central system.
  • the playback system comprises an acquisition component for acquiring audio data of the audio piece and a means for acquiring the additional information related to the audio piece.
  • the playback system comprises an encoding component, which is located at the playback site of the audio piece and which is configured to combine the audio data and the additional information and to generate an audio signal containing the audio piece and the additional information.
  • the playback system also comprises a playback component, which is connected to the encoding component and which is configured to generate an acoustic signal from the audio signal.
  • the encoding component is configured to generate a first intermediate audio signal from the additional information and a second intermediate audio signal from the audio data and to superpose the second intermediate audio signal with the first intermediate audio signal to generate the audio signal from which the acoustic signal is generated, the frequency range of the first intermediate audio signal being above the highest frequency the human ear is able to perceive so that the additional information contained in the acoustic signal is essentially imperceptible to the listener of the audio piece.
  • the playback system is connected to the central system via a data network and in that the means for acquiring the additional information is an interface component of the playback system which is configured to send a request to the central system that indicates the audio piece for which the additional information are requested, and wherein the playback system is configured to retrieve the additional information from the central system via the data network in response to the acquisition of the audio data of the audio piece and the sent request.
  • the invention proposes a method for playing back an audio piece at a playback site and for providing additional information related to the audio piece.
  • the method comprises acquiring audio data of the audio piece and acquiring information related to the audio piece. Further, the method comprises combining the audio signal and the additional information at the playback site of the audio piece and generating an audio signal containing the audio piece and the additional information. The method also comprises generating an acoustic signal from the audio signal.
  • the invention involves the idea, that the additional information is not embedded in the audio data provided at the playback site, but is added to the audio signal representing the audio piece at the playback site of the audio piece. Thus, it is prevented, that the additional information is corrupted by manipulations of the audio data that are carried out before or during the transmission of the audio data to the playback site. Moreover, the invention allows for a greater flexibility, if different audio data is provided to different playback systems, since in this case, identical audio data can be provided to the different playback systems and only the additional information have to be customized to the different playback systems.
  • the acoustic signal is generated in such a way that the additional information is essentially imperceptible to the human ear, when the audio signal is transformed into an acoustic signal.
  • the additional information may be extracted from the acoustic signal by means of a receiving device.
  • the encoding component is configured to generate a first intermediate audio signal from the additional information and a second intermediate audio signal from the audio data and to superpose the second intermediate audio signal with the first intermediate audio signal to generate the audio signal from which the acoustic signal is generated.
  • This embodiment has the advantage, that the additional information can be added to the audio signal of the audio data in a simple manner.
  • the audio data itself is not manipulated, so that less computing power is necessary for adding the additional information to the audio signal of the audio piece.
  • the frequency range of the first intermediate audio signal is above the highest frequency the human ear is able to perceive.
  • the human hearing is able to perceive sounds up to a certain frequency that is typically about 20 kHz.
  • a higher frequency for the first intermediate audio signal representing the additional information it can be assured that the additional information contained in the acoustic signal of the audio piece is essentially imperceptible to the listener of the audio piece.
  • the additional information contains identification information for identifying the audio piece and/or ordering information for ordering the audio piece electronically.
  • the identification information a listener of the audio piece can obtain information related to an audio piece he is listening to.
  • the identification information may comprise the title and/or the performer of the audio piece.
  • the ordering information the listener can order the audio piece electronically, if he is interested in obtaining a copy of the audio piece.
  • the order information may indicate an ordering system, from which the audio piece can be obtained, and/or a price for ordering the audio piece.
  • At least part of the additional information is encrypted in such a way that it can be decrypted using a decryption key that is present in a receiving device, which is configured to receive the acoustic signal and to extract the additional information from the acoustic signal, and the audio signal comprises the encrypted additional information.
  • the playback system is connected to a central system via a data network and the means for acquiring the additional information is configured to retrieve the additional information from the central system via the data network in response to the acquisition of the audio data of the audio piece.
  • the additional information is provided by a central system separate from the audio data of the audio piece.
  • the additional data are retrieved in response to the acquisition of the audio data by the acquisition component.
  • this leads to a greater flexibility and to a simpler implementation of the invention, if customized additional information is provided for different playback systems that receive the same audio data.
  • an identical data stream can be provided to the different playback systems and the additional information that is customized to the playback systems can be retrieved from the central system.
  • the means for acquiring the additional information is configured to transmit identification data identifying the playback system to the central system and at least part of the additional information that is retrieved by the playback system is chosen as function of the identification data from a plurality of different additional information related to the audio piece.
  • a receiving device is configured to receive an acoustic signal played back by a playback system of the type described before and to extract the additional information from the acoustic signal.
  • the receiving device may be used by a listener of the audio piece to receive and to evaluate the additional information contained in the acoustic signal of the audio piece.
  • the receiving device comprises a display unit for depicting at least part of the additional information.
  • This embodiment has the advantage that the additional information related to the audio piece can be depicted to the listener of the audio piece.
  • the identification information related to the audio piece may be depicted at the displaying unit of the receiving device.
  • the receiving device is configured to place an electronic order for the audio piece on the basis of the ordering information contained in the additional information in response to an operating action on the receiving device and in that it is configured to receive audio data of the audio piece from an ordering system as an answer to the order.
  • the listener can use the receiving device for placing an order for the audio piece.
  • the listener is able to order the audio piece, while he is listening to the audio piece.
  • the receiving device is configured to place an electronic order for the audio piece on the basis of the ordering information contained in the additional information in response to an operating action on the receiving device and in that it is configured to receive audio data of the audio piece from an ordering system as an answer to the order.
  • the listener can use the receiving device for placing an order for the audio piece.
  • the listener is able to order the audio piece, while he is listening to the audio piece.
  • the receiving device is a mobile communication device.
  • the invention provides a system comprises a playback system of the type described before and a receiving device of the type described before.
  • the system may be configured to obtain verification for the playback of the audio piece at the playback site.
  • the verification may be used for evaluating royalty payments for the playback of the audio piece at a playback site that is located in a commercial facility, such as, for example, a shop or a restaurant.
  • the receiving device is configured to store the additional information of the audio piece extracted from the acoustic signal played back by the playback system.
  • the playback system and/or the receiving device is arranged in a protected housing.
  • figure 1 is a schematic diagram of an embodiment of the present invention.
  • audio data 10 of audio pieces is provided to a playback system 20 located at the playback site of the audio pieces, as is schematically shown by the arrow 30.
  • the playback site may be located in a facility open to the public, such as, for example, a hotel, a bar, a restaurant or a shop.
  • the same audio data 10 may be provided by a provider to plurality of facilities of the aforementioned type, each facility having a playback system 20 to its disposal.
  • the audio data 10 may concern any kind of audio information, such as, for example, music pieces or radio dramas. In particular, it may concern music pieces that are played back in the aforementioned facilities as background music. Further it is to be understood that, within the scope of the invention, the term audio data also refers to audio data that is contained in video data that is provided at the playback site.
  • the audio data 10 can be provided at the playback site in any manner known to a person skilled in art.
  • the audio data 10 may be provided on a sound storage medium such as a compact disc, a memory card or a record.
  • the playback system 20 may also receive the audio data 10 via the radio or via a data network such as, for example, the internet.
  • the audio data 10 may be contained in an audio stream the playback system 20 receives via radio or via a data network and that contains consecutively transmitted audio pieces.
  • the audio stream may be broadcasted to a plurality of playback systems 20 or facilities by a broadcast station or by another provider offering a music program that facilities have taken out a subscription to. The latter may particularly apply to bars or restaurants, which often have taken a subscription to a music program customized to their needs.
  • the playback system 20 comprises an acquisition component 40 for acquiring the audio data 10.
  • the acquisition module is configured as a reading device that is able to read in the audio data from the sound storage medium.
  • the reading device may, for example, be a compact disc reader, a memory card reader or a reading device for reading in records.
  • the acquisition module may be configured as a radio receiver or as a network interface that is able to receive the audio data 10.
  • the additional information related to a specific audio piece may contain identification information related to that audio piece such as, for example, the title and the performer of the audio piece as well as the performer's album on which the audio piece can be found.
  • the additional information also contains order information related to the specific audio piece comprising parameters that allow for ordering the audio piece electronically.
  • the order information may indicate an order system 50 from which the audio piece can be obtained, and it may comprise an identification code assigned to the audio piece that allows for identifying the audio piece within the order system 50. Further, the order information may indicate the price for ordering the audio piece.
  • the order information for the same audio piece may differ depending on the facility, at which the audio piece is being played back. In particular, different order systems 50 and/or different prices for ordering the audio piece may be provided to different facilities or playback systems. Since the identification code that is contained in the order information is used to identify the audio piece in the order system 50, the identification code may also differ depending on the order system 50 indicated by the order information. Another reason for assigning several identification codes to one audio piece may be that each of the several identification codes may be assigned to a specific price for ordering the audio piece so that the order system 50 can not only identify the audio piece, but can also ascertain the price for ordering the audio piece on the basis of the identification code. However, the order information may likewise contain further data that allow for determining the order price in the order system 50.
  • the order information related to an audio piece may be customized to one facility at which the audio piece is played back or it may be customized to a predefined group of facilities.
  • a predefined group of facilities For example, such a group may contain facilities to which equal order parameters are provided.
  • the additional information or at least the order information may be encrypted using an encryption key.
  • the encryption may be carried out in the playback system 20, or the playback system 20 may receive already encrypted additional information.
  • the facility, in which the audio piece is played back, or the provider of the audio data 10 may issue a decryption key for decrypting the additional information or the order information to authorized persons. By encrypting the additional information or the order information, unauthorized listeners of the audio piece are prevented from using the additional information or the order information.
  • the playback system 10 receives the audio data 10 of the audio pieces and the additional information related to the audio pieces from the same source. If the audio data 10 of one or several audio pieces is provided on a sound storage medium, the sound storage may contain data which is attached to the audio data 10 and comprise the additional information assigned to the audio pieces. If the audio data is transmitted via radio or via a data network, it is provided that the audio data is transmitted together with the additional information related to the audio pieces 10 that are contained in the audio data 10 or attached to the audio data 10.
  • the additional information may, for example, be added to the so called headers of the digital audio files that contain information about the audio file.
  • Digital audio files of certain formats such as, for example, the widely used mp3 format already comprise headers that contain identification information of the audio piece contained in the file.
  • these headers may be expanded to allow the addition of the order information.
  • Digital audio streams usually also contain headers, which contain information about the audio pieces contained in the stream.
  • the additional information may thus be added to that header, in case the audio data is transmitted to the playback system 20 in form of a digital audio stream via radio or via a data network.
  • the radio signal may also contain the additional information related to the audio pieces.
  • the additional information may be transmitted via a separate carrier frequency.
  • An analog method is known, for example, from the radio data system (RDS) that allows for transmitting additional data in addition to the radio program.
  • RDS radio data system
  • the acquisition component 40 of the playback system 20 acquires the additional information related to the audio pieces in addition to the audio data 10 of the audio pieces.
  • the acquisition component subsequently transfers the audio data 10 and the additional information to an encoding component 60 of the playback system 20 that will be described thereinafter.
  • the additional information related to the audio pieces is provided by a central system separately from the audio data 10 containing the audio pieces.
  • this allows for providing customized additional information to a plurality of playback systems 20 in an especially simple manner, since the customized additional information does not need to be directly linked to the audio data 10.
  • the playback system 20 comprises an interface component 55 for connecting the playback system 20 to a central system 70 via a data network such as, for example, the internet.
  • the interface component 55 is configured to retrieve the additional information related to the audio pieces that are to be played back in the playback system 20 from the central system 70.
  • the central system 70 may be connected to a plurality of playback systems 20, which retrieve the additional information form the central system 70.
  • the central systems may be operated by the provider of the audio data 10.
  • the playback system 20 sends a request to the central system 70 that indicates the audio pieces for which the additional information are requested.
  • the audio data 10 may be provided to the playback system 20 together with identification data that allows for identifying the audio pieces in the central system 70.
  • the identification data may be contained in the headers of the audio files or it may be stored on the sound storage medium and suitably linked to the audio data in another form.
  • the identification data is acquired by the acquisition component 70 and transfers the identification data to the interface component 55 of the playback system 20, which sends a request for transmittal of the additional information that contains the identification data to the central system 70.
  • the central system 70 Upon receipt of the request, the central system 70 ascertains the additional information related to the audio pieces specified in the request and sends the additional information to the playback system 20 as an answer to the request.
  • the additional information may be stored in a database of the central system 70 that also comprises an allocation between the additional information and the identification data.
  • the request sent to the central system 70 by a specific playback system 20 additionally contains identification data that allows the server to ascertain the custom ized additional information related to the audio pieces the request is related to.
  • the answer of the central system 70 to the request sent by specific playback system 20 comprises the additional information customized to this playback system 20.
  • the interface component 55 retrieves the additional information online, i.e., it retrieves the additional information related to a specific audio piece, when the playback system 20 plays back this audio piece.
  • the central system 20 provides at least one interface for the retrieval of the additional information related to the audio piece currently contained in the audio stream.
  • the interface components 55 of the playback systems 20 asses the interface of the central component this interface to retrieve the additional information related to the audio piece that is currently contained in the stream.
  • a specific playback system 20 retrieving the additional information identifies itself to the central system 70, whereupon the central system 70 provides access to the additional information intended for this playback system 20.
  • the encoding component 60 of the playback system 20 receives the additional information related to the audio pieces from the interface component 55 and assigns the additional information to the audio pieces that are contained in the audio data 10 received from the acquisition component 40.
  • the encoding component 60 of the playback system 20 may be implemented in form of a software program that is run in a processor of the playback system 20. It generates an audio signal that represents the audio pieces contained in the audio data 10 and the additional information related thereto. The audio signal can subsequently be transformed into an acoustic signal 75 that is played back by the playback system 20.
  • the audio signal comprises the additional information in such a form that it is essentially imperceptible to the human hearing in the subsequently generated acoustic signal 75.
  • the additional information is embedded into the audio data as an audio watermark by the encoding component 60.
  • the audio watermark is generated by modifying the audio data 10 in such a way that the modifications are essentially inaudible to the human ear, but can be detected and evaluated electronically.
  • the encoding component 60 After having embedded the audio watermark into the audio data 10 of an audio piece, the encoding component 60 generates the audio signal from the audio data 10 containing the audio watermark. Since a plurality of methods for embedding audio watermarks into audio data 10 are known to a person skilled in the art, these methods will not be described in greater detail here. In principle, any method for audio watermarking known to a person skilled in the art may be applied within the scope of the present invention.
  • the encoding component 60 generates a frequency modulated audio signal from the additional information, hereinafter referred to as audio barcode signal.
  • the frequency range of the audio barcode signal lies above the highest frequency the human ear is able to hear. As is known, the human hearing is able to hear sounds in the frequency range from approximately 20 Hz to 20 kHz.
  • the audio barcode signal has a carrier frequency that is sufficiently higher than 20 kHz to guarantee that the audio barcode signal is inaudible to the listener of the audio piece. For example, the carrier frequency is higher than 23 kHz.
  • the audio barcode signal generated from the additional information is superposed with an audio signal that is generated from the audio data 10 of the audio piece to form a new audio signal that is subsequently played back by the playback system 20.
  • this embodiment is that less computing power is necessary for generating the audio barcode signal than for embedding the audio watermark into the audio data. Therefore, this embodiment is particularly suitable for real time processing of the audio data and the additional information that may particularly be executed, when the playback system 20 receives the audio data in form of an audio stream.
  • the audio signal output by the encoding component 70 is transferred to a playback component 80 of the playback system 20, which converts the audio signal into an acoustic signal 75.
  • the playback component 80 may comprise one or several loudspeakers that are configured in a manner known to person skilled in the art.
  • a listener staying at playback site may listen to audio pieces played back by the playback system 20, but does not perceive that additional information is contained in the acoustic signal 75 of the audio pieces.
  • the listener may read in the additional information by means of a receiving device 90.
  • the receiving device 90 is portable device comprising a microphone 100 for acquiring acoustic signals 70 and transforming them into electric signals. Further, the receiving device 90 comprises a processor and a memory for storing software applications that can be run on the processor and for storing further data.
  • the processor is connected to a display unit 110 for displaying text and graphical information and to an input unit 120 that allows the listener to input commands and other data.
  • the receiving device 90 also comprises an interface for connecting the receiving device 90 to the order system 50 via a data network such as, for example, the internet.
  • the receiving device 90 is a mobile communication device such as, for example, a cell phone or a personal data assistant (PDA) that is equipped with a software program for processing the additional information contained in acoustic signals 70 that are played back by the playback system 20.
  • the signal generated by the microphone 100 is processed by a decoding component 130 of the receiving device 90 that may be implemented in form of a software application that is run on the processor of the receiving device 90.
  • the software program may be started by actuating the input unit 120 of the receiving device 90.
  • the signal generated by the microphone 100 is processed by the decoding component 130 to extract the additional information related to the audio piece from the signal.
  • the decoding component applies the decryption key to the extracted additional information to obtain decrypted addition information that is used for the further processing of the additional information in the receiving device 90.
  • the decryption key may be entered by the user of the receiving device 90 and may then be stored in the memory of receiving device 90. As aforementioned, the user may obtain the decryption key from the facility, in which the acoustic signal 75 containing the additional is played back, or from the provider of the audio data 10.
  • the additional information particularly contains identification information of an audio piece. Therefore, the decoding component 130 particularly extracts the identification information of the audio piece from the acoustic signal 75 of the audio piece. In order to display the identification information at the receiving device, the decoding component 130 transfers the identification information to the display unit 110 of the receiving device 90.
  • the additional information may contain order information that allow for placing an electronic order for the audio piece with the ordering system 50.
  • the ordering process is carried out by means of the receiving device 90.
  • the receiving device 90 provides an option for inputting a command that activates an order component 140, which is configured to generate an order for the audio piece and to send it to ordering system 50.
  • the command may be issued by actuating a soft key button of the receiving device 90 to which the command is assigned, when the display unit 110 depicts the identification information.
  • the display unit 110 preferably also depicts the order price contained in the order information extracted from the acoustic signal 75 of the audio piece so that the listener is able to take the decision for ordering the audio piece on the basis of the price information.
  • the order component 140 Upon the command for ordering the audio piece, the order component 140 that may also be implemented as software program run by the processor of the receiving device 90, sends an order to the order system 50 specified in the order information.
  • the order comprises at least the identification code of the audio piece contained in the additional information to the order system 50 via the data network. If different prices for ordering the same audio piece are provided, the order also contains data indicating the order price.
  • the order may comprise the identification code of the audio piece, to which a certain order price is assigned.
  • the order may contain additional data that are contained in the order information extracted from the acoustic signal 75 of the audio piece and that allow for determining the order price in the order system 50.
  • the order system 50 Upon receipt of the order, the order system 50 identifies the audio piece using the identification code and sends a digital audio file of the audio piece to the receiving device 90 via the data network as an answer to the order.
  • the digital audio file is received in the receiving device 90 and may be stored in the memory of the receiving device for future use.
  • the server evaluates the price for ordering the audio piece and starts out a payment process that may be carried out within the scope of a suitable payment method, such as, for example, a payment by credit card, debiting an account or payment via a telephone bill of the mobile communication provider that provides the mobile communication network the mobile communication device is used in. If several prices are assigned to the same audio piece, the order system 50 ascertains the order price on the basis of the aforementioned data that is contained in the order and that indicates the order price.
  • a suitable payment method such as, for example, a payment by credit card, debiting an account or payment via a telephone bill of the mobile communication provider that provides the mobile communication network the mobile communication device is used in.
  • the invention may particularly be used to provide additional information related to audio pieces that are played back in facilities such as, for example, shops, bars, restaurants or hotels. Those facilities may provide the additional information within the scope of an extra service to its customers.
  • the facilities may take out a subscription to a music program from a music provider.
  • the music program may be provided in form of an audio stream that is received by playback systems 20 located in the facilities.
  • the music provider may operate a central system 70, from which the playback systems 20 can retrieve additional information related to the audio pieces contained in the audio stream.
  • the music provider may provide different order information to different facilities, so that the facilities are able to chose an ordering system 50 from which their customers can order audio pieces.
  • the facilities may, for example, operate their own ordering system 50.
  • the facilities may specify the price for ordering the audio pieces individually. This makes it possible, that the facilities define their own price structures. For example, they may offer a flat rate for ordering audio pieces to its customers, so that the costumers can order audio pieces without paying for each order.
  • At least the order information may be encrypted.
  • the facilities may issue decrypting keys for decrypting the order information to the customers, which have subscribed to the extra service.
  • the invention is used to obtain verification of the playback of an audio piece at the playback site.
  • the receiving device may store the identification information related to an audio piece contained in the additional information to verify that the audio piece has been played back at the playback site.
  • the receiving device 90 may be connected to a verification system via a data network such as, for example, the internet and transmits the identification data of the audio piece to the verification system, where it is stored as a verification of the playback of the audio piece at the playback site.
  • the playback system 20 and the receiving device 90 may be arranged in one protected housing, so that it is not possible to manipulate the playback system 20 or the receiving device 20 or to remove the receiving device 90 out the area, where it is able to receive the acoustic signal 75 generated by the playback system 20.
  • the housing it may be plumbed, for example.
  • the playback system 20 and the receiving device are arranged in separate housings, and the housings are securely attached to their position, for example, by means of plumbs.
  • the invention may be used for evaluating royalty payments that are due for playing back an audio piece in a commercial facility such as for example a bar, a restaurant or a shop.
  • Those royalty payments are typically levied by collecting societies such as for example, the German "GEMA", which collect the royalty payments for the copyright holders of the audio pieces.
  • those collection societies may provide systems comprising a playback system 20 and a receiving device 30 to commercial facilities.
  • the identification data of the audio pieces that are played back in the facilities are stored within the receiving devices 90 or within a central verification system connected to a plurality of receiving devices 90. The stored identification may then be used for evaluating the royalty fees due on the basis of the audio pieces that actually have been played back in the facilities.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Reverberation, Karaoke And Other Acoustics (AREA)

Claims (8)

  1. System, das ein Wiedergabesystem (20) zum Wiedergeben eines Audiostücks an einem Wiedergabestandort und zum Bereitstellen zusätzlicher Informationen in Bezug auf das Audiostück und ein Zentralsystem (70) umfasst,
    wobei das Wiedergabesystem (20) umfasst:
    - eine Erfassungskomponente (40) zum Erfassen von Audiodaten (10) des Audiostücks,
    - ein Mittel (40, 55) zum Erfassen der zusätzlichen Informationen in Bezug auf das Audiostück,
    - eine Codierungskomponente (60), die sich bei dem Wiedergabestandort des Audiostücks befindet und die dafür konfiguriert ist, die Audiodaten und die zusätzlichen Informationen zu kombinieren und ein Audiosignal zu erzeugen, das das Audiostück und die zusätzlichen Informationen enthält, und
    - eine Wiedergabekomponente (80), die mit der Codierungskomponente (60) verbunden ist und die dafür konfiguriert ist, aus dem Audiosignal ein akustisches Signal (75) zu erzeugen, wobei die Codierungskomponente (60) dafür konfiguriert ist, aus den zusätzlichen Informationen ein erstes Zwischenaudiosignal und aus den Audiodaten ein zweites Zwischenaudiosignal zu erzeugen und das zweite Zwischenaudiosignal mit dem ersten Zwischenaudiosignal zu überlagern, um das Audiosignal zu erzeugen, aus dem das akustische Signal (75) erzeugt wird, wobei der Frequenzbereich des ersten Zwischenaudiosignals über der höchsten Frequenz liegt, die das menschliche Ohr wahrnehmen kann, so dass die in dem akustischen Signal (75) enthaltenen zusätzlichen Informationen für den Hörer des Audiostücks im Wesentlichen nicht wahrnehmbar sind, und wobei
    das Wiedergabesystem (20) über ein Datennetz mit dem Zentralsystem (70) verbunden ist und wobei das Mittel (55) zum Erfassen der zusätzlichen Informationen eine Schnittstellenkomponente des Wiedergabesystems (20) ist, die dafür konfiguriert ist, an das Zentralsystem (70) eine Anforderung zu senden, die das Audiostück angibt, für das die zusätzlichen Informationen angefordert werden, und wobei das Wiedergabesystem (20) dafür konfiguriert ist, die zusätzlichen Informationen in Ansprechen auf die Erfassung der Audiodaten (10) des Audiostücks und der gesendeten Anforderung über das Datennetz von dem Zentralsystem (70) auszulesen.
  2. System nach Anspruch 1, dadurch gekennzeichnet, dass das Mittel (55) zum Erfassen der zusätzlichen Informationen dafür konfiguriert ist, an das Zentralsystem (70) Identifizierungsdaten zu senden, die das Wiedergabesystem (20) identifizieren, und wobei wenigstens ein Teil der zusätzlichen Informationen, die durch das Wiedergabesystem (20) ausgelesen werden, aus mehreren verschiedenen zusätzlichen Informationen, die sich auf das Audiostück beziehen, in Abhängigkeit von den Identifizierungsdaten gewählt werden.
  3. System nach einem der vorhergehenden Ansprüche, dadurch gekennzeichnet, dass die zusätzlichen Informationen Identifizierungsdaten zum Identifizieren des Audiostücks und/oder Bestellinformationen zum elektronischen Bestellen des Audiostücks enthalten.
  4. System nach einem der vorhergehenden Ansprüche, dadurch gekennzeichnet, dass wenigstens ein Teil der zusätzlichen Informationen in einer Weise verschlüsselt sind, dass sie unter Verwendung eines Schlüssels, der in einer Empfangsvorrichtung (90), die dafür konfiguriert ist, das akustische Signal zu empfangen und die zusätzlichen Informationen aus dem akustischen Signal zu extrahieren, vorhanden ist, entschlüsselt werden können, und wobei das Audiosignal die verschlüsselten zusätzlichen Informationen umfasst.
  5. System nach einem der vorhergehenden Ansprüche, das eine Empfangsvorrichtung (90), die ein Mikrofon zum Erfassen des durch das Wiedergabesystem (20) wiedergegebenen akustischen Signals (75) und zum Umwandeln des akustischen Signals (75) in ein elektrisches Signal umfasst, und
    eine Decodierungskomponente (130) zum Verarbeiten des elektrischen Signals, um die zusätzlichen Informationen aus dem akustischen Signal zu extrahieren, umfasst.
  6. System nach Anspruch 5, dadurch gekennzeichnet, dass die Empfangsvorrichtung (90) dafür konfiguriert ist, die zusätzlichen Informationen des Audiostücks, die aus dem durch das Wiedergabesystem (20) wiedergegebenen akustischen Signal (75) extrahiert werden, zu speichern.
  7. System nach Anspruch 6, dadurch gekennzeichnet, dass das Wiedergabesystem (20) und/oder die Empfangsvorrichtung (90) in einem geschützten Gehäuse angeordnet sind.
  8. Verfahren zum Wiedergeben von Audiostücken bei einem Wiedergabestandort und zum Bereitstellen zusätzlicher Informationen über ein Audiostück, wobei das Verfahren die folgenden Schritte umfasst:
    - Erfassen von Audiodaten (10) des Audiostücks in einem Wiedergabesystem (20) bei dem Wiedergabestandort,
    - Senden einer Anforderung, die das Audiostück angibt, für das die zusätzlichen Informationen mittels einer Schnittstellenkomponente (55) des Wiedergabesystems (20) über ein Datennetz angefordert werden, an ein Zentralsystem (70),
    - Erfassen der zusätzlichen Informationen in Bezug auf das Audiostück in Ansprechen auf die Anforderung von dem Zentralsystem (70),
    - Kombinieren des Audiosignals und der zusätzlichen Informationen bei dem Wiedergabestandort des Audiostücks und Erzeugen eines Audiosignals, das das Audiostück und die zusätzlichen Informationen enthält, und
    - Erzeugen eines akustischen Signals (75) aus dem Audiosignal,
    wobei eine Codierungskomponente (60) aus den zusätzlichen Informationen ein erstes Zwischenaudiosignal und aus den Audiodaten ein zweites Zwischenaudiosignal erzeugt und das zweite Zwischenaudiosignal mit dem ersten Zwischenaudiosignal überlagert, um das Audiosignal zu erzeugen, aus dem das akustische Signal (75) erzeugt wird, wobei der Frequenzbereich des ersten Zwischenaudiosignals über der höchsten Frequenz liegt, die das menschliche Ohr wahrnehmen kann, so dass die in dem akustischen Signal (75) enthaltenen zusätzlichen Informationen für den Hörer des Audiostücks im Wesentlichen nicht wahrnehmbar sind.
EP07006014.0A 2007-03-23 2007-03-23 System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück Active EP1973103B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP07006014.0A EP1973103B1 (de) 2007-03-23 2007-03-23 System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP07006014.0A EP1973103B1 (de) 2007-03-23 2007-03-23 System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück

Publications (2)

Publication Number Publication Date
EP1973103A1 EP1973103A1 (de) 2008-09-24
EP1973103B1 true EP1973103B1 (de) 2017-08-30

Family

ID=38124044

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07006014.0A Active EP1973103B1 (de) 2007-03-23 2007-03-23 System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück

Country Status (1)

Country Link
EP (1) EP1973103B1 (de)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10573329B2 (en) * 2017-05-31 2020-02-25 Dell Products L.P. High frequency injection for improved false acceptance reduction

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040030900A1 (en) * 2001-07-13 2004-02-12 Clark James R. Undetectable watermarking technique for audio media

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4313107C1 (de) * 1993-04-22 1994-04-14 Walter E J Koerber Verfahren und Anordnung zum Identifizieren eines Sendebeitrags
US6505160B1 (en) * 1995-07-27 2003-01-07 Digimarc Corporation Connected audio and other media objects
DE10000933A1 (de) * 1999-07-19 2001-03-29 Fraunhofer Ges Forschung Vorrichtung und Verfahren zum Identifizieren von Audiostücken
JP2006174065A (ja) * 2004-12-15 2006-06-29 Toshiba Corp 信号出力装置及び信号出力方法
EP1739618A1 (de) * 2005-07-01 2007-01-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Verfahren zur Einbettung eines Wasserzeichens während der Wiedergabe von Multimediadaten

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040030900A1 (en) * 2001-07-13 2004-02-12 Clark James R. Undetectable watermarking technique for audio media

Also Published As

Publication number Publication date
EP1973103A1 (de) 2008-09-24

Similar Documents

Publication Publication Date Title
US7913273B2 (en) System and method for receiving broadcast audio/video works and for enabling a consumer to purchase the received audio/video works
US8155582B2 (en) Methods and systems employing digital content
KR100770187B1 (ko) 오디오 신호에 삽입된 은닉 데이터를 추출, 디코딩, 및이용하기 위한 시스템 및 방법
US6904264B1 (en) Method requesting and paying for download digital radio content
CN1612526B (zh) 内容分布方法和内容获得设备
US8468274B2 (en) Digital data distribution system with switching unit, online acquisition unit, and conversion unit for converting from first to second format
US20050028189A1 (en) System to provide access to information related to a broadcast signal
US20100046744A1 (en) Methods and Devices Responsive to Ambient Audio
EP1113605A2 (de) Verfahren und Gerät zur Identifizierung und Kauf von digitaler Musik sowie von anderen Informationsarten
EP1304622A1 (de) Verfahren und system zur steuerung eines inhaltszirkulationssystems
EP1973103B1 (de) System und Verfahren zum Bereitstellen von zusätzlichen Informationen im Zusammenhang mit einem Audiostück
JP2003022337A (ja) コンテンツ配信システム及び方法、情報提供装置、並びに情報端末装置
JP2006195061A (ja) 音響信号に対する情報の埋め込み装置、音響信号からの情報の抽出装置および音響信号再生装置
JP2002183779A (ja) 入場管理システムおよび入場管理方法
WO2000011867A1 (fr) Procede de delivrance certifiee d'une sequence audio, video ou textuelle
JP2002196985A (ja) コンテンツ流通システムの制御方法、システム、コンテンツの流通制御プログラム、および、そのプログラムを記録した記録媒体
JP2002049377A (ja) 視聴コンテンツ再生装置及び視聴コンテンツ配信システム
JP2000322443A (ja) 楽曲特定情報の端末装置、楽曲の送受信端末装置およびデータベース装置
JP2004328090A (ja) コンテンツ配信システム

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

17P Request for examination filed

Effective date: 20090323

17Q First examination report despatched

Effective date: 20090417

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20170406

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 924301

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170915

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007052148

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20170830

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 924301

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171201

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171230

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20171130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007052148

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20180531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180331

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180331

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20070323

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170830

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240320

Year of fee payment: 18

Ref country code: GB

Payment date: 20240320

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240321

Year of fee payment: 18