EP1417583A1 - Procede et dispositif permettant de rechercher et d'enregistrer des signaux multimedia - Google Patents

Procede et dispositif permettant de rechercher et d'enregistrer des signaux multimedia

Info

Publication number
EP1417583A1
EP1417583A1 EP02707866A EP02707866A EP1417583A1 EP 1417583 A1 EP1417583 A1 EP 1417583A1 EP 02707866 A EP02707866 A EP 02707866A EP 02707866 A EP02707866 A EP 02707866A EP 1417583 A1 EP1417583 A1 EP 1417583A1
Authority
EP
European Patent Office
Prior art keywords
segments
signal
search key
common
method further
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP02707866A
Other languages
German (de)
English (en)
Other versions
EP1417583B1 (fr
EP1417583A4 (fr
Inventor
Jacob Berg
Rickard Berg
Tomas Ahrne
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Popcatcher AB
Original Assignee
Popcatcher Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from SE0100642A external-priority patent/SE0100642D0/xx
Priority claimed from US10/047,532 external-priority patent/US7062442B2/en
Application filed by Popcatcher Inc filed Critical Popcatcher Inc
Publication of EP1417583A1 publication Critical patent/EP1417583A1/fr
Publication of EP1417583A4 publication Critical patent/EP1417583A4/fr
Application granted granted Critical
Publication of EP1417583B1 publication Critical patent/EP1417583B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use

Definitions

  • the invention refers to a method and a system for recording time-limited signal sequences in media channels that may contain undesirable signal components.
  • the invention may be used for recording music in radio transmissions.
  • the patent application DE 19810114 describes a method of searching and matching previously stored parts of music, called keys, against transmitted music over chosen radio channels for automatic recording of a chosen song when these keys match the transmitted song.
  • keys For each song that is to be searched for and recorded, a start key in the form of a part of the beginning of the song and an end key in the form of an end-piece of the song, is stored in a memory in the radio.
  • Those in advanced chosen keys are compared against everything that is transmitted over a number of radio channels and when a key is found, the part in-between is recorded. It is also possible to search for a certain type of music by storing category keys for matching and recording of a specific music category such as pop music, rock music, classical music or other type of music.
  • One disadvantage of this way of recording music is that only previously chosen music in the form of parts called keys of music previously stored on, e.g., a CD can be matched against radio channels for recording of wanted music. It is not possible to extract one or more keys from any song that is played on the radio for continuous matching against radio channels, enabling one to automatically get a full-length version of that song.
  • Another disadvantage is that it is not possible to record music completely without undesirable signal components since everything between the keys is recorded, which will mean that undesirable signal components such as talk and distortion due to bad transmission will be included in the songs . It is common that radio talkers or commercials interrupt the music in radio transmissions.
  • the present invention is meant to solve the above mentioned problems by supplying a procedure and a device for the searching and recording of desired source material in media channels containing undesirable signal components, where the same source material is transmitted at least twice, either in the same channel or in different channels.
  • a piece of source material can be a song, a film or anything else that is time-limited and can be considered as separate from other material .
  • the signals are continuously buffered in memory in a receiving member, over at least one media channel.
  • the next step may involve identifying and choosing a desired source material by an activation member connected to the receiving member. Out of this desired source material, a section or a representation of the section may be taken as a search key.
  • the device may also select search keys automatically in one version of performing the invention.
  • the media signal located around the search key may then be stored in a memory.
  • the search key is compared to other stored media signals or current transmissions of media signals. If a second instance of the search key is detected, signal sections that in time are connected to the search keys are compared.
  • the signal sequences that by comparison have been found to be substantially identical are identified as belonging to the same source material. Identifying common segments between the first signal segment and the second signal segment enables one to find the beginning and end of the commonality, and thus the beginning and end of the whole or part of the source material . These common segments may be stored for later use.
  • the next step may be an iteration of the above mentioned detecting of search key, storage in memory and comparison among media signals where signal segments that are identified as originating from the same source material can complement the earlier found common segment . This can result in a longer, more complete and higher quality segment of source material than could be gotten initially.
  • the iteration may be terminated by a threshold value for termination and whereby an acceptably long common segment of sufficient quality has been identified and stored in the final memory place for playing later on.
  • the invention gives the user unique new ways of continuously obtaining recordings of source material, such as music and film. If this invention is used for radio transmissions, the invention can continuously record all songs repeated on the radio and save them in a play list for later use. In addition to this, when the user of the devise hears a song he wants to record, the user only has to push a button to automatically get a full-length recording of that song.
  • the invention may distinguish between music, commercials and talk on the radio.
  • Fig. 1 schematically illustrates a procedure for creation of a search key of a section or a representation of a section of the music that is stored in a memory for comparison and matching against the same piece of music over for instance radio channels;
  • Fig. 2 illustrates an example of a procedure for recognition of the music by use of the stored search key;
  • Fig. 3 illustrates an example how a more complete piece of the music is created out of a repeated number of detections, comparisons and storage of substantially identical music sequences by continuous matching of search keys against pieces of music that are transmitted over for instance radio channels;
  • Fig. 4 exemplifies a procedure for creation of more search keys;
  • Fig. 5 shows an example of a procedure for creation of additional search keys after the matching and detecting with a first search key.
  • the searching and recording of hit songs in a radio transmission have been used as an illustrative example in this application. It is to be understood that the invention is not limited to identifying and recording hit songs; it may be used for films, music videos and other kinds of source material as well.
  • the searching and recording may be done by an iterative procedure comprising finding, comparing and storing of signal segments that are indicated by search keys derived from the source material that is to be recorded.
  • a user can by using the method and device, according to present invention, at any moment choose to record a source material that currently is transmitted over a media channel to a receiving member.
  • the user will also automatically have source materials recorded from the media channel.
  • the devise will automatically identify the beginning and end of the full source material or parts of the source material and save these sections for later use.
  • An example of a source material could be a hit song that is transmitted over a radio channel to a radio receiver.
  • the listener may after a while and without further manual effort obtain a high- quality full-length version of the hit song, stored in the device.
  • the user can at any time during the playing of the song initiate a recording of the full version of it by simply pressing a button.
  • the device may also automatically extract music in a radio transmission and record each song separately.
  • This invention gives the user of the invention at least two new unique ways of obtaining music. One way is pushing the button when hearing a desired song, and the other way is by having the devise automatically record songs in whole and save them in a play list.
  • Media signals such as radio transmissions and television transmissions, that are sent over media channels to a receiver organ, such as a radio, television, PC or similar equipment is temporally stored in one or more buffer memories.
  • the older stored media signal may continuously be replaced with the latest transmitted media signal of one or many channels.
  • the media signals are accessible to the user, who may activate the device.
  • buffer memories adjusted for, e.g., five days of temporary storage, it is possible to at a moments notice record complete source materials, as described in detail below. The recording is even possible when the user decides to record late in the transmission of the source material .
  • a section or a representation of the section of the media signal at that point in time may be selected as a search key.
  • the search key may also be a derivation of the full source material .
  • the devise may also save a sufficiently long section of the recorded media signal surrounding the search key; for hit songs a sufficient length could be 5 minutes before and after the time of activation. This procedure gives the user the whole transmission of the source material that was transmitted at that time.
  • the activation of the recording function may be done by pressing a button, turning a wheel or by activating a handle or any other member on the receiver. The activation may also be done automatically by the devise.
  • This automated activation may be triggered randomly, periodically, or may be triggered by some recognizable feature of the transmission.
  • this enables the devise to automatically construct lists of music that has been played on the radio.
  • the music may be stored much like on an ordinary CD player and gives the user a possibility of listening to one song after the next.
  • the necessary length of the recorded sections before and after the time of activation can be determined by estimating likely lengths of that type of source material. For hit songs, 5 minutes before and after the time of activation should be enough in most cases.
  • the media signal transmission of the source material stored in memory might not be free from undesirable signal components. In radio transmissions, for example, it is very common to interrupt the music with talk, at least in the beginning or at the end of a song. Sometimes, the disc jockey may even break in the middle of the playing of a song, although most of the time a piece of music is played on the radio, a large part of it is transmitted without any interruptions. Another problem is that it is not known where the source material starts and ends in the stored recording.
  • This invention provides a solution to how to find the beginning and end of a source material in a continuous media signal, e.g., the beginning and end of a song in a continuous radio transmission. If the device is automatically activated, it may continuously record music that is repeated on the radio and thus be able to automatically save songs from the radio.
  • Fig. 1 illustrates a procedure for creating a search key 100 of a section of a source material or a representation of that section.
  • the media signal 10 may, e.g., be a piece of music 12 that may contain undesirable signal components 102, 104 and other undesirable segments 103, 105 before and after the song 12.
  • the desired source material 12 is marked with a bold line in Fig. 1.
  • the segment 12 has a start 13 and an end 15.
  • the search key 100 may be used for detecting previous transmissions and future transmissions of the same source material, e.g., the same piece of music. The detection may be done through matching and comparison of the content of the search key with segments of the media signal stored in the buffer memory or being transmitted later on.
  • the detection of previous or future transmissions of the desired source material may be carried out by a direct match of the search key. It may also be carried out by a process of identifying sections of the transmission that may contain the source material and then checking these sections in one or many ways and in one or many steps to test if they actually are from the desired source material.
  • the media signal 10 is longer than the desired source material 12 to make sure the entire source material 12 is eventually recorded.
  • the media signal 10 should extend a time period before and after the search key that is long enough to accommodate the full source material. As an example, most popular pieces of music are shorter than 5 minutes and since the recording activation might take place any time during the play of that piece of music, it is desirable to save about 5 minutes before and 5 minutes after the time of activation to ensure that a whole piece of music is captured. In this way the media signal 10 may be about 10 minutes. Of course, any time period could be selected, as desired.
  • the iterative process of the present invention reduces the corrupted segments 102, 104 to a minimum by gradually replacing those segments with uncorrupted clean signal segments copied from other transmissions of the same source material that either have been transmitted in the past or will be transmitted in the future.
  • An important assumption of the present invention is that the receptions of the desired source material are substantially identical for every transmission of the same source material, e.g., the reception of a song is close to identical every time it is transmitted over the radio. While the undesirable signal segments such as talk, commercials and distortions usually are different each time the same song is played. Fig.
  • the media signal 20 has a shorter corrupted segment 202 in the beginning of the desired source material 22, that has a beginning 24 and an end 26.
  • the signal 20 has a relatively long corrupted segment 204 compared to the segment 104 of the signal 10.
  • the parts of the two media signals that are identical is the time between 107 and 109 and this may be saved as the common segment.
  • One object of the iterative process of the present invention is to take advantage of the relatively short distorted segment 202 but ignore the relatively long segment 204.
  • media signals are buffered, as mentioned above, on a continuous basis in the buffer memory.
  • the media signal 20 that is detected by recognizing that the search key 100 is identical, or close to identical, to the second instance 200 of that search key, can then be further tested for likeness by expanding the testing, possibly with other methods, beyond the area of the search keys.
  • segment 20 may be copied to a memory or its start and stop points in the memory are stored. This may be done by copying a sufficiently long segment before the second instance of the search key 200 and a sufficiently long signal segment after the second instance of the search key 200.
  • the device may save the media signal in its original place but not overwriting it for a predetermined time.
  • the identification of the search key and the saving of the media signal results in two media signals, i.e., the media signals 10, 20, being stored.
  • the media signal 20 is compared with the initially stored media signal 10.
  • the parts of the two media signals 10, 20 that are identical, or close to identical, are treated as if they are free from undesired signal components and therefore represent at least part of the desired source material. This could be, e.g., part of or a whole desired song, without any interfering talk or commercials.
  • a segment 106 of the signal 10 is identical to a segment 206 of the signal 20.
  • the common segment may be saved for later use, for example, to be listened to.
  • segment 106 may be stored in memory and be added to by future iterations until the entire desired source material 12 has been stored in the final memory or a threshold value for termination is reached.
  • the segment 106 of the source material 12 is, in this way, available for playing and the segment 106 has an identified end 109 and an identified beginning 107. Since only the portions of the media signals that are identical or substantially identical are identified, only a shorter section 106 of the desired source material 12 is likely to be identified the first time the section 106 is saved. If the user is lucky he or she may get the whole source material, e.g., a whole song, the first time the second instance of the search key is found.
  • the device only works through the process once.
  • the first found common segment that comprises a copy of the search key is used to identify the beginning and end of the source material. This process is described above in Figs. 1 and 2.
  • This simpler version of the invention may only give the user of the device the first identified common segment as the final version and thus giving the user a smaller chance of finding the whole source material .
  • the above-described procedure is repeated numerous times.
  • the steps of detecting media signals, storing the detected media signal in a memory and comparing the media signals to find matching common segment may continue.
  • One object is to detect more common segments by pairing identical media signals that supplement the previously identified signal segment 106 by adding the new matching section to the signal segment 106 stored in the final memory. This iteration leads to a longer and longer common segment 106 stored in the final memory.
  • Fig. 3 illustrates how an almost complete and non- corrupted source material 110 may grow out from the repetitive process of matching the search key 112 of the media signal 70, the search key 114 in the segment 703 of the media signal 80 and the search key 116 in the segment 705 of the media signal 90.
  • the media signal 70 contains the desirable source material 702 that has a beginning 704 and an end 706. It should be noted that the media signals 70, 80, 90 contain the same source material and the search keys 112, 114, 116 also are identical or close to identical.
  • a section 118 may be added to the common segment stored in the final memory because the section 120 of the signal 80 is identical to the section 122 of the signal 90.
  • a section 124 may be added to the common segment stored in the final memory because the section 126 of the media signal 90 is identical to the section 128 of the media signal 70. If the start point 130 and the end point 132 represent the start and the end of the common segment, the segment 110 almost covers the entire source material 702. The only missing segment is a segment 133 at the beginning 704 and a segment 135 at the end 706 of the signal segment 702. The procedure may continue the iteration in this manner until the entire source material has been recorded. To prevent the iterative search procedures, including the comparison and add-on procedures, to go on forever, a threshold value for termination may be set. This could be a predetermined number of iterative steps for the iterative search procedure.
  • the termination of iteration may also be triggered when the lengths of a number of added common segments are smaller than a certain value since this condition indicates that there might not be much more to be found of the full source material.
  • the iteration may also be set to stop if no additional common segment has been added despite a certain numbers of identifications of identical source material.
  • the common segment When a common segment is found the first time, the common segment may be stored in a final memory and be ready for being played by the user. This will give the user the option to repeatedly enjoy the common segment, e.g., repeatedly enjoy a song by connecting a music- reproducing device to the final memory. Each song may over time be added to with new parts of the song and thus giving the listener a longer and more complete version of the desired music.
  • the device works through the identification process as described above, as illustrated in Figs. 1 and 2, and works through the iteration process as described above and in Fig. 3, but instead of adding the common segments together, the devise only uses the longest possible identified part of the source material, the longest common segment, as the final version.
  • FIG. 4 illustrates an example of creating multiple search keys 300, 310, 320 in the media signal 30. This method is particularly useful when the media signal contains a substantial amount of undesirable signal components. The method increases the chances that at least one of the search keys 300, 310, 320, are free from undesirable signal components.
  • search key 310 is free from undesirable signal components and can later be matched with an identical search key when the source material 31 is found in the memory or retransmitted.
  • the search keys 300 and 320 are not likely to be matched in a later media signal because the undesirable signal components are not likely to be repeated exactly the same way in a later transmission.
  • the procedure may be designed to detect supplemental pairs of identical signal segments to complement the identified common segment by adding these additional common segments to the common segment in the memory.
  • This method improves the chances of finding and identifying a non-corrupted part of the desired source material in memory or next time the source material is transmitted. This also speeds up the process of finding and obtaining an acceptable length on the desired source material 31. The whole procedure may be repeated in the iterative steps as described above.
  • Fig. 5 shows a procedure for creating multiple search keys 500, 510, 520 in the segment 501 of media signal 50, after matching and detecting of a first one search key 400, of the media signal 40.
  • the procedure continuous with comparing the three search keys 510, 500 and 520 with the media signal, 60.
  • the search key 520 being substantially identical to the search key 620 and thus indicating a match between the segments.
  • the media signals 40, 50, 60 may contain the same source material but the three different media signals have different amounts of undesirable signal components, such as talk and commercials, interfering with the source material. This provides the opportunity to compare three stored versions 40, 50, 60 that contain at least parts of the same source material . Since there is a match between the search key 400 and the search key 500, a first common segment 402 may be saved in the final memory. The above iteration may then add common segments before and after the common segment 402 as other common segment are found by using the search keys.
  • the media signal 40 is assumed to at least in part originate from the same source material as the media signal 50. The difference is that both signals have a different amount of undesirable signal components.
  • An important feature is that because there is a match between the search key 520 and the search key 620, the media signals 40, 50 are assumed to have common parts with the media signal 60, and that these then originate from the same source material. This means that signal segment 602 of media 60 signal is substantially identical to segment 404 of media signal 40, and this common segment can then be added to the common segment in the final memory. The whole procedure may be repeated in the iterative steps as described above.
  • One object of the iteration method of the present invention is to in the final memory acquire a full-length version of the source material that does not have any undesirable signal segments, i.e., talk, commercials, distortions, etc.
  • the method identifies source material, such as hit songs on the radio, with the help of a search key that is a selected section of the source material or a representation of that section.
  • the search key may represent a very short section of a desired hit song or a representation of that section.
  • the desired source material may be recognized by identifying similarities between the search key and the media signal.
  • a correct match where the section under investigation is actually from the same time period of the same source material from which the search key was taken, may yield a more distinct pattern with a much higher value at the match than the surrounding wrong time periods, the longer the section that is involved in the correlation process.
  • longer sections also demand more processing power and therefore there is a practical limit to how long sections one can use .
  • Other methods can be used to determine similarities between sections of media signals.
  • the search key as for correlation, is a section of a media signal, which is then compared to other sections of media signals.
  • the search key and the section of media signal that are to be compared for likeness are first normalized in gain so that they have almost the same gain. Then the samples from one section are subtracted from the samples of the other section and the absolute values of these differences are summed up to get a final cancellation value. If the sections are exactly identical, the resultant value will be zero. In practical use, a correct match will yield a very low cancellation value. The method is called cancellation since the sections will cancel each other if they are identical, or near cancel each other if they are very similar.
  • modified correlation and modified cancellation These new devised methods can give very distinct results, when searching for a match and when searching for the beginning and end of source material, but the penalty from not using every sample in the process is that the average noise level away from a correct match can be higher than when all samples are involved.
  • the device may solve the problem of comparing media signals that are transmitted with different gain by normalizing their respective gain as part of the comparison process.
  • the normalization of gain could also be done as part of the process of recording the media signals.
  • the comparison method utilized to determine the degree of similarity between the search key and a media signal is the correlation method or any other method whose result is dependent on gain in the signal chain, then a method of compensation for gain variations could be applied to normalize the measurements.
  • One particular method of the present invention that has many advantages is to normalize the calculated similarity values with the sum of the absolute values of samples in the section of interest. This may effectively cancel the influence of variable signal gain, such as for example when a DJ plays the same song at two different occasions at different gain settings in the mixing console .
  • correlation or modified correlation When correlation or modified correlation is used as a method to determine the degree of similarity between a search key section and a section of a media signal, it can be of use to know in advance about how high the correlation value at a correct match is expected to be. Since media signals that are almost identical are reviewed, which is so because they originate from the same source material, it is possible to know in advance how the expected section at a correct match could look like. The correct match must be very similar to the search key section. Therefore, it is possible to in advance calculate the expected correlation value at a correct match by simply correlating the search key section with itself and normalizing the result with the aid of the moving average of the absolute values of samples of the search key section. This value has arbitrarily been called a T-value. When looking for correlation values that can be the result of possible correct matches, one search criterion could be that the correlation value is near the expected T-value.
  • T-value Another use for the T-value is when trying to determine the quality of recordings of the same source material. When several signal segments are found that have been determined to originate from the same source material, then it is possible to use the T-value to indicate something about their relative quality in regards to noise, interference and distortion. If instead of only calculating the T-value for a media signal at the correct match, the continuous T-value over part of or the whole section is calculated. This section may then be correlated with another section from the same source material and the resulting correlation values and corresponding T-values are compared. It must be noted here that the signal segments that are to be compared should be aligned in time and normalized in gain and that the number of samples in the calculation of the T-value should be the same as the number in the correlation.
  • the earlier calculated T- values should be exactly the same as the later calculated correlation values. Any departure from the expected T- value may be due to some kind of unwanted signal alteration since it is assumed that both sections originate from the same source material . The greater the departure from the expected T-values, the greater the difference between the sections is likely to be. It may also be assumed that if the correlation values are close to the T-values then two sections are of high quality since it is unlikely that similar random disturbances corrupt both sections.
  • sections can be compared to get an indication of their relative quality. With three sections, sections 1 and 2 may be compared, then 1 and 3, and finally 2 and 3.
  • This method of determining the quality of sections of media signals can be used to set a criterion for when a section will be accepted as good enough, and it can also be used to select sections of like quality. The latter can be important when pieces from different recordings of the same source material are spliced together to form a longer continuous section of the source material. It could be disturbing to the user to suddenly note a jump in quality when playing the spliced longer section.
  • the expected value at a match may be close to zero.
  • the degree of similarity determines how far from zero the cancellation value is.
  • Cancellation can be used to determine when sections are similar, and the method can also be used to determine the relative quality between sections when they have been determined to originate from the same source material . The more two sections from the same part of the same source material have been contaminated with noise and other disturbances, the more the cancellation values are expected to depart from zero although the sections are normalized in gain and correctly aligned in time.
  • the searching and matching of sections of media signals is performed only on a subset of the available data and/or a transformation of that data. This could be done in many ways. Either the device uses only a fraction of the samples building up the material when creating a search key. Another way is that the device may record the media signal in two or more separate files, one ore more search files and one or more files for later use, e.g., for playing.
  • a search file may be a recording of the media signal but of lower bandwidth, or might be a file that only contains certain frequency intervals.
  • a search file may also be a representation of the recorded media signal . The search file can be used to create the search key and also to search for a second incident of the search key.
  • the search file may also be used to find the beginning and end of the source material.
  • a search file could be a separate recording of the media signal at a lower sample rate, e.g., 6 kHz.
  • This search file can be used to create the search key as well as to find another incident of the search key and also for finding the beginning and the end of the source material. Then this start and stop information can be used to find the start and stop of the source material in the full-quality recording.
  • One reason to use separate search files is to decrease the need for processing power.
  • the device creates a search key and searches for it in files stored on a hard drive.
  • the devise may create a plurality of search keys continuously as the media signal is transmitted and searches simultaneously for many search keys. Since the search may be done completely in the RAM memory of the device this decreases the need for accessing information from an eventual hard drive and thus saves power for the devise. For example, by loading one hour of music or search file into RAM memory from the hard drive or the transmission, and searching the RAM memory with many search keys, the hard drive is given a rest and thus the device may save battery power and also work faster.
  • the device may perform the searching and matching of signal sections in a hierarchical way, first selecting out a number of possible matches, and then using a more precise method to find the correct matches among the possible ones. For example, one way of doing this could be to first calculate the correlation between the search key and the media signal, identifying the sections of media signals that have a high enough correlation with the search key and after this is done test the identified sections in another more precise way. This other way could be using a larger search key or some completely different method.
  • the search key used to find copies of source material can be composed in different ways. In one way of performing the invention, the used search keys are short, such as 0.1-2 second long sections of the media signal.
  • the search key might be a representation of a section, for instance by applying a mathematical transformation to that section or by extracting some describing characteristics.
  • the search keys are much longer and can also be used in combination with compression or using programs or algorithms to, for example, describe a media signal.
  • the different types of search keys can also be used in combinations to better find the desired media signal. Instead of only using samples, the instantaneous amplitude values, of the media signal in the comparison process, it may be possible to index the music so that a short signal segment may be stored where the segment has some features that distinguishes that segment from other music.
  • a song may have a unique drum segment and only a portion of the drum segment may be stored and compared to other media signals until the same drum segment is located. Any time this drum segment is played again, the segment is stored in an indexed memory so that it is not necessary to search the entire memory but only the indexed portion of the memory.
  • the drum segment may be transformed by a mathematical algorithm in a way to reduce the necessary storage requirements or to facilitate matching.
  • the steps of searching for and comparing the stored search keys with current media signals or recorded transmissions may be done by continuously searching for certain frequencies.
  • the search key may not include the whole frequency register, but only certain predetermined frequencies.
  • the search key When used for music in a radio transmission, the search key may only contain the frequencies 30-31 Hz and 13000-13100 Hz.
  • the 30-31 Hz signal may be used to identify identical drum-sounds in a song of certain lengths at certain time intervals.
  • the 13000-13100 Hz signal may be used to identify identical guitar sounds at certain time intervals and lengths.
  • the search procedure may therefore be done by only searching for 30-31 Hz signals of a radio transmission. When a matching signature on the 30-31 Hz frequencies is found in the memory, then the 13000-13100 Hz frequencies are searched and compared. If the media signal has the same guitar sound at the 13000- 13100 Hz frequencies, then it is assumed to be the same media signal .
  • the beginning and the end of a source material may be found by comparing a few frequencies.
  • the signal segments that are compared are considered to be identical as long as the compared frequencies of the signals segments are substantially identical.
  • the search process may search for embedded codes in the media signal that identifies the transmitted source materials. For example, in digital radio transmission there are possibilities to send codes to identify music that is currently playing. Some CD's contain code that identifies artist and song for each track. This coded information may be used to find the desired song. This information may then be utilized by a procedure for finding the copy of the song and to locate the beginning and end of it and to cut out undesirable signal components.
  • the memory capacity of the receiving organ must be at least 2-3 hours of stored transmission. For music in standard MP3 format, this is about 100-200 MB of stored music.
  • the memory could also be much larger to be able to, e.g., contain many different media channels over a much longer time period.
  • the memory could also contain previous recordings of source material that the device has found.
  • the search process may either be triggered by the user when he notices a source material that the he would like to have recorded, or by the device itself.
  • the device When the device is not occupied with a manually triggered search request, it can automatically create search keys and conduct searches to build common-segments libraries or lists stored in memory. These lists of common segments that have been repeated in the media signals can be used for future searches or for playing later on by the user.
  • This automatic searching is particularly useful when a radio station is only playing a limited number of songs, such as a top 40s radio station. For stations that have a greater variety of music a larger buffer memory needs to be searched to find the songs that are repeated, but as soon as a song is repeated the devise will identify it and save it.
  • the device may already have conducted several iterations for a long time period so that the entire song may be available to the listener without having to wait for all the iterations to be completed.
  • the search may be much faster, since the desired source material may already earlier have been identified and saved by the devise.
  • the device tests the search key to make sure that it contains sufficient information to be of use. For example, if the device itself has generated a search key automatically, it will not be of any good use if it is in the middle of a silent part of the transmission. This can also happen when the search request is triggered manually.
  • the search key can be made as unique as possible. This may lead to a greater chance of finding a match of the search key.
  • One method of improving the quality of the search key is to test several possible search keys near the time of activation, and select the one that is deemed to be most unique in the sense that it will be of best use to find the desired matching signal segment.
  • Another method of improvement the quality of the search key when the search key is triggered at a silent moment of the transmission, is to move the taking of the search key to the moment before or the moment after the silence. This enables the device to get a search key that contains more information.
  • a search key When a search key has been compared to another section of a media signal and the likelihood of them being from the same part of the same source material is high as indicated by some set criterion, then a second step of the identification process can take place. If this actually is a correct match, then it can be assumed that by moving some time before and after the time of the match in both sections and performing a new comparison, then it is likely that the signals still are very similar and thus still from the same source material. At some point in the sections, the likeness will be lower than a certain level, and it can be assumed that an endpoint has been reached of the parts of the sections that are similar. In a similar way the other endpoint may be searched for.
  • the searching for endpoints can be performed in many ways.
  • the sections may be tested by continuously moving the test along the sections until the lowest likeness level is reached that is deemed to be acceptable, and this is determined to be an endpoint. It is also possible to jump a certain time away from last comparison point and test again, and if still deemed to be sufficiently similar iterate this jumping and testing until the likeness level is below a certain point.
  • the step size could then be reduced and the jump direction reversed. This new point is tested and the step size reduced again.
  • the new step direction is changed if the sections are now deemed sufficiently similar, or unchanged if they are deemed not to be sufficiently similar.
  • the iteration process is continued until a predetermined smallest step size is reached, and this is point is taken as an endpoint.
  • the other endpoint can be gotten in the same way.
  • M in the latter section is compared to a same-length part of the other section around t JUMP .
  • M is then decreased and the process is iterated until M has reached a certain value, often -M, where the process is terminated.
  • the first method is to calculate a degree of similarity according to some method, and if the value is better than some set level, then it is likely that it is a correct match.
  • the second method that further assures that the sections are from the same source material in this point is to note how close to the theoretical point in time that the actual maximum similarity is achieved. As an example, we may assume that the comparison process is started 1000 samples before the expected point and continues until 1000 samples after this point and that it has earlier been determined that a correct match must appear within 10 samples before or after the theoretical point. It is now possible to calculate all 2000 possible comparisons and note at which point the best value was obtained.
  • the method also includes a counter that counts the number of times the same source material is detected, either in part or full. One may also count the number of times a second instance of the search key is identified.
  • a counter that counts the number of times the same source material is detected, either in part or full. One may also count the number of times a second instance of the search key is identified.
  • One application of this is that the more times a song has been played, the higher the likelihood that the quality of the final obtained recording of the song is high and that almost the entire song is recorded.
  • the counting may also be used to generate source material lists that are arranged according to how many times a source material has been played during a certain time period in one or more media channels.
  • the method can be used to create a list of last weeks most played music on a certain radio station or stations and may rank that music according to how often it has been played.
  • the method may also generate lists based on the selection and preferences of the user.
  • the user identifies a source material when it is played, activates the device and the source material may automatically be saved in the list of the listener's choice.
  • This may be one list or a plurality of lists for different source material styles or users; for radio, e.g., a list of Hard Rock, one list of Pop Music and a third list that a friend of the main user of the devise has created.
  • the user may also categorize media channels so that source material played on the same format media channels are saved in the same lists or libraries.
  • a library might contain hard rock, which is from radio stations that the user knows plays that type of music, and another library is for soft music from that type of radio stations, and so on.
  • the device may also in one version of performing the invention, identify when a source material is played less frequently and remove such a source material from the list. For example, if the time period between each time the source material is played exceeds a specified time, the source material may be considered to be less popular and thus removed from the top list.
  • the method may remove certain undesirable signal components, such as commercials. For example, the method may remove common segments that are shorter than a certain time period, such as thirty seconds or one minute, because most commercials are shorter than desired source material.
  • the device may recognize the undesired signal components and save them in a separate list.
  • the method may also remove signal segments that are found being identical over a longer period of time. This is done to remove recordings of total programs that are retransmitted. If, e.g., a radio transmission is identical to another transmission for more than five to ten minutes, it is probably not one song, but instead a retransmission of a full program and thus not of interest to the user who wants to record separate songs. These time parameters may be adjustable to the user so that he might use the device to record both separate source material and collections of source material. In one version of the present invention it may also be possible for the device to generate lists of material that the user prefers not to be exposed to. This could be done, e.g., by the user pressing an activation button when undesirable material is played.
  • this list could include commercials, talk, jingles, etc. These signal segments may then be stored in an undesired-list that then can be used to screen out these segments from the list of desired material.
  • the user can also mark source material in the desired-list as undesired and thus prevent them from further being played or presented to the user.
  • the user is not exposed to the direct transmission but a slightly delayed version so that the devise may have time to remove any undesirable signal components before they reach the user and fill these gaps with desired content.
  • This may be done by automatically searching the transmission for undesired signal components and changing the delay when an undesired signal component is detected to jump over it. This can eventually create gaps big enough to be filled from, e.g., earlier recorded desired material, and when playing of them is over, the source can be switched back to the earlier program.
  • the device may also automatically change the media channel, such as radio station, when certain conditions are met. For example, the device may change the radio station after a certain time period such as every five minutes or every 24 hours.
  • the change to a new media channel may extend the number of pieces of source material that can be found.
  • the device may also be programmed to find a predefined number of source material, such as twenty, on one media channel and then switch media channel and find a predefined number of different source material on a second media channel.
  • the device may also change media channel when the device cannot find any new source material after a certain time period such as when the device has not found a new source material in forty-eight hours.
  • the device may also switch media channel if no recognizable media signals can be found, such as when there is something wrong with the transmission or the transmitter is inactive.
  • the device may also store signals from many media channels in a buffer memory. Searching many media channels can increase the chances of eventually obtaining the entire desirable source material, e.g., an entire song.
  • the device can restart the iteration process to achieve higher quality recordings of source material.
  • a too short piece of the desired song can have been gotten or it can have lower quality than desired.
  • the device, or the user using an activation member might in that case start a process of getting a new search key from the common segments of the source material already recorded which will then lead to a new search for the desired source material in memory or in transmissions.
  • the devise will connect to an external system for naming of the desired source material. This could be done by the device transmitting a part of the desired source material, or a search key from the desired source material, to the external system and getting a reply which identifies the source material. If the method is used on music in a radio transmission, the devise will connect to the system and send a piece of the recorded music for identification.
  • the identification system may send the title of the music, the artist or group to the device, in return. This may make it possible for the user to not only listen to the music but also get the title and to know what artist or group that is playing. This identification could be done automatically or being triggered by the user.
  • the quality, i.e., the nearness to the source material, of recorded media sections from the same part of the same source material can be improved by utilizing more than one recording of the same source material.
  • undesirable signal components may be removed by replacing a section with undesirable signal components with a corresponding section from the other two media signals that are identical and therefore considered free from undesirable signal components. More particularly, if a certain section of the first media signal has a low similarity to the same section of the second media signal but there is a high similarity between the second section and the third section, then the method may be designed to replace the section of the first media signal with the corresponding section of the second or third media signal .
  • the search key may operate in a similar way in that the search key will only identify segments that are higher than a certain predetermined value of similarity. If the value of similarity is set too high, then there is a risk that segments that do originate from the same source material may be missed out by the search key. If the value of similarity is set too low, then the wrong signal segment, or poorly transmitted signal segment from the correct source material, may be selected.
  • the device may also be set to select the segments that have an equal value of similarity instead of merely maximizing the sound quality to avoid certain sound sections from being extremely clear while others are not so clear.
  • an entire song may have a small acceptable and evenly distributed level of distortion.
  • One method used in one version of the invention to increase the quality of the media signal is to add time-aligned recordings from the same source material together sample by sample, and dividing the resulting amplitude values by the number of recordings taking part in the addition process.
  • the desired signal information may not be affected since it will be the same in all recordings. Undesirable signal components, such as noise and distortion, will not be unaffected in the same way as the wanted signal information.
  • Noise and other similar types of unwanted information can be regarded as more or less random in nature, and therefore the average noise level may not double when two signals with the same average noise levels are added together.
  • the resultant noise level only increases by the square root of the number of noise signals added together if they have the same average noise levels.
  • the average noise level may be decreased below that of the original recordings .
  • noise levels in recordings of the same source material differ more than a certain level, then it is actually better to just select the best recording and not trying to improve the quality by adding the recordings together.
  • Other types of unwanted signal information than noise and similar signals can also be decreased with this method.
  • the sections of source material originate from radio transmissions or from other disturbance prone transmission channels, then a possible quality indication can be gotten from the signal strength in the receiver.
  • a weaker reception will generally be more noisy and distorted.
  • Other parameters of the received signal can also be measured and be used to give a quality indication of the obtained source material.
  • the iteration method of the present invention adds new undisturbed source material segments to a source material segment that is stored in a memory.
  • the device may try to match two segments that are to be spliced together by conducting a mathematical calculation of the similarity of the two segments so that, for example, the end of the first segment is precisely matched with the beginning of the second segment resulting in the two segments are placed exactly right in time.
  • the device may test different overlapping and when the similarity is the highest, the device merges the two segments together, so the user might not notice that a first segment has been added to a second segment.
  • the device automatically checks if a signal segment is transmitted with inverted phase.
  • the signal segment with inverted phase may have a negative similarity or correlation to a signal segment that is played with opposite phase although they originate from the same part of the same source material .
  • the device may check both the positive and the negative similarity of the search key to be able to use the inverted phase signal segment.
  • the device may automatically adjust for this by changing the phase of one of the media signals before merging the two media signals together.
  • Two sections that are to be merged together might not have their sampling points aligned so that when merged there may be a discontinuity at the meeting point in the final merged section.
  • To make the transition between two sections that are to be merged together as smooth as possible one may gradually over a limited time near the meeting point mathematically stretch out or compress the signal of one of the sections, or both, so that the merging between the two sections can take place without discontinuity.
  • Another way of solving this problem of discontinuity would be to mathematically shift the sampling points of one, or both, of the sections in a way that the transition will exhibit no discontinuity.
  • Media signals can be radio transmissions, television transmissions, transmissions over computer networks, computer files, on the devise already stored files or equal .
  • Media channels can be radio and television networks, a mobile telephone network, a computer network or equivalent .
  • a receiving member can be a radio apparatus, a television apparatus, a VCR, a personal computer, a mobile phone or other apparatuses for receiving media signals .
  • An activating member may be a button, leverage, computer-program, algorithm, steering wheel or equaling member. It may also be voice controlled, infrared or a blue-tooth connection, a wireless connection, or combinations thereof. All the above members may be used, as well as programmed, automated or time controlled activation members .
  • Undesirable signal components in the transmissions may be a speech from a radio talker, a DJ, VJ, television person, a reader or news or equivalent. Undesirable signal components in the transmission may also be caused by, for example, the transmission being weak or by any other reason for an interrupted or disturbed transmission.
  • Source material can be a piece of music, a movie, a commercial, a TV-program, news, a speech, sound effects, film effects or similar.
  • a detecting member can be made out of an LP filter, HP filter, BP filter, BS filter or active and digital filter constructions for frequency filtering or a computer program, a processor or an algorithm.
  • An iteration member may, for example, be a computer program or an algorithm.
  • the final memory may be an internal memory in the media signal player.
  • the final memory may also be a CD-R, mini-disc, floppy disk, hard disk drive, cassette recorder, multimedia card, compact flash card or other external or internal memory or a combination of the above.
  • the final memory may also be part of an external or internal memory or a part of the buffer memory.
  • a playing member may be a CD-player, minidisk- player, cassette deck, a stereo-equipment, a radio, a television, a VCR, a MP3 player a PC, a PDA or any other device for media playing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management Or Editing Of Information On Record Carriers (AREA)
  • Indexing, Searching, Synchronizing, And The Amount Of Synchronization Travel Of Record Carriers (AREA)

Abstract

La présente invention concerne un procédé et un système qui permettent de localiser et d'enregistrer des séquences (10, 20, 30, 40, 50, 60, 70, 80, 90) de signal à limitation temporelle dans des canaux multimédia qui peuvent contenir des composantes indésirables (102, 103, 104, 105, 202, 204) de signal, par exemple pour effectuer l'enregistrement de musique dans des transmissions radio. Les signaux sont continuellement mis en mémoire tampon. L'utilisateur identifie une matière source désirée de laquelle une partie peut être prélevée pour faire office de clé de recherche (100, 112, 400). Le dispositif peut également sélectionner des clés, automatiquement. Si une deuxième instance de la clé de recherche (114, 200, 500, 510, 520) est détectée, des séquences de signal, qui sont reliées en temps aux clés de recherche, sont comparées. Les séquences de signal qui, suite à une comparaison, sont sensiblement identiques, sont identifiées comme appartenant à la même matière source désirée. L'étape suivante est une itération des résultats de la procédure précédente dans un segment de matière source plus long et de meilleure qualité que le segment commun initial.
EP02707866A 2001-02-23 2002-02-21 Procédé de réception d'un signal multimédia Expired - Lifetime EP1417583B1 (fr)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
SE0100642A SE0100642D0 (sv) 2001-02-23 2001-02-23 Förfarande och anordning
SE0100642 2001-02-26
US27490401P 2001-03-09 2001-03-09
US274904P 2001-03-09
US47532 2001-10-23
US10/047,532 US7062442B2 (en) 2001-02-23 2001-10-23 Method and arrangement for search and recording of media signals
PCT/US2002/005537 WO2002069148A1 (fr) 2001-02-26 2002-02-21 Procede et dispositif permettant de rechercher et d'enregistrer des signaux multimedia

Publications (3)

Publication Number Publication Date
EP1417583A1 true EP1417583A1 (fr) 2004-05-12
EP1417583A4 EP1417583A4 (fr) 2006-02-01
EP1417583B1 EP1417583B1 (fr) 2006-10-11

Family

ID=56290253

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02707866A Expired - Lifetime EP1417583B1 (fr) 2001-02-23 2002-02-21 Procédé de réception d'un signal multimédia

Country Status (5)

Country Link
EP (1) EP1417583B1 (fr)
CN (1) CN100399296C (fr)
AT (1) ATE342562T1 (fr)
BR (1) BR0207553A (fr)
DE (1) DE60215357T2 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104778218A (zh) * 2015-03-20 2015-07-15 广东欧珀移动通信有限公司 一种不完整歌曲处理的方法及装置
CN111554329A (zh) * 2020-04-08 2020-08-18 咪咕音乐有限公司 音频剪辑方法、服务器及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0283570A2 (fr) * 1984-04-26 1988-09-28 A.C. Nielsen Company Système d'identification d'un signal
WO1999001862A1 (fr) * 1997-07-04 1999-01-14 Sextant Avionique Procede de recherche d'un modele de bruit dans des signaux sonores bruites
DE19810114A1 (de) * 1998-03-09 1999-09-16 Stefan Weigelt Suchverfahren für Musikstücke, insbesondere für Rundfunkempfangsgeräte

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3353381B2 (ja) * 1993-04-23 2002-12-03 ソニー株式会社 記録再生装置
KR0138464B1 (ko) * 1994-06-29 1998-06-01 배순훈 화음데이터가 기록된 영상가요매체와 그 재생장치
US5739451A (en) * 1996-12-27 1998-04-14 Franklin Electronic Publishers, Incorporated Hand held electronic music encyclopedia with text and note structure search
US6185527B1 (en) * 1999-01-19 2001-02-06 International Business Machines Corporation System and method for automatic audio content analysis for word spotting, indexing, classification and retrieval

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0283570A2 (fr) * 1984-04-26 1988-09-28 A.C. Nielsen Company Système d'identification d'un signal
WO1999001862A1 (fr) * 1997-07-04 1999-01-14 Sextant Avionique Procede de recherche d'un modele de bruit dans des signaux sonores bruites
DE19810114A1 (de) * 1998-03-09 1999-09-16 Stefan Weigelt Suchverfahren für Musikstücke, insbesondere für Rundfunkempfangsgeräte

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KOSUGI N ET AL: "Music retrieval by humming-using similarity retrieval over high dimensional feature vector space" COMMUNICATIONS, COMPUTERS AND SIGNAL PROCESSING, 1999 IEEE PACIFIC RIM CONFERENCE ON VICTORIA, BC, CANADA 22-24 AUG. 1999, PISCATAWAY, NJ, USA,IEEE, US, 22 August 1999 (1999-08-22), pages 404-407, XP010356604 ISBN: 0-7803-5582-2 *
See also references of WO02069148A1 *

Also Published As

Publication number Publication date
CN100399296C (zh) 2008-07-02
DE60215357D1 (de) 2006-11-23
BR0207553A (pt) 2004-09-14
EP1417583B1 (fr) 2006-10-11
EP1417583A4 (fr) 2006-02-01
ATE342562T1 (de) 2006-11-15
DE60215357T2 (de) 2007-08-23
CN1493029A (zh) 2004-04-28

Similar Documents

Publication Publication Date Title
US7062442B2 (en) Method and arrangement for search and recording of media signals
Haitsma et al. A highly robust audio fingerprinting system.
Haitsma et al. A highly robust audio fingerprinting system with an efficient search strategy
US6604072B2 (en) Feature-based audio content identification
US6748360B2 (en) System for selling a product utilizing audio content identification
US6574594B2 (en) System for monitoring broadcast audio content
JP4658598B2 (ja) ストリームに埋め込まれた反復オブジェクトに対するユーザ制御を提供するシステムおよび方法
US7031921B2 (en) System for monitoring audio content available over a network
US7453038B2 (en) Musical piece extraction program, apparatus, and method
US20060041753A1 (en) Fingerprint extraction
JP2007534995A (ja) 音声信号を分類する方法及びシステム
CN100545834C (zh) 基于特征的音频内容识别的方法和装置
US20050229204A1 (en) Signal processing method and arragement
EP1417583B1 (fr) Procédé de réception d'un signal multimédia
US10819884B2 (en) Method and device for processing multimedia data
JP4056057B2 (ja) メディア信号の検索及び記録方法及び装置
KR100798524B1 (ko) 매체 신호 검색 및 레코딩 방법 및 장치
US20030084459A1 (en) Method and apparatus for modifying a media database with broadcast media
CN101442645A (zh) 记录/回放设备和方法、程序及记录介质
Haitsma Audio Fingerprinting
Haitsma et al. A New Technology To Identify Music
JP2009053297A (ja) 楽曲記録装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20031002

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: POPCATCHER AB

Owner name: POPCATCHER INC.

A4 Supplementary search report drawn up and despatched

Effective date: 20051221

RIC1 Information provided on ipc code assigned before grant

Ipc: G10H 1/00 20060101ALI20051215BHEP

Ipc: G06F 17/30 20060101ALI20051215BHEP

Ipc: H04H 9/00 20060101ALI20051215BHEP

Ipc: G10L 11/00 20060101AFI20051215BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 11/02 20060101AFI20060227BHEP

Ipc: H04H 9/00 20060101ALI20060227BHEP

Ipc: G06F 17/30 20060101ALI20060227BHEP

Ipc: G10H 1/00 20060101ALI20060227BHEP

RTI1 Title (correction)

Free format text: METHOD FOR RECEIVING A MEDIA SIGNAL

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: POPCATCHER AB

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60215357

Country of ref document: DE

Date of ref document: 20061123

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070111

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070111

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070122

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: BOVARD AG PATENTANWAELTE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070319

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070712

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070112

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070221

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061011

REG Reference to a national code

Ref country code: CH

Ref legal event code: PFA

Owner name: POPCATCHER AB

Free format text: POPCATCHER AB#SOEDERMAELARSTRAND 27B, 6 TR#11825 STOCKHOLM (SE) -TRANSFER TO- POPCATCHER AB#SOEDERMAELARSTRAND 27B, 6 TR#11825 STOCKHOLM (SE)

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20140722

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20150227

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20150302

Year of fee payment: 14

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150228

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150228

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20160831

Year of fee payment: 15

Ref country code: GB

Payment date: 20160831

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20161028

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160229

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60215357

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170901

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170221