EP1763017B1 - Toncodiereinrichtung und toncodierverfahren - Google Patents

Toncodiereinrichtung und toncodierverfahren Download PDF

Info

Publication number
EP1763017B1
EP1763017B1 EP05765807A EP05765807A EP1763017B1 EP 1763017 B1 EP1763017 B1 EP 1763017B1 EP 05765807 A EP05765807 A EP 05765807A EP 05765807 A EP05765807 A EP 05765807A EP 1763017 B1 EP1763017 B1 EP 1763017B1
Authority
EP
European Patent Office
Prior art keywords
section
encoding
encoded code
speech
predictive
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP05765807A
Other languages
English (en)
French (fr)
Other versions
EP1763017A1 (de
EP1763017A4 (de
Inventor
Masahiro c/o Matsushita El. Ind. Co. Ltd OSHIKIRI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Corp
Original Assignee
Panasonic Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Corp filed Critical Panasonic Corp
Publication of EP1763017A1 publication Critical patent/EP1763017A1/de
Publication of EP1763017A4 publication Critical patent/EP1763017A4/de
Application granted granted Critical
Publication of EP1763017B1 publication Critical patent/EP1763017B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/018—Audio watermarking, i.e. embedding inaudible data in the audio signal
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques

Definitions

  • the present invention relates to a speech encoding apparatus and speech encoding method.
  • VoIP Voice over IP
  • IP Internet Protocol
  • patent document 1 or non-patent document 1 speech encoding methods of embedding additional information in an encoded code using the steganographic technology are disclosed. For example, even if the least significant bit of the encoded code is changed to some extent, a person cannot auditorily perceive the difference. In order to add new information at a transmission apparatus, bits indicating additional information are embedded in the least significant bit of speech data that does not cause auditory problems, and this data is transmitted. According to this technology, even if the encoding apparatus is provided with some kind of an extension function, and information about this extension function is embedded in the original encoded code as an extension code and transmitted, there is no case where the decoding apparatus cannot perform decoding. Namely, it is possible to interpret this encoded code and generate a decoding signal at the decoding apparatus that is not compatible with the extension function as well as at the decoding apparatus compatible with the extension function.
  • a time-correlated signal such as a speech signal
  • a prediction by predicting an amplitude value of a sample for an encoding target from amplitude values of past samples and using predictive encoding that carries out encoding after eliminating time redundancy, it is possible to implement a lower bit rate.
  • the amplitude value of the sample for the encoding target is estimated by multiplying the amplitude values of past samples by specific coefficients. If the residual in which a prediction value is subtracted from the amplitude value for the encoding target, is quantized, it is possible to perform encoding with a less code amount than direct quantization of the amplitude value of the sample for the encoding target and achieve a low bit rate.
  • coefficients for multiplying the amplitude values of the past samples there are, for example, LPC (Linear Predictive Coding) coefficients.
  • the used codec is an ITU-T recommended G.711.
  • This G.711 is an encoding method for directly quantizing the amplitude value of the sample, and the above-described predictive encoding is not carried out.
  • the following problems occur.
  • the predictive encoding is a part of encoding processing, and therefore is carried out within an encoding section.
  • An extension code is embedded in the encoded code generated by the encoding section and is outputted from the speech encoding apparatus.
  • predictive encoding is carried out on the encoded code in which the extension code has already been embedded and the speech signal is then decoded.
  • the target of predictive encoding is that the code before embedding the extension code.
  • the target is the code after embedding the extension code.
  • the present invention it is possible to prevent deterioration in quality of the decoded signal even when a combination of the steganographic technology and the predictive encoding is applied to speech encoding.
  • FIG.1 is a block diagram showing the main configuration of the packet transmission apparatus provided with speech encoding apparatus 100 according to Embodiment 1 of the present invention.
  • speech encoding apparatus 100 carries out speech encoding using anADPCM (Adaptive Differential Pulse Code Modulation) scheme.
  • ADPCM Adaptive Differential Pulse Code Modulation
  • an encoding efficiency is enhanced by achieving adaptation using backward prediction at a predictive section and an adaptive section.
  • G.726 that is an ITU-T standard specification is a speech encoding method based on the ADPCM scheme. It is possible to encode a narrow band signal at 16 to 40 kbit/s, and achieve a lower bit rate than G.711 that does not use prediction.
  • G.722 is an encoding method based on the ADPCM scheme, and is capable of encoding the wide band signal at a bit rate of 48 to 64 bit/s.
  • the packet transmission apparatus has A/D converting section 101, encoding section 102, function extension encoding section 103, bit embedding section 104, packetizing section 105 and synchronization information generating section 106, and each section operates as follows.
  • A/D converting section 101 converts an input speech signal to digital, and outputs digital speech signal X to encoding section 102 and function extension encoding section 103.
  • Encoding section 102 decides encoded code I so that quantization distortion between digital speech signal X and the decoded signal generated by the decoding apparatus becomes minimum, or so that the distortion is difficult for a person to perceive auditorily, and outputs the result to bit embedding section 104.
  • function extension encoding section 103 generates encoded code J of information necessary for the function extension of speech encoding apparatus 100, and outputs the code to bit embedding section 104.
  • extension function for example, frequency band is extended from narrow band (frequency band of 0.3 to 3.4 kHz, that is, signal frequency band used in a typical telephone line) to wide band (frequency band of 0.05 to 7 kHz, in which naturalness and clarity increase more than the narrow band), or error compensation is carried out using the next packet even when a current packet is dropped (lost) at the decoding apparatus, and compensation information is generated so that deterioration in quality is suppressed to a minimum.
  • narrow band frequency band of 0.3 to 3.4 kHz, that is, signal frequency band used in a typical telephone line
  • wide band frequency band of 0.05 to 7 kHz, in which naturalness and clarity increase more than the narrow band
  • Bit embedding section 104 embeds information of encoded code J obtained from function extension encoding section 103 in bits of part of encoded code I obtained from encoding section 102, and outputs encoded code I' obtained as a result to packetizing section 105.
  • Packetizing section 104 packetizes encoded code I', and, for example, in the case of VoIP, packets are transmitted to the communicating party via an IP network.
  • Synchronization information generating section 106 generates synchronization information as described later based on encoded code I' after bits are embedded, and outputs the information to encoding section 102.
  • Encoding section 102 updates an internal state etc. based on this synchronization information, and encodes next digital speech signal X.
  • Encoding section 102 adopts G.726, and, when extension code J is embedded in the LSB (Least Significant Bit) of encoded code I, it is possible to embed extension code J at a bit rate of 8 kbit/s.
  • the procedure of speech encoding processing according to this embodiment is arranged as follows.
  • an internal state of predictive section 132, prediction coefficients used at predictive section 132, and a quantization code of one sample previous used at adaptive section 133 are supplied from synchronization information generating section 106 to encoding section 102.
  • encoding processing is carried out at encoding section 102, and information about an extension function is encoded at function extension encoding section 103.
  • encoded code I' is generated at bit embedding section 104, outputted, and provided to synchronization information generating section 106.
  • Synchronization information generating section 106 updates the internal state of predictive section 132, prediction coefficients used at predictive section 132, and the quantization code of one sample previous used at adaptive section 133, and supplies the results to encoding section 102, and encoding section 102 is prepared for next input digital signal X.
  • FIG.2 is a block diagram showing the main configuration within encoding section 102.
  • Synchronization information is supplied from synchronization information generating section 106 shown in FIG.1 to update section 111.
  • Update section 111 then updates the prediction coefficients used at predictive section 115, the internal state of predictive section 115, and the quantization code of one sample previous used at adaptive section 113.
  • the processing after encoding section 102 is carried out using updated adaptive section 113 and predictive section 115.
  • Digital speech signal X is supplied to encoding section 102 and inputted to subtraction section 116.
  • Subtraction section 116 then subtracts the output of predictive section 115 from digital speech signal X and supplies this error signal to quantizing section 112.
  • Quantizing section 112 then quantizes the error signal using a quantization step size decided using the quantization code of one sample previous, outputs this encoded code I, and supplies this to adaptive section 113 and inverse quantization section 114.
  • Inverse quantization section 114 decodes the error signal after quantization in accordance with the quantization step size supplied from adaptive section 113, and provides this signal to predictive section 115.
  • adaptive section 113 Based on an amplitude value of the error signal indicated in the quantization code of one sample previous, adaptive section 113 enlarges a quantization step width in the case where the amplitude value is large, and reduces the quantization step width in the case where the amplitude value is small. Predictive section 115 then carries out prediction in accordance with the following equation (1) using the error signal after quantization and a prediction value of the input signal.
  • y(n) is a prediction value of the input signal of annth sample
  • u (n) is an error signal after quantization of an nth sample
  • a(i) is an AR prediction coefficient
  • b (i) is a prediction coefficient
  • Land M are numbers of AR prediction and MA prediction, respectively.
  • a(i) andb(i) are sequentially updated by adaptation using backward prediction.
  • FIG.3 is a block diagram showing the main configuration within bit embedding section 104.
  • Bit mask section 121 masks a predetermined bit position of inputted encoded code I and always sets a value of the bit of this position to zero.
  • Embedding section 122 embeds information for extension code J in this bit position of the masked encoded code, replaces the value of the bit of this position with extension code J, and outputs encoded code I' after embedding.
  • FIG.4 shows an example of a bit configuration of a signal inputted and outputted from bit embedding section 104. Further, MSB is an abbreviation of Most Significant Bit.
  • the extension code is embedded in the LSB, but this is by no means limiting.
  • the extension code is embedded every one sample, it is possible to embed additional information for a bit rate of 4 kbit/s.
  • the bit rate for additional information is 16 kbit/s. It is possible to set the bit rate of the additional information with a comparatively great flexibility. Further, it is possible to adaptively change the number of embedded bits according to the properties of the inputted speech signal. In this case, information about the number of embedded bits is separately reported to the decoding apparatus.
  • FIG.5 is a block diagram showing the main configuration within synchronization information generating section 106.
  • Synchronization information generating section 106 carries out decoding processing as follows using encoded code I' that is the output of bit embedding section 104.
  • the residual signal after quantization is decoded at inverse quantization section 131 using quantization step information provided from adaptive section 133 and is supplied to predictive section 132.
  • predictive section 132 the internal state and prediction coefficients shown in equation (1) are updated using the residual signal after quantization and the signal outputted in processing for the previous time of predictive section 132 in accordance with the equation (1).
  • adaptive section 133 Based on an amplitude value for the error signal, adaptive section 133 enlarges the quantization step width in the case where the amplitude value is large, and reduces the quantization step width in the case where the amplitude value is small.
  • extraction section 134 extracts the internal state of predictive section 132, the prediction coefficients used at predictive section 132, and the quantization code of one sample previous used at adaptive section 133 and outputs the results as synchronization information.
  • synchronization information generating section 106 The basic operation of synchronization information generating section 106 is such that processing corresponding to the decoding section existing within the speech decoding apparatus--processing of the decoding section corresponding to encoding section 102--is carried out in a similar manner within speech encoding apparatus 100 using encoded code I', and parameters (prediction coefficients used at predictive section 132, internal state of predictive section 132, and the quantization code of one sample previous used at adaptive section 133) relating to predictive encoding obtained from these results are reflected in predictive encoding (processing of adaptive section 113 and predictive section 115) occurring at encoding section 102.
  • parameters relating to predictive encoding generated based on encoded code I' are reported from synchronization information generating section 106 as synchronization information, so that it is possible to synchronize (conform) the prediction coefficients used at the predictive section within the speech decoding apparatus, the internal state of this predictive section, and the quantization code of one sample previous used at the adaptive section within the speech decoding apparatus with the prediction coefficients used at predictive section 115 within encoding section 102, the internal state of predictive section 115, and the quantization code of one sample previous used at adaptive section 113.
  • parameters relating to predictive encoding can be obtained based on the same encoded code I' at both speech encoding apparatus 100 and the speech decoding apparatus corresponding to speech encoding apparatus 100.
  • parameters relating to predictive encoding used at the predictive section within the encoding section are updated using the code after bits of the extension code are embedded, so that it is possible to synchronize parameters used in the predictive section within the speech encoding apparatus with parameters used at the predictive section within the speech decoding apparatus, and prevent deterioration in speech quality of the decoded signal.
  • bit embedding section 104 embeds part or all of additional information in the LSB of the encoded code.
  • speech encoding apparatus 100 may also be provided to a non-packet communication type mobile telephone.
  • speech encoding apparatus 100 may also be provided to a non-packet communication type mobile telephone.
  • a line-exchange type communication network is used instead of packet communication, and therefore a multiplex section is provided instead of packetizing section 105.
  • the speech encoding apparatus it is also possible to determine the conditions of the communication terminal apparatus of the communicating party (transmission errors occur easily/with difficulty), and decide the embedding position upon signaling. As a result, it is possible to improve robustness to transmission errors.
  • the size of the encoded code of the extension function at the terminal is also possible.
  • the user of the terminal it is possible for the user of the terminal to select the extent of the addition function.
  • a frequency band width of the extended band from either 7 kHz, 10 kHz or 15 kHz.
  • FIG.6A and FIG. 6B are block diagrams showing configuration examples of the speech decoding apparatus corresponding to speech encoding apparatus 100.
  • FIG.6A shows an example of speech decoding apparatus 150 that is not compatible with the function extension
  • FIG. 6B shows an example of speech decoding apparatus 160 compatible with this function extension. Components that are identical are assigned the same reference numerals.
  • packet separating section 151 separates encoded code I' from the received packet.
  • Decoding section 152 then carries out decoding processing of encoded code I'.
  • D/A converting section 153 converts decoded signal X' obtained as a result to an analog signal, and outputs a decoded speech signal.
  • bit extraction section 161 extracts extension code bit J from encoded code I' outputted from packet separating section 151.
  • Function extension decoding section 162 decodes extracted bit J, obtains information relating to the extension function, and outputs the information to decoding section 163.
  • Decoding section 163 decodes encoded code I' (the same as the encoded code outputted from packet separating section 151) outputted from bit extraction section 161 using the extension function based on information outputted from function extension decoding section 162.
  • the encoded code inputted to decoding sections 152 and 163 is also I' in both cases, and the difference is that encoded code I' is decoded using the extension function, or is encoded without using the extension function.
  • the speech signal obtained by speech decoding apparatus 160 and the speech signal obtained by speech decoding apparatus 150 are in a state in which a transmission path error occurs in the information of the LSB. As a result, deterioration of the speech quality occurs in the decoded signal due to LSB reception errors, but the extent of this speech deterioration is small.
  • the speech encoding apparatus carries out speech encoding using the CELP scheme.
  • CELP there are G.729, AMR, and AMR-WB, etc.
  • the speech encoding apparatus has the same basic configuration as speech encoding apparatus 100 shown in Embodiment 1, and a description of the same portions will be omitted.
  • FIG.7 is a block diagram showing the main configuration of encoding section 201 within the speech encoding apparatus according to this embodiment.
  • Update section 211 updates information relating to the internal states of adaptive codebook 219 and auditory weighting synthesis filter 215.
  • LPC coefficients for the speech signal inputted to encoding section 201 is then obtained at LPC analyzing section 212.
  • the LPC coefficients are used in order to improve auditory quality, and are provided to auditory weighting filter 216 and auditory weighting synthesis filter 215. Further, at the same time, the LPC coefficients are also supplied to LPC quantizing section 213, and LPC quantizing section 213 converts the LPC coefficients to a parameter appropriate for quantization, such as LSP coefficients, and carries out quantization.
  • An index obtained by this quantization is then provided to multiplex section 225 and LPC decoding section 214.
  • LPC decoding section 214 calculates the LSP coefficients after quantization from the encoded code and converts to LPC coefficients. In this way, the LPC coefficients after quantization are obtained.
  • the LPC coefficients after this quantization are then supplied to auditory weighting synthesis filter 215, and used at adaptive codebook 219 and noise codebook 220.
  • Auditory weighting filter 216 assigns a weight to the input speech signal based on the LPC coefficients obtained by LPC analyzing section 212. This is carried out with the object of carrying out spectrum re-shaping so that a quantization distortion spectrum is masked with the spectrum envelope of the input signal.
  • Adaptive codebook 219 holds an excitation signal generated in the past as an internal state, and generates an adaptive vector by repeating this internal state at a desired pitch period. It is appropriate that a range of a pitch period is between 60 Hz to 400 Hz. Further, noise codebook 220 outputs the noise vector stored in advance in a storage area or a vector generated in accordance with a rule without having a storage area like an algebraic structure, as a noise vector. An adaptive vector gain multiplied by the adaptive vector and a noise vector gain multiplied by the noise vector are outputted from gain codebook 223, and the gains are multiplied by the vectors at multipliers 221 and 222.
  • Adder 224 adds the adaptive vector multiplied by the adaptive vector gain and the noise vector multiplied by the noise vector gain, generates an excitation signal, and supplies the signal to auditory weighting synthesis filter 215.
  • Auditory weighting synthesis filter 215 generates an auditory weighting synthesis signal via the excitation signal and provides the auditory weighting synthesis signal to subtracter 217.
  • Subtracter 217 subtracts the auditory weighting synthesis signal from an auditory weighting input signal and supplies the signal after subtraction to search section 218.
  • Search section 218 efficiently searches a combination of the adaptive vector, adaptive vector gain, noise vector and noise vector gain, in which distortion defined from the signal after subtraction becomes minimum, and transmits these encoded codes to multiplex section 225.
  • Search section 218 then decides index i, j, m or index i, j, m, n, in which distortion defined by following equations (2) and (3) becomes minimum, and transmits these to multiplex section 225.
  • t(k) is an auditory weighting input signal
  • p i (k) is a signal obtained by passing an ith adaptive vector through an auditory weighting synthesis filter
  • e j (k) is a signal obtained by passing a jth noise vector through the auditory weighting synthesis filter
  • ⁇ and ⁇ are adaptive vector gain and noise vector gain, respectively.
  • the configuration of the gain codebook is different between equation (2) and equation (3).
  • the gain codebook is expressed as a vector having elements of adaptive vector gain ⁇ m and noise vector gain ⁇ m , and index m for specifying a vector is decided.
  • the gain codebook has adaptive vector gain ⁇ m and noise vector gain ⁇ n independently, and the indexes m and n are decided independently.
  • multiplex section 225 multiplexes the indexes into one and generates and outputs the encoded code.
  • FIG.8 is a block diagram showing the main configuration within synchronization information generating section 206 according to this embodiment.
  • synchronization information generating section 206 The basic operation of synchronization information generating section 206 is the same as synchronization information generating section 106 shown in Embodiment 1. Namely, processing of the decoding section existing within the speech decoding apparatus is carried out in a similar manner within the speech encoding apparatus using encoded code I' , and an adaptive codebook and the internal state of a synthesis filter (with auditory weight) obtained as a result are reflected to adaptive codebook 219 and auditory weighting synthesis filter 215 within encoding section 201. As a result, it is possible to prevent quality deterioration in the decoded signal.
  • Separating section 231 separates the encoded code from inputted encoded code I' and supplies the code to adaptive codebook 233, noise codebook 234, gain codebook 235 and LPC decoding section 232.
  • LPC decoding section 232 the LPC coefficients are decoded using the supplied encoded code and supplied to synthesis filter 239.
  • Adaptive codebook 233, noise codebook 234 and gain codebook 235 decode adaptive vector q(k), noise vector c(k), adaptive vector gain ⁇ q and noise vector gain ⁇ q , respectively, using the encoded code.
  • Multiplier 236 multiplies the adaptive vector gain by the adaptive vector
  • multiplier 237 multiplies the noise vector gain by the noise vector
  • adder 238 adds the signals after the respective multiplications, and generates an excitation signal.
  • excitation signal is expressed as ex (k)
  • excitation signal ex(k) can be obtained from following equation (4).
  • ex k ⁇ q ⁇ q k + ⁇ q ⁇ c k
  • synthesis signal syn(k) is generated in accordance with the following equation (5) at synthesis filter 239 using the decoded LPC coefficients and excitation signal ex(k).
  • ⁇ q (i) is the decoded LPC coefficient
  • NP represents a number of the LPC coefficients.
  • extraction section 240 extracts and outputs the internal states of adaptive codebook 233 and synthesis filter 239.
  • FIG.9 is a block diagram showing the main configuration of speech encoding apparatus 300 according to Embodiment 3 of the present invention.
  • This speech encoding apparatus 300 has the same basic configuration as speech encoding apparatus 100 shown in Embodiment 1. Components that are identical will be assigned the same reference numerals without further explanations. Here, a case will be described as an example where speech encoding is carried out using the ADPCM scheme.
  • a feature of this embodiment is to hold information corresponding to extension code J of function extension encoding section 103 as is out of encoded code I' supplied from bit embedding section 104, set the restriction that this information is not to be changed, carry out encoding processing again on encoded code I' at re-encoding section 301 under this restriction, and decide final encoded code I".
  • Input digital signal X and encoded code I' which is an output of bit embedding section 104 are supplied to re-encoding section 301.
  • Re-encoding section 301 re-encodes encoded code I' supplied from bit embedding section 104.
  • Information corresponding to extension code J out of encoded code I' is eliminated from the encoding target so that no change is applied.
  • the finally obtained encoded code I" is then outputted. As a result, it is possible to hold information of encoded code J of function extension encoding section 103 and generate an optimal encoded code.
  • the prediction coefficients used at the predictive section at this time, the internal state of the predictive section, and the quantization code used one sample previous at the adaptive section, it is possible to synchronize them with the prediction coefficients used at the predictive section of a speech decoding apparatus (not shown) that carries out decoding processing with encoded code I", the internal state of the predictive section, and the quantization code for one sample previous used at the adaptive section, so that it is possible to prevent deterioration in speech quality of the decoded signal.
  • FIG.10 is a block diagram showing the main configuration within re-encoding section 301. With the exception of quantizing section 311 and internal state extraction section 312, this has the same configuration as encoding section 102 (refer to FIG.2 ) shown in Embodiment 1 and is therefore not described.
  • Encoded code I' generated by bit embedding section 104 is supplied to quantizing section 311.
  • Quantizing section 311 leaves embedded information for encoded code J of function extension encoding section 103 as is, and decides again the other encoded codes.
  • FIG. 11 illustrates an outline of re-deciding processing of quantization section 311.
  • encoded code J of function extension encoding section 103 is ⁇ 0, 1, 1, 0 ⁇
  • the encoded code is 4 bits
  • encoded code J is embedded in the LSB.
  • quantizing section 311 re-decides the encoded code for a quantization value in which distortion becomes minimum with respect to a target residual signal, in a state where the LSB is fixed at encoded code J.
  • quantization section 311 is capable of adopting eight types of the encoded code for the quantization value, 0x0, 0x2, 0x4, 0x6, 0x8, 0xA, 0xC and 0xD.
  • quantization section 311 is capable of adopting eight types of the encoded code for the quantization value, 0x1, 0x3, 0x5, 0x7, 0x9, 0xB, 0xD and 0xF.
  • re-decided encoded code I is outputted, and the internal state of predictive section 115, prediction coefficients used at predictive section 115, and the quantization code of one sample previous used at adaptive section 113 are outputted via internal state extraction section 312. This information is then supplied to encoding section 102 to prepare for next input X.
  • the procedure of encoding processing according to this embodiment is arranged as follows.
  • bit embedding section 104 embeds encoded code J supplied from function extension encoding section 103 in encoded code I obtained from encoding section 102, and generates encoded code I'.
  • This encoded code I' is then supplied to re-encoding section 301.
  • Re-encoding section 301 re-decides the encoded code based on the restriction of holding encoded code J, and generates encoded code I".
  • encoded code I is outputted, the prediction coefficients used at the predictive section within re-encoding section 301, the internal state of the predictive section, the quantization code of one sample previous used at the adaptive section within re-encoding section 301 are supplied to encoding section 102 to prepare for next input X.
  • synchronization is achieved between parameters used at the predictive section of the encoding section and parameters used at the predictive section of the decoding section, so that it is possible to prevent the occurrence of deterioration in speech quality.
  • an optimum encoding parameter is decided again based on the restriction due to bit-embedded information, so that it is possible to suppress deterioration due to bit-embedding to a minimum.
  • FIG.12 is a block diagram showing a configuration of re-encoding section 301 in the case of using the CELP scheme.
  • this has the same configuration as encoding section 201 (refer to FIG.7 ) shown in Embodiment 2, and therefore a description thereof will be omitted.
  • Encoded code I' generated by bit embedding section 104 is supplied to noise codebook 321.
  • Noise codebook 321 leaves embedded information for encoded code J as is, and decides again the other encoded codes.
  • Noise codebook 321 then decides the candidate in which distortion becomes minimum through searching and outputs the index.
  • Re-encoding section 301 outputs encoded code I" re-decided in this way, and outputs internal states of adaptive codebook 219, auditory weighting filter 216 and auditory weighting synthesis filter 214 via internal state extraction section 322. This information is then supplied to encoding section 102.
  • the case has been described where information for the extension function is embedded in part of the index for the noise vector, but this is by no means limiting, and, for example, it is also possible to embed information for the extension function in the index for LPC coefficients, adaptive codebook or gain codebook.
  • the principle of operation in this case is the same as described for noise codebook 321 and is characterized in that the index when distortion becomes minimum is re-decided under the restriction of holding information for the extension function.
  • FIG.13 is a block diagram showing a configuration of a variation of speech encoding apparatus 300.
  • Speech encoding apparatus 300 shown in FIG.9 is configured so that the processing result of function extension encoding section 103 changes depending on the processing result of encoding section 102.
  • a configuration is adopted so that processing of function extension encoding section 103 can be carried out independently of the processing result of encoding section 102.
  • the above configuration can be applied to the case where, for example, an input speech signal is divided into two band (for example, 0-4 kHz, and 4-8 kHz), encoding section 102 encodes 0-4 kHz band, function extension encoding section 103 encodes 4-8 kHz band, independently. In this case, it is possible to carry out encoding processing of function extension encoding section 103 without depending on the processing result of encoding section 102.
  • function extension encoding section 103 carries out encoding processing and generates extension code J.
  • This extension code J is then provided to encoding processing restricting section 331. It is then assumed that extension code J is embedded, and restriction information indicating that information relating to this code J is not to be changed is supplied to encoding section 102 from encoding processing restricting section 331.
  • encoding section 102 carries out encoding processing under this restriction, and final encoded code I' is decided.
  • re-encoding section 301 is no longer necessary, so that it is possible to implement speech encoding according to Embodiment 3 with a small amount of calculation.
  • the speech encoding apparatus according to the present invention is by no means limited to Embodiments 1 to 3 described above, and various modifications thereof are possible.
  • the speech encoding apparatus can be provided to a communication terminal apparatus and base station apparatus of a mobile communication system, so that it is possible to provide a communication terminal apparatus and base station apparatus having the same operation results as described above.
  • the present invention can be implemented with software.
  • the present invention can be implemented with software.
  • storing this program in a memory and making an information processing section execute this program it is possible to implement the same function as the speech encoding apparatus of the present invention.
  • each function block used to explain the above-described embodiments is typically implemented as an LSI constituted by an integrated circuit. These may be individual chips or may partially or totally contained on a single chip.
  • each function block is described as an LSI, but this may also be referred to as "IC”, “system LSI”, “super LSI”, “ultra LSI” depending on differing extents of integration.
  • circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible.
  • LSI manufacture utilization of a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor in which connections and settings of circuit cells within an LSI can be reconfigured is also possible.
  • FPGA Field Programmable Gate Array
  • the speech encoding apparatus and speech encoding method according to the present invention can be applied to use on a VoIP network and mobile telephone network, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Electrophonic Musical Instruments (AREA)
  • Reduction Or Emphasis Of Bandwidth Of Signals (AREA)

Claims (12)

  1. Sprachcodiervorrichtung (300), die so eingerichtet ist, dass sie zusätzliche Informationen in einen codierten Code einbettet, wobei die Vorrichtung umfasst:
    einen Codierabschnitt (102), der so eingerichtet ist, dass er einen ersten codierten Code (I) aus einem Sprachsignal (X) unter Verwendung von prädiktivem Codieren erzeugt;
    einen Einbettungsabschnitt (104), der so eingerichtet ist, dass er zusätzliche Informationen (J) in das niedrigstwertige Bit des ersten codierten Codes einbettet, um einen zweiten codierten Code (I') zu gewinnen;
    gekennzeichnet durch
    einen Umcodierabschnitt (301), der umfasst:
    einen Quantisierabschnitt (311);
    einen Abschnitt (312) zum Extrahieren eines internen Status;
    einen adaptiven Abschnitt (113);
    einen prädiktiven Abschnitt (115);
    wobei
    der Quantisierabschnitt (311) so eingerichtet ist, dass er das Sprachsignal umcodiert, um einen dritten codierten Code (I") zu erzeugen, indem er den zweiten codierten Code (I') für einen Quantisierungswert, bei dem Verzerrung in Bezug auf ein Soll-Restsignal unter der Einschränkung, dass die zusätzlichen Informationen in dem zweiten codierten Code (I') gehalten werden, in einem Zustand neu festlegt, in dem das niedrigstwertige Bit des zweiten codierten Codes auf die zusätzlichen Informationen (J) fixiert ist, wobei der dritte codierte Code Verzerrung in Bezug auf das Sprachsignal und einen beim prädiktiven Codieren in dem Codierabschnitt verwendeten prädiktiven Parameter minimiert, und er den prädiktiven Parameter ausgibt; und
    der Abschnitt (312) zum Extrahieren eines internen Status so eingerichtet ist, dass er dem Codierabschnitt (102) zum Vorbereiten auf die nächste Eingabe des Sprachsignals (X) einen internen Status des prädiktiven Abschnitts (115), in dem prädiktiven Abschnitt (115) verwendete Prädiktions-Koeffizienten sowie einen Quantisierungscode eines vorhergehenden Abtastwertes zuführt, der in dem adaptiven Abschnitt (113) verwendet wird.
  2. Sprachcodiervorrichtung nach Anspruch 1, wobei:
    der Codierabschnitt den codierten Code unter Verwendung eines ADPCM-Schemas (Adaptive Differential Pulse Encoded code Modulation scheme) erzeugt.
  3. Sprachcodiervorrichtung nach Anspruch 1, wobei:
    der Codierabschnitt den codierten Code unter Verwendung eines CELP-Schemas erzeugt; und
    der Einbettungsabschnitt die zusätzlichen Informationen in einen codierten Code, der eine CELP-Schema-Anregungsquelle anzeigt, von dem codierten Code einbettet.
  4. Sprachcodiervorrichtung nach Anspruch 1, wobei der Einbettungsabschnitt die Anzahl von Bits der eingebetteten zusätzlichen Informationen entsprechend einer Eigenschaft des Sprachsignals ändert und einer Sprachdekodiervorrichtung die Anzahl von Bits meldet.
  5. Sprachcodiervorrichtung nach Anspruch 1, die des Weiteren einen Bestimmungsabschnitt umfasst, der die Anzahl von Bits der zusätzlichen Informationen aus vorgegebenen Optionen bestimmt.
  6. Kommunikations-Endgerätvorrichtung, die die Sprachcodiervorrichtung nach Anspruch 1 umfasst.
  7. Kommunikations-Endgerätvorrichtung nach Anspruch 6, die des Weiteren einen Sendeabschnitt umfasst, der eine Position, an der der Einbettungsabschnitt die zusätzlichen Informationen einbettet, und die Anzahl von Bits der zusätzlichen Informationen signalisiert.
  8. Kommunikations-Endgerätvorrichtung nach Anspruch 7, wobei der Einbettungsabschnitt eine Position zum Einbetten der zusätzlichen Informationen entsprechend Empfangsbedingungen einer Kommunikations-Endgerätvorrichtung eines Kommunikationsteilnehmers festlegt.
  9. Basisstationsvorrichtung, die die Sprachcodiervorrichtung nach Anspruch 1 umfasst.
  10. Basisstationsvorrichtung nach Anspruch 9, die des Weiteren einen Sendeabschnitt umfasst, der eine Position, an der der Einbettungsabschnitt die zusätzlichen Informationen einbettet, und die Anzahl von Bits der zusätzlichen Informationen signalisiert.
  11. Basisstationsvorrichtung nach Anspruch 10, wobei der Einbettungsabschnitt eine Position zum Einbetten der zusätzlichen Informationen entsprechend Empfangsbedingungen einer Kommunikations-Endgerätvorrichtung eines Kommunikationsteilnehmers festlegt.
  12. Sprachcodierverfahren zum Einbetten zusätzlicher Informationen in einen codierten Code, das umfasst:
    einen Codierschritt, in dem ein erster codierter Code aus einem Sprachsignal unter Verwendung von prädiktivem Codieren erzeugt wird;
    einen Einbettungsschritt, in dem zusätzliche Informationen in das niedrigstwertige Bit des ersten codierten Code eingefügt werden, um einen zweiten codierten Code zu gewinnen;
    gekennzeichnet durch
    einen Umcodierungsschritt, in dem das Sprachsignal umcodiert wird, um einen dritten codierten Code zu erzeugen, indem der zweite codierte Code für einen Quantisierungswert, bei dem Verzerrung in Bezug auf ein Soll-Restsignal unter der Einschränkung, dass die zusätzlichen Informationen in dem zweiten codierten Code gehalten werden, in einem Zustand neu festgelegt wird, in dem das niedrigstwertige Bit des zweiten codierten Code an den zusätzlichen Informationen fixiert ist, wobei der dritte codierte Code Verzerrung in Bezug auf das Sprachsignal und einen beim prädiktiven Codieren in dem Codierabschnitt verwendeten prädiktiven Parameter minimiert, und in dem der prädiktive Parameter ausgegeben wird; und
    einen Schritt, in dem dem Codierabschnitt zum Vorbereiten auf die nächste Eingabe des Sprachsignals ein interner Status eines prädiktiven Abschnitts, in dem prädiktiven Abschnitt verwendete Prädiktions-Koeffizienten sowie ein Quantisierungscode eines vorhergehenden Abtastwertes zugeführt werden, der in dem adaptiven Abschnitt verwendet wird.
EP05765807A 2004-07-20 2005-07-14 Toncodiereinrichtung und toncodierverfahren Active EP1763017B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2004211589 2004-07-20
PCT/JP2005/013052 WO2006009075A1 (ja) 2004-07-20 2005-07-14 音声符号化装置および音声符号化方法

Publications (3)

Publication Number Publication Date
EP1763017A1 EP1763017A1 (de) 2007-03-14
EP1763017A4 EP1763017A4 (de) 2008-08-20
EP1763017B1 true EP1763017B1 (de) 2012-04-25

Family

ID=35785188

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05765807A Active EP1763017B1 (de) 2004-07-20 2005-07-14 Toncodiereinrichtung und toncodierverfahren

Country Status (6)

Country Link
US (1) US7873512B2 (de)
EP (1) EP1763017B1 (de)
JP (1) JP4937746B2 (de)
CN (1) CN1989546B (de)
AT (1) ATE555470T1 (de)
WO (1) WO2006009075A1 (de)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1921608A1 (de) * 2006-11-13 2008-05-14 Electronics And Telecommunications Research Institute Verfahren für die Einfügung von Vektorinformationen zum Schätzen von Sprachdaten in der Phase der Neusynchronisierung von Schlüsseln, Verfahren zum Übertragen von Vektorinformationen und Verfahren zum Schätzen der Sprachdaten bei der Neusynchronisierung von Schlüsseln unter Verwendung der Vektorinformationen
JP5195402B2 (ja) * 2008-12-25 2013-05-08 パナソニック株式会社 無線通信装置及び無線通信システム
US8589166B2 (en) * 2009-10-22 2013-11-19 Broadcom Corporation Speech content based packet loss concealment
JP5447628B1 (ja) 2012-09-28 2014-03-19 パナソニック株式会社 無線通信装置及び通信端末
JP6079230B2 (ja) * 2012-12-28 2017-02-15 株式会社Ｊｖｃケンウッド 付加情報挿入装置、付加情報挿入方法、付加情報挿入プログラム、付加情報抽出装置、付加情報抽出方法、及び付加情報抽出プログラム
CA3238615A1 (en) * 2018-04-25 2019-10-31 Dolby International Ab Integration of high frequency reconstruction techniques with reduced post-processing delay
KR20210005164A (ko) 2018-04-25 2021-01-13 돌비 인터네셔널 에이비 고주파 오디오 재구성 기술의 통합

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5054072A (en) * 1987-04-02 1991-10-01 Massachusetts Institute Of Technology Coding of acoustic waveforms
US5327520A (en) * 1992-06-04 1994-07-05 At&T Bell Laboratories Method of use of voice message coder/decoder
CA2095882A1 (en) * 1992-06-04 1993-12-05 David O. Anderton Voice messaging synchronization
KR100322706B1 (ko) * 1995-09-25 2002-06-20 윤종용 선형예측부호화계수의부호화및복호화방법
CN1183771C (zh) * 1997-01-27 2005-01-05 皇家菲利浦电子有限公司 在编码信号中嵌入补充数据
JP3088964B2 (ja) 1997-03-18 2000-09-18 興和株式会社 振動波の符号化方法、復号化方法、及び振動波の符号化装置、復号化装置
US6182030B1 (en) * 1998-12-18 2001-01-30 Telefonaktiebolaget Lm Ericsson (Publ) Enhanced coding to improve coded communication signals
US7423983B1 (en) * 1999-09-20 2008-09-09 Broadcom Corporation Voice and data exchange over a packet based network
US7574351B2 (en) * 1999-12-14 2009-08-11 Texas Instruments Incorporated Arranging CELP information of one frame in a second packet
US6697776B1 (en) * 2000-07-31 2004-02-24 Mindspeed Technologies, Inc. Dynamic signal detector system and method
SE519985C2 (sv) * 2000-09-15 2003-05-06 Ericsson Telefon Ab L M Kodning och avkodning av signaler från flera kanaler
JP2002135715A (ja) * 2000-10-27 2002-05-10 Matsushita Electric Ind Co Ltd 電子透かし埋め込み装置
US7310596B2 (en) 2002-02-04 2007-12-18 Fujitsu Limited Method and system for embedding and extracting data from encoded voice code
JP4022427B2 (ja) 2002-04-19 2007-12-19 独立行政法人科学技術振興機構 エラー隠蔽方法、エラー隠蔽プログラム、送信装置、受信装置及びエラー隠蔽装置
JP3960959B2 (ja) * 2002-11-08 2007-08-15 三洋電機株式会社 電子透かし埋め込み装置と方法ならびに電子透かし抽出装置と方法
US7009533B1 (en) * 2004-02-13 2006-03-07 Samplify Systems Llc Adaptive compression and decompression of bandlimited signals
US8332218B2 (en) * 2006-06-13 2012-12-11 Nuance Communications, Inc. Context-based grammars for automated speech recognition

Also Published As

Publication number Publication date
CN1989546B (zh) 2011-07-13
JPWO2006009075A1 (ja) 2008-05-01
EP1763017A1 (de) 2007-03-14
US7873512B2 (en) 2011-01-18
JP4937746B2 (ja) 2012-05-23
WO2006009075A1 (ja) 2006-01-26
EP1763017A4 (de) 2008-08-20
ATE555470T1 (de) 2012-05-15
US20080071523A1 (en) 2008-03-20
CN1989546A (zh) 2007-06-27

Similar Documents

Publication Publication Date Title
US7848921B2 (en) Low-frequency-band component and high-frequency-band audio encoding/decoding apparatus, and communication apparatus thereof
US7783480B2 (en) Audio encoding apparatus, audio decoding apparatus, communication apparatus and audio encoding method
EP1763017B1 (de) Toncodiereinrichtung und toncodierverfahren
US20090248404A1 (en) Lost frame compensating method, audio encoding apparatus and audio decoding apparatus
US8086452B2 (en) Scalable coding apparatus and scalable coding method
KR20070038041A (ko) 전기 통신을 위한 멀티-레이트 음성 부호화기에 있어서음성 트랜스-레이팅을 위한 방법 및 장치
US20100250244A1 (en) Encoder and decoder
US20100010810A1 (en) Post filter and filtering method
KR20070092240A (ko) 음성 부호화 장치 및 음성 부호화 방법
KR20070051910A (ko) 스케일러블 부호화 장치, 스케일러블 복호화 장치,스케일러블 부호화 방법, 스케일러블 복호화 방법, 통신단말 장치 및 기지국 장치
KR20140027519A (ko) 오디오 코딩 및 디코딩을 위한 방법 및 장치
JP2005515486A (ja) Ｃｅｌｐによる音声符号間のトランスコーディング・スキーム
US9129590B2 (en) Audio encoding device using concealment processing and audio decoding device using concealment processing
AU6533799A (en) Method for transmitting data in wireless speech channels
US7991611B2 (en) Speech encoding apparatus and speech encoding method that encode speech signals in a scalable manner, and speech decoding apparatus and speech decoding method that decode scalable encoded signals
JP5923517B2 (ja) 階層型符号器における改良ステージの改良符号化
JP2005091749A (ja) 音源信号符号化装置、及び音源信号符号化方法
US7346503B2 (en) Transmitter and receiver for speech coding and decoding by using additional bit allocation method
US20100153099A1 (en) Speech encoding apparatus and speech encoding method
JPWO2008018464A1 (ja) 音声符号化装置および音声符号化方法
JP4236675B2 (ja) 音声符号変換方法および装置
KR100718487B1 (ko) 디지털 음성 코더들에서의 고조파 잡음 가중
US20120278067A1 (en) Vector quantization device, voice coding device, vector quantization method, and voice coding method
EP1560201B1 (de) Kodeumsetzungsverfahren und -vorrichtung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20070116

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20080722

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/00 20060101AFI20080716BHEP

17Q First examination report despatched

Effective date: 20080820

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: PANASONIC CORPORATION

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 555470

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120515

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602005033867

Country of ref document: DE

Effective date: 20120621

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20120425

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 555470

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120425

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20120425

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120825

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120827

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120726

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120731

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20130128

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120805

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120731

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120731

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602005033867

Country of ref document: DE

Effective date: 20130128

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120725

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120714

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120425

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120714

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20140612 AND 20140618

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005033867

Country of ref document: DE

Representative=s name: GRUENECKER, KINKELDEY, STOCKMAIR & SCHWANHAEUS, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20050714

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: III HOLDINGS 12, LLC, WILMINGTON, US

Free format text: FORMER OWNER: PANASONIC CORPORATION, KADOMA-SHI, OSAKA, JP

Effective date: 20140711

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: III HOLDINGS 12, LLC, WILMINGTON, US

Free format text: FORMER OWNER: MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD., KADOMA-SHI, OSAKA, JP

Effective date: 20120502

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF, US

Free format text: FORMER OWNER: PANASONIC CORP., KADOMA-SHI, OSAKA, JP

Effective date: 20140711

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005033867

Country of ref document: DE

Representative=s name: GRUENECKER, KINKELDEY, STOCKMAIR & SCHWANHAEUS, DE

Effective date: 20140711

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF, US

Free format text: FORMER OWNER: MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD., KADOMA-SHI, OSAKA, JP

Effective date: 20120502

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005033867

Country of ref document: DE

Representative=s name: GRUENECKER PATENT- UND RECHTSANWAELTE PARTG MB, DE

Effective date: 20140711

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF, US

Free format text: FORMER OWNER: PANASONIC CORPORATION, KADOMA-SHI, OSAKA, JP

Effective date: 20140711

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF, US

Effective date: 20140722

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005033867

Country of ref document: DE

Representative=s name: GRUENECKER PATENT- UND RECHTSANWAELTE PARTG MB, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005033867

Country of ref document: DE

Owner name: III HOLDINGS 12, LLC, WILMINGTON, US

Free format text: FORMER OWNER: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA, TORRANCE, CALIF., US

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20170727 AND 20170802

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: III HOLDINGS 12, LLC, US

Effective date: 20171207

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20220719

Year of fee payment: 18

Ref country code: DE

Payment date: 20220727

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20220725

Year of fee payment: 18

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602005033867

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20230714

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240201

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230714