JP6065452B2 - Data embedding device and method, data extraction device and method, and program - Google Patents

Description

  Embodiments discussed herein relate to techniques for embedding other information in the data, and techniques for extracting the other information that is embedded.

  Several encoding methods for compressing the data amount of multi-channel audio signals have been proposed. As one of them, MPEG Surround standardized by the Moving Picture Experts Group (MPEG) is known.

  In MPEG Surround, for example, a 5.1 channel audio signal (time signal) to be encoded is frequency-converted, and a frequency signal of 3 channels is first generated by downmixing the obtained frequency signal. Thereafter, the three-channel frequency signal is downmixed again, whereby a two-channel frequency signal corresponding to the stereo signal is calculated. The two-channel frequency signals are then encoded based on the Advanced Audio Coding (AAC) encoding scheme and the Spectral Band Replication (SBR) encoding scheme. In MPEG Surround, spatial information representing the spread or localization of sound is calculated when the signal is downmixed from 5.1 channel to 3 channel and when the signal is downmixed from 3 channel to 2 channel. This spatial information is also encoded.

  Thus, in MPEG Surround, a stereo signal generated by downmixing a multi-channel audio signal and spatial information with a relatively small amount of data are encoded. Therefore, MPEG Surround can provide higher compression efficiency than independently encoding the signals of each channel included in the multi-channel audio signal.

  In this MPEG Surround, a prediction parameter is used to encode spatial information calculated when generating a stereo frequency signal having two channels. The prediction parameter is a prediction for obtaining a 3-channel signal by up-mixing a 2-channel signal after downmixing, that is, a prediction of a signal of one channel among the 3 channels, and a signal of the other two channels. Is a coefficient used to perform based on This upmix will be described with reference to FIG.

In FIG. 1, a downmixed 2-channel signal is represented by an l vector and an r vector, and one signal obtained by upmixing from the 2-channel signal is represented by a c vector. In MPEG Surround, in this case, the c vector is predicted by the following [Equation 1] using the prediction parameters c ₁ and c ₂ .

Note that the prediction parameter stores a plurality of values in advance in a table called “codebook”. The code book is used for improving the bit efficiency used. In MPEG Surround, for c ₁ and c ₂ , a group of 51 × 51 in which areas of −2.0 to +3.0 are cut with a width of 0.1 is prepared as a codebook. Therefore, when this set of prediction parameters is plotted on orthogonal two-dimensional coordinates with each of c ₁ and c ₂ as two coordinate axes, 51 × 51 grid points are obtained.

  As a method for selecting a set of prediction parameters from the codebook, an error defined by the difference between before and after predictive coding of the channel signal is calculated using all the sets of prediction parameters stored in the codebook. A technique for selecting a set that minimizes an error is known. In addition, a technique is disclosed in which a set having the smallest error is selected by a calculation method using the least square method.

  By the way, a technique for embedding other information in data is known. For example, digital watermark information is inserted into the compressed encoded data by re-encoding the compressed encoded data using an encoding control parameter value different from the encoded control parameter value used in the compressed encoded data. Technology is known. In addition, regarding a technique for concealing encryption information in image data, the prediction mode signal of the prediction image data is corrected in accordance with the encryption information, encoding is performed according to the corrected prediction mode signal, and prediction is performed at the time of decoding. A technique is known in which corresponding encryption information is first extracted from a mode signal.

Special table 2008-517338 gazette JP 2009-213074 A Special Table 2000-013800

  In the technique of embedding other information in the data, it is desirable that the deterioration of the quality of the original information is not recognized. For example, in the embedding of other information in the encoded data of an audio signal, the quality of the sound from the original sound of the decoded sound caused by the embedding of information depends on the difference of the original sound source or the difference of the listener. It is preferable to prevent the deterioration from being recognized.

  In view of the above-described problems, the data embedding device described later in the present specification embeds other information in the data representing the original information so that deterioration of the quality of the original information is not recognized. . Further, the data extraction device described later in this specification extracts the other information embedded by the data embedding device.

  One of the data embedding devices described later in this specification includes a code book, a candidate extraction unit, and a data embedding unit. Here, a plurality of prediction parameters are stored in advance in the codebook. The candidate extraction unit extracts a plurality of prediction parameter candidates from the codebook in which the prediction error in the prediction encoding based on the signals of the other two channels with respect to the signal of one channel in the signals of the plurality of channels is within a predetermined range To do. Then, the data embedding unit embeds the data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding from the extracted candidates according to a predetermined data embedding rule.

Also, one of the data extraction devices described later in this specification is to extract data embedded in a prediction parameter by the above-described data embedding device. Some of the data extraction devices include the above-described codebook, a candidate identification unit, and a data extraction unit. Here, the candidate specifying unit specifies the prediction parameter candidate extracted by the candidate extracting unit from the codebook based on the prediction parameter determined as a result of the predictive coding and the signals of the other two channels described above. . Then, the data extraction unit, from the candidate prediction parameters identified, a prediction parameter results have been predictive coding, based on the aforementioned data embedding rule, it extracts the data of the embedded object.

  According to the data embedding device described later in this specification, it is possible to embed other information in the data representing the original information so that the deterioration of the quality of the original information is not recognized. Further, the data extraction device described later in this specification enables the extraction of the other information embedded by the data embedding device.

It is explanatory drawing of the upmix from 2 channels to 3 channels. 1 is a block diagram illustrating the configuration of an encoding system. It is the block diagram which illustrated the structure of the decoding system. It is a structural example of a computer. It is the flowchart which illustrated the processing content of the control processing performed in a data embedding apparatus. It is an example of a parabolic error curved surface. It is an example of an elliptical error surface. It is the figure which drew the error straight line in the projection figure of the error curved surface of FIG. 5A. It is explanatory drawing of the prediction parameter candidate extraction process of the pattern A. It is explanatory drawing of a data embedding process. It is the flowchart which illustrated the processing content of the control processing performed in a data extraction device. It is explanatory drawing of a data extraction process. It is the flowchart which illustrated the detail of the processing content of a prediction parameter candidate extraction process. It is explanatory drawing of the prediction parameter candidate extraction process of the pattern B. FIG. It is explanatory drawing of the specific process example of the prediction parameter candidate extraction process of the pattern B. FIG. It is explanatory drawing of the prediction parameter candidate extraction process of the pattern C. FIG. It is explanatory drawing of the prediction parameter candidate extraction process of the pattern D. FIG. It is explanatory drawing of the specific example of the 1st method of embedding another data. It is explanatory drawing of the specific example of the 2nd method of embedding another data. It is explanatory drawing of the specific example of the 3rd method of embedding another data. It is the flowchart which illustrated the processing content of the modification of the control processing performed in a data embedding apparatus. It is explanatory drawing of an example of the error correction encoding process with respect to embedding target data.

  Hereinafter, an embodiment of a data embedding device and method and a data extraction device and method according to an embodiment will be described in detail with reference to the drawings. Note that this embodiment does not limit the disclosed technology.

  First, FIG. 2A and FIG. 2B will be described. FIG. 2A is a functional block diagram illustrating the configuration of an encoding system including an embodiment of a data embedding device, and FIG. 2B illustrates the configuration of a decoding system including an embodiment of a data extraction device. It is a functional block diagram.

  These components included in the encoding system and the decoding system illustrated in FIGS. 2A and 2B, respectively, are formed as separate circuits. Alternatively, each of the encoding system and the decoding system may be implemented as an integrated circuit in which some or all of these components are integrated. Further, each of these components may be a functional module realized by a program executed on an arithmetic processing device included in each of the encoding system and the decoding system.

  The encoding system in FIG. 2A includes an encoder device 10 and a data embedding device 20. The encoder device 10 includes a 5.1-channel time domain consisting of signals for a total of 5 channels including left front, center, right front, left rear, and right rear, and a 0.1-channel signal dedicated to low frequencies. An audio signal is input. The encoder device 10 encodes the 5.1 channel audio signal and outputs encoded data. On the other hand, the data embedding device 20 is a device that embeds other data in the encoded data output from the encoder device 10. The data embedding device 20 receives data to be embedded in the encoded data. The output of the encoding system in FIG. 2A is encoded data that is output from the encoder device 10 and in which embedded data is embedded.

  On the other hand, the decoding system in FIG. 2B includes a decoder device 30 and a data extraction device 40. The encoded data that is the output of the encoder system of FIG. 2A is input to the decoder device 30, and the original 5.1 channel time domain audio signal is restored from the encoded data and output. The data extraction device 40 extracts the data embedded by the data embedding device 20 from the encoded data and outputs it.

First, the encoding system of FIG. 2A will be described.
The encoder device 10 includes a time-frequency conversion unit 11, a first downmix unit 12, a second downmix unit 13, a stereo encoding unit 14, a prediction encoding unit 15, and a multiplexing unit 16.

  The time-frequency converter 11 converts the 5.1-channel time domain audio signal input to the encoder device 10 into a 5.1-channel frequency signal. In this embodiment, the time-frequency conversion unit 11 performs time-frequency conversion in units of frames performed using, for example, a Quadrature Mirror Filter (QMF). By this conversion, a frequency component signal of each band is obtained when the input time-domain audio signal is equally divided (for example, 64 equal) into one channel of the audio frequency band. The processing performed in each functional block of the encoder device 10 and the data embedding device 20 constituting the encoding system of FIG. 2A is performed for each frequency component signal in each band.

  Each time the first downmix unit 12 receives a 5.1 channel frequency signal, the first downmix unit 12 downmixes the frequency signal of each channel to generate a total of three frequency signals of left, center, and right.

  The second downmix unit 13 generates a total of two channels of left and right stereo frequency signals by downmixing the frequency signals of the respective channels every time three frequency signals are received from the first downmix unit 12. .

  The stereo encoding unit 14 encodes the stereo frequency signal received from the second downmix unit 13 according to, for example, the AAC encoding method and the SBR encoding method described above.

  The prediction encoding unit 15 performs processing for obtaining the above-described prediction parameter value used for prediction performed in the upmix for restoring the three-channel signal from the stereo frequency signal that is the output of the second downmixing unit 13. Do. Note that the upmix for restoring the three-channel signal from the stereo frequency signal is performed in the first upmix section 33 of the decoder device 30 described later according to the method of FIG.

  The multiplexing unit 16 arranges and multiplexes the aforementioned prediction parameter and the encoded data output from the stereo encoding unit 14 in a predetermined order, and outputs the multiplexed encoded data. When the encoder device 10 is operated alone, the multiplexing unit 16 multiplexes the prediction parameter output from the prediction encoding unit 15 with the encoded data. On the other hand, in the case of the configuration of the encoder system illustrated in FIG. 2A, the multiplexing unit 16 multiplexes the prediction parameter output from the data embedding device 20 with the encoded data.

  The data embedding device 20 includes a code book 21, a candidate extraction unit 22, and a data embedding unit 23.

  The codebook 21 stores a plurality of prediction parameters in advance. This codebook 21 uses the same codebook 21 that the predictive encoding unit 15 of the encoder device 10 uses to obtain a prediction parameter. 2A, the code embedding device 20 itself is provided with the code book 21, but instead, the one provided in the prediction encoding unit 15 of the encoder device 10 may be used.

  The candidate extraction unit 22 selects, from the codebook 21, a plurality of prediction parameter candidates whose prediction errors in prediction encoding based on the signals of the other two channels with respect to the signal of one channel among the signals of the plurality of channels are within a predetermined range. Extract. More specifically, the candidate extraction unit 22 extracts a plurality of prediction parameter candidates from the codebook 21 whose error from the prediction parameter obtained by the prediction encoding unit 15 is within a predetermined threshold.

  The data embedding unit 23 embeds the data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding from the candidates extracted by the candidate extracting unit 22 according to a predetermined data embedding rule. More specifically, the data embedding unit 23 selects a prediction parameter to be input to the multiplexing unit 16 from the candidates extracted by the candidate extracting unit 22 according to a predetermined data embedding rule, thereby embedding target data. Are embedded in the prediction parameter.

Next, the decoding system of FIG. 2B will be described.
The decoder device 30 includes a separation unit 31, a stereo decoding unit 32, a first upmix unit 33, a second upmix unit 34, and a frequency time conversion unit 35.

  The demultiplexing unit 31 converts the multiplexed encoded data, which is the output of the encoding system in FIG. 2A, into the prediction parameter and the stereo encoding unit 14 according to the arrangement order in the multiplexing used in the multiplexing unit 16. To the encoded data output from.

  The stereo decoding unit 32 decodes the encoded data received from the separation unit 31 and restores the left and right two-channel stereo frequency signals.

  The first upmixing unit 33 uses the prediction parameter received from the separating unit 31 to upmix the stereo frequency signal received from the stereo decoding unit 32 according to the method of FIG. A total of three channels of frequency signals are restored.

  The second up-mix unit 34 up-mixes the three-channel frequency signals received from the first up-mix unit 33, so that a total of 5. dedicated to the left front, center, right front, left rear, right rear, and low frequency range. Restore the frequency signal of one channel.

  The frequency time conversion unit 35 performs frequency time conversion, which is inverse conversion of the time frequency conversion by the time frequency conversion unit 11, on the 5.1 channel frequency signal received from the second upmix unit 34. Restores and outputs the audio signal in the time domain of the channel.

  The data extraction device 40 includes a code book 41, a candidate identification unit 42, and a data extraction unit 43.

  The codebook 41 stores a plurality of prediction parameter candidates in advance. The code book 41 is the same as the code book 21 provided in the data embedding device 20. In the configuration of FIG. 2B, the code book 41 is provided in the data extraction device 40 itself. Instead, the code device 41 is provided in the decoder device 30 in order to obtain a prediction parameter used in the first upmix unit 33. You may make it use.

  The candidate specifying unit 42 specifies, from the codebook 41, prediction parameter candidates extracted by the candidate extracting unit 22 based on the prediction parameters determined as a result of the predictive encoding and the signals of the other two channels described above. . More specifically, the candidate specifying unit 42 selects the prediction parameter candidates extracted by the candidate extraction unit 22 based on the prediction parameters received from the separation unit 31 and the stereo frequency signal restored by the stereo decoding unit 32. It is specified from the code book 41.

  Based on the data embedding rules used by the data embedding unit 23 when embedding data, the data extracting unit 43 embeds the data embedding unit 23 in the encoded data from the prediction parameter candidates identified by the candidate identifying unit 42. Extract data.

  According to the data embedding device 20 and the data extraction device 40 having the above configuration, another data is embedded in the encoded data, and the other data is extracted from the encoded data in which the other data is embedded. Is possible. In addition, any prediction parameter candidate that is an option when the data embedding device 20 selects a prediction parameter for data embedding has a prediction error in prediction encoding performed using the selected prediction parameter. It stays within a predetermined range. Therefore, when the range of the prediction error is sufficiently narrowed, deterioration of information restored by predictive coding for upmixing by the first upmixing unit 33 of the decoder device 30 is not recognized.

  The data embedding device 20 and the data extraction device 40 can be realized by a computer having a standard configuration.

  Here, FIG. 3 will be described. FIG. 3 illustrates a configuration example of a computer 50 that can be operated as the data embedding device 20 and the data extraction device 40.

  The computer 50 includes an MPU 51, a ROM 52, a RAM 53, a hard disk device 54, an input device 55, a display device 56, an interface device 57, and a recording medium drive device 58. These components are connected via a bus line 59, and various data can be exchanged under the management of the MPU 51.

  An MPU (Micro Processing Unit) 51 is an arithmetic processing unit that controls the operation of the entire computer 50.

  A ROM (Read Only Memory) 52 is a read-only semiconductor memory in which a predetermined basic control program is recorded in advance. The MPU 51 reads out and executes this basic control program when the computer 50 is activated, thereby enabling operation control of each component of the computer 50.

  A RAM (Random Access Memory) 53 is a semiconductor memory that can be written and read at any time and used as a working storage area as needed when the MPU 51 executes various control programs.

  The hard disk device 54 is a storage device that stores various control programs executed by the MPU 51 and various data. The MPU 51 can perform control processing described later by reading and executing a predetermined control program stored in the hard disk device 54. The code books 21 and 41 are stored in advance in the hard disk device 54, for example. Before the computer 50 is operated as the data embedding device 20 and the data extraction device 40, the MPU 51 is caused to perform processing for reading the codebooks 21 and 41 from the hard disk device 54 and storing them in the RAM 53. .

  The input device 55 is, for example, a keyboard device or a mouse device. When operated by a user of the computer 50, the input device 55 acquires input of various information from the user associated with the operation content, and acquires the acquired input information. Is sent to the MPU 51. For example, the input device 55 acquires data to be embedded in the encoded data.

  The display device 56 is, for example, a liquid crystal display, and displays various texts and images according to display data sent from the MPU 51.

  The interface device 57 manages the exchange of various data with various devices connected to the computer 50. For example, the interface device 57 exchanges data such as encoded data and prediction parameters with the encoder device 10 and the decoder device 30.

  The recording medium driving device 58 is a device that reads various control programs and data recorded on the portable recording medium 60. The MPU 51 can read out and execute a predetermined control program recorded on the portable recording medium 60 via the recording medium driving device 58 to perform various control processes described later. As the portable recording medium 60, for example, a flash memory equipped with a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) standard connector. and so on.

  In order to operate such a computer 50 as the data embedding device 20 and the data extraction device 40, first, a control program for causing the MPU 51 to perform each processing step of the control processing described later is created. The created control program is stored in advance in the hard disk device 54 or the portable recording medium 60. Then, a predetermined instruction is given to the MPU 51 to read and execute this control program. By doing so, the MPU 51 functions as each unit included in the data embedding device 20 and the data extracting device 40 illustrated in FIGS. 2A and 2B, respectively, and the computer 50 operates as the data embedding device 20 and the data extracting device 40. It becomes possible to make it.

Next, control processing performed in the data embedding device 20 will be described with reference to FIG. FIG. 4 is a flowchart illustrating the contents of this control process.
In FIG. 4, first, in S100, the candidate extraction unit 22 performs candidate extraction processing. This process is a process of extracting a plurality of prediction parameter candidates from the codebook 21 whose error from the prediction parameter obtained by the prediction encoding unit 15 of the encoder device 10 is within a predetermined threshold. Details of this candidate extraction processing will be further described.

  First, in S101, the candidate extraction unit 22 performs an error curved surface determination process. This process is a process for determining the shape of the error curved surface.

  Here, the error curved surface will be described. The error curved surface is obtained when the prediction parameter regarding the error (prediction error) between the prediction result using the prediction parameter for the signal of one channel among the plurality of channels and the actual signal of the one channel is changed. Is a curved surface obtained by graphing the distribution of. In this embodiment, the error curved surface is obtained by graphing the distribution when the prediction parameter is changed with respect to the prediction error when the signal of the center channel is predicted as shown in FIG. 1 using the prediction parameter. It is a curved surface.

Examples of error surfaces are shown in FIGS. 5A and 5B. FIG. 5A is an example of a parabolic error curved surface, and FIG. 5B is an example of an elliptical error curved surface.
In both FIG. 5A and FIG. 5B, an error curved surface is drawn on orthogonal three-dimensional coordinates. Here, the directions of the arrows c ₁ and c ₂ represent the magnitudes of the prediction parameter values for the left channel and the right channel, respectively, and the direction perpendicular to the plane stretched by the arrows c ₁ and c ₂ (on the paper surface) (Upward) represents the magnitude of the prediction error. Therefore, on the plane parallel to the plane stretched by the arrows c ₁ and c ₂ , the prediction error is the same value no matter how the prediction parameter value pair is selected and the central channel signal is predicted. Become.

By the way, assuming that the actual signal of the center channel is the signal vector c ₀ and the prediction result of the signal of the center channel using the left and right channel signals and the prediction parameters is the signal vector c, the prediction error d is [Expression 2]

Note that l and r are signal vectors representing left channel and right channel signals, respectively, and c ₁ and c ₂ are prediction parameters for the left channel and the right channel, respectively.

When this [Expression 2] is modified for c ₁ and c ₂ , the following [Expression 3] is obtained.
The function f represents an inner product of vectors.

Attention is paid to the denominator on the right side of this [Expression 3], that is, the following [Expression 4].

  When the value of this [Equation 4] is zero, the shape of the error curved surface is a parabolic shape of FIG. 5A, and when this value is not zero, the shape of the error curved surface is an elliptical shape of FIG. 5B. . Therefore, in the error surface determination process of S101 of FIG. 4, the inner product of the signal vectors for the left channel and right channel signals output from the first downmix unit 12 is obtained to calculate the value of [Equation 4], The shape of the error curved surface is determined based on whether this value is zero.

  In addition, the case where the value of [Equation 4] is zero is the following three cases: (1) when the r vector is a zero vector, (2) when the l vector is a zero vector, (3) Limited to one where l vector is a constant multiple of r vector. Therefore, in the error surface determination process of S101, by checking whether the left channel and right channel signals output from the first downmix unit 12 correspond to any of these three cases, The shape may be determined.

  Next, in S102, the candidate extraction unit 22 performs processing for determining whether or not the shape of the error curved surface determined by the error curved surface determination processing in S101 is a parabolic shape. Here, when the candidate extraction unit 22 determines that the shape of the error curved surface is a parabolic shape (when the determination result is Yes), the process proceeds to S103, and the process for data embedding is performed. On the other hand, when the candidate extraction unit 22 determines that the shape of the error curved surface is not parabolic (ellipse) (when the determination result is No), the process proceeds to S114. In this case, data is not embedded.

In step S103, the candidate extraction unit 22 performs error straight line determination processing.
The error straight line is a set of points where the prediction error on the error curved surface is the smallest, and when the error curved surface is a parabolic type, the set of points is a straight line. When the error curved surface is elliptical, the point with the smallest prediction error is one point and does not become a straight line. Therefore, it can be said that the determination process of S102 described above is a process of determining whether or not a set of points with the smallest prediction error is a straight line.

In the example of the parabolic error curved surface in FIG. 5A, the tangent when the plane defined by the prediction parameters c ₁ and c ₂ and the error curved surface are in contact is the error straight line. The prediction parameters c ₁ and c ₂ specified by the points on the error straight line have the same prediction error even if any of them is selected to predict the center channel signal.

  The error straight line equation is expressed by the following three equations in accordance with the signal levels of the left channel and the right channel. In the error straight line determination process of S103, the error straight line is determined by substituting the left channel and right channel signals output from the first downmix unit 12 into each signal vector on the right side of each equation.

First, when the r vector is a zero vector, that is, when the right channel is a silence signal, the equation of the error line is expressed by the following [Equation 5].

FIG. 6 is a projection of the error curved surface of FIG. 5A onto the plane stretched by the arrows c ₁ and c ₂ , in which a straight line represented by the above [Equation 5] is drawn.

Next, secondly, when the r vector is a zero vector, that is, when the right channel is a silence signal, the equation of the error straight line is expressed by the following [Equation 6].

Third, when the l vector is a constant multiple of the r vector, that is, when the ratio between the l vector and the r vector is constant in all the samples in the processing target frame, The equation is expressed by the following [Equation 7].

  When both the r vector and the l vector are zero vectors, that is, when both the R and L channel signals are zero, the set of points with the smallest prediction error is not a straight line. In this case, the candidate extraction unit 22 advances the process to S104 without determining the error line in the error line determination process of S103. Processing in this case will be described later.

  Next, in S104, the candidate extraction unit 22 performs a prediction parameter candidate extraction process. This process is a process of extracting prediction parameter candidates from the codebook 21 based on the error straight line obtained by the process of S103.

In the prediction parameter candidate extraction process, based on the positional relationship between the error straight line and each point corresponding to each prediction parameter stored in the codebook 21 on the plane defined by the prediction parameters c ₁ and c ₂ . Then, prediction parameter candidates are extracted. In the prediction parameter candidate extraction process according to the present embodiment, as the positional relationship, a point where the distance from the error line is within a predetermined range among the points corresponding to the prediction parameter candidates stored in the codebook 21. Select. Then, a set of prediction parameters represented by the selected point is extracted as a prediction parameter candidate. A specific example of this process will be described with reference to FIG.

In the example of FIG. 7, each point corresponding to each prediction parameter stored in the codebook 21 is arranged on orthogonal two-dimensional plane coordinates defined by the prediction parameters c ₁ and c ₂ . The points are arranged as grid points. In this example, some of these points are on the error line. In FIG. 7, among the points arranged as these lattice points, those existing on the error line are represented by white circles. The plurality of points represented by these white circles are the same, with the distance from the error line being the smallest (ie, zero) among all the grid points. Therefore, the prediction parameters c ₁ and c ₂ represented by these points have the same minimum and the same prediction error, regardless of which set is used to predict the central channel signal. Therefore, in the example of FIG. 7, a set of prediction parameters c ₁ and c ₂ represented by these white circle points is extracted from the codebook 21 as a prediction parameter candidate.

  Note that in the prediction parameter candidate extraction process, several prediction parameter candidate extraction patterns are prepared, and extracted according to the positional relationship between the error straight line on the plane and the corresponding point of the prediction parameter of the codebook 21. Select a pattern and extract prediction parameter candidates. Details of selection of this extraction pattern will be described later.

The candidate extraction unit 22 performs the processes from S101 to S104 as the candidate extraction process of S100.
When the candidate extraction process of S100 by the candidate extraction unit 22 is completed, the data embedding unit 23 performs a data embedding process in S110. This process is performed by selecting a prediction parameter as a result of the predictive encoding by the predictive encoding unit 15 from the candidates extracted by the process of S104 according to a predetermined data embedding rule, thereby setting the data to be embedded as the predictive parameter. This is an embedding process. Details of this data embedding process will be further described.

  First, in S111, the data embedding unit 23 performs an embedding capacity calculation process. In this process, a process of calculating the maximum number of bits that does not exceed the number of prediction parameter candidates extracted in the prediction parameter candidate extraction process of S104 as a data capacity that can be embedded is performed. For example, in the example of FIG. 7, since the number of white circles extracted as prediction parameter candidates is 6, it can be seen that at least four values, that is, data of 2 bits can be embedded. Therefore, in the calculation process of S111, “2 bits” is obtained as the calculation result.

  Next, in S112, the data embedding unit 23 performs an embedding value giving process. This process is a process of assigning an embedded value to each prediction parameter candidate extracted in the prediction parameter candidate extraction process of S104 according to a predetermined rule. In S113, the data embedding unit 23 performs a prediction parameter selection process. This process refers to a bit string corresponding to the embedding capacity in the data to be embedded, selects a prediction parameter candidate to which an embedding value that matches the value of this bit string is selected, and multiplexes the selected candidate in the encoder 10. This is a process of outputting to the conversion unit 16.

A specific example of the processing of S112 and S113 will be described with reference to FIG.
It is assumed that prediction parameter candidates are extracted as shown in FIG. 7, for example. Also, rules to impart padding value in this case is assumed to have been established for the ascending order of the value of prediction parameter c _2. In this case, as illustrated in FIG. 8, the embedding values “00”, “01”, “10”, and “11” are assigned to the points with white circles in ascending order of the value of the prediction parameter c _2. Is done.

Here, it is assumed that the data to be embedded is “11010101101101010...” As shown in FIG. When the leading 2-bit bit string “11” in this data is embedded in the prediction parameter, the prediction parameters c ₁ and c ₂ corresponding to the white circle points to which the embedding value of “11” is assigned in FIG. A set of values is selected and output to the multiplexing unit 16.

When the above processing up to S113 is completed, the control processing of FIG. 4 ends.
On the other hand, if it is determined by the determination process of S102 described above that the shape of the error curved surface is not parabolic (ellipse), the data embedding unit 23 performs the process of S114. This process is a process in which a set of values of the prediction parameters c ₁ and c ₂ output from the prediction encoding unit 15 of the encoder device 10 is output to the multiplexing unit 16 as it is and multiplexed into encoded data. Therefore, in this case, data is not embedded. When the process of S114 is completed, the control process of FIG. 4 ends.

  By performing the above-described control process in the data embedding device 20, other data is embedded in the encoded data generated by the encoder device 10.

  Here, the simulation result about the data amount which can be embedded by the control process mentioned above is demonstrated. In this simulation, 12 types (sound, music, etc.) of 1-minute audio signals of 5.1 channels of the MPEG Surround system with a sampling frequency of 48 kHz and a transmission rate of 160 kb / s were used.

  In this simulation, the average number of prediction parameters was 1312 per second, the probability that the shape of the error surface was parabolic was 5%, and the embedding capacity in the prediction parameters was an average of 5 bits. As a result, the amount of data that can be embedded is 320 kb / s, and it has been found that 2.4 kilobytes of data can be embedded when converted into an audio signal for one minute.

  Next, control processing performed in the data extraction device 40 in FIG. 2B will be described with reference to FIG. FIG. 9 is a flowchart illustrating the contents of this control process.

  In FIG. 9, first, in S200, the candidate specifying unit 42 performs candidate specifying processing. This process is a process of identifying prediction parameter candidates extracted by the candidate extraction unit 22 from the codebook 41 based on the prediction parameters received from the separation unit 31 and the stereo frequency signal restored by the stereo decoding unit 32. is there. Details of this candidate specifying process will be further described.

  First, in S201, the candidate specifying unit 42 performs an error curved surface determination process. This process is a process for determining the shape of the error curved surface, and is the same process as the process performed by the candidate extraction unit 22 as the process of S101 in FIG. However, in the processing of S201, the inner product of the signal vectors for the left channel and right channel stereo signals output from the stereo decoding unit 32 is obtained to calculate the value of the above-mentioned [Equation 4], and this value is zero. The shape of the error curved surface is determined depending on whether or not there is.

  Next, in S202, the candidate specifying unit 42 performs processing for determining whether or not the shape of the error curved surface determined by the error curved surface determination processing in S201 is a parabolic shape. Here, when the candidate specifying unit 42 determines that the shape of the error curved surface is a parabolic shape (when the determination result is Yes), the process proceeds to S203 and the process for data extraction is performed. On the other hand, when the candidate specifying unit 42 determines that the shape of the error curved surface is not parabolic (ellipse) (when the determination result is No), data is embedded in the prediction parameter. It is determined that it is not, and the control process of FIG. 9 is terminated.

  Next, in S203, the error identifying process is performed by the candidate specifying unit 42. This process is a process of estimating the error straight line determined by the candidate extraction unit 22 by the error straight line determination process of S103 of FIG. The process of S203 is the same process as the error straight line determination process of S103 of FIG. However, in the error straight line estimation process of S203, the left channel and right output from the stereo decoding unit 32 are added to each signal vector on the right side of each of the equations [Formula 5], [Formula 6], and [Formula 7]. The error straight line is estimated by substituting the stereo signal of the channel.

  Next, in S204, the candidate specifying unit 42 performs a prediction parameter candidate estimation process. This process is a process of estimating the prediction parameter candidates extracted by the candidate extraction unit 22 in the prediction parameter candidate extraction process of S104 of FIG. 4, and based on the error straight line estimated by the process of S203, the prediction parameter candidates Is extracted from the codebook 41. The process of S204 is the same process as the prediction parameter candidate extraction process of S104 of FIG. However, in the prediction parameter candidate estimation process of S204, a point having the smallest and the same distance from the error line is selected from the points corresponding to each prediction parameter stored in the codebook 41, and the selected point A process of extracting a set of prediction parameters represented by is performed. The extracted set of prediction parameters is the result of specifying the prediction parameter candidate by the candidate specifying unit 42.

The candidate specifying unit 42 performs the processes from S201 to S204 as the candidate specifying process of S200.
When the candidate identification process of S200 by the candidate identification unit 42 is completed, the data extraction unit 43 performs a data extraction process in S210. In this process, the data embedding unit 23 embeds the data embedded in the encoded data from the prediction parameter candidates specified by the candidate specifying unit 42 based on the data embedding rule used when the data embedding unit 23 embeds the data. It is a process to extract.

Details of this data embedding process will be further described.
First, in S <b> 211, the data extraction unit 43 performs an embedded capacity calculation process. This process is a process of calculating the data capacity that can be embedded, and is the same process as the process performed by the data embedding unit 42 as the process of S111 in FIG.

  Next, in S212, the data extraction unit 43 performs an embedding value assignment process. In this process, each of the prediction parameter candidates extracted in the prediction parameter candidate estimation process in S204 is embedded according to the same rule as that used by the data embedding unit 23 in the embedding value assignment process in S112 of FIG. This is a process of assigning a value.

  Next, in S213, the data extraction unit 23 performs embedded data extraction processing. In this process, the embedded value assigned by the embedded value assigning process in S212 is acquired with respect to the prediction parameter received from the separation unit 31, and this value is stored as a result of extraction of the data embedded by the data embedding unit 23. This is a process of buffering in the area in the order of acquisition.

A specific example of the processing in S212 and S213 will be described with reference to FIG.
Assume that prediction parameter candidates are specified as shown in FIG. 7, for example. Also, rules to impart padding value in this case, as the same rule in the example of FIG. 8, it is assumed that has been defined as the ascending order of the value of prediction parameter c _2. In this case, as illustrated in FIG. 10, the embedding values “00”, “01”, “10”, and “11” are given to the points indicated by white circles in ascending order of the value of the prediction parameter c _2. Is done.

Here, the prediction parameter received from the separation unit 31 is one set of values for the point having the maximum value of the prediction parameter c ₂ among the points indicated by white circles, that is, the point to which the embedded value “11” is assigned. Suppose you were doing it. In this case, the embedding value “11” is extracted as data embedded by the data embedding unit 23 and buffered in a predetermined information buffer.

When the above processing up to S213 is completed, the control processing of FIG. 9 ends.
By performing the above control processing in the data extraction device 40, the data embedded by the data embedding device 20 is extracted.

Next, FIG. 11 will be described. FIG. 11 is a flowchart illustrating the details of the processing contents of the prediction parameter candidate extraction process, which is the process of S104 of FIG.
In S104-1 following the error straight line determination process in S103 of FIG. 4, the candidate extraction unit 22 performs a process of determining whether or not the set of points with the smallest error is a straight line.

  As described above, when both the r vector and the l vector are zero vectors, the set of points with the smallest prediction error is not a straight line. In the determination process of S104-1, it is determined whether or not this is the case.

  In S104-1, the candidate extraction unit 22 determines that at least one of the r vector and the l vector is not a zero vector, and therefore the set of points with the smallest error is a straight line (when the determination result is Yes). Then, the process proceeds to S104-2. On the other hand, when the candidate extraction unit 22 determines that both the r vector and the l vector are both zero vectors, and therefore the set of points with the smallest error is not a straight line (when the determination result is No). ), The process proceeds to S104-8.

Next, in S104-2, the candidate extraction unit 22 performs a process of determining whether or not the error straight line obtained by the error straight line determination process in S103 of FIG. Note that the area of the codebook 21 is a circumscribed rectangular area including points corresponding to the respective prediction parameters stored in the codebook 21 on a plane defined by the prediction parameters c ₁ and c ₂ . Here, when the candidate extraction unit 22 determines that the error straight line intersects the area of the codebook 21 (when the determination result is Yes), the process proceeds to S104-3, and the error straight line matches the area of the codebook 21. When it is determined that the two do not intersect (when the determination result is No), the process proceeds to S104-7.

  Next, in S <b> 104-3, the candidate extraction unit 22 performs a process of determining whether or not the error straight line is parallel to any boundary side of the codebook 21. The boundary side of the code book 21 is a rectangular side that demarcates the area of the code book 21 described above. In this determination process, when the error straight line expression is expressed in the form of the above-described [Expression 5] or [Expression 6], the determination result is Yes. On the other hand, when the error straight line equation is expressed in the form of the above-mentioned [Expression 7], that is, when the ratio of the signal magnitudes of the L and R channels is a constant value for a predetermined period. The error straight line is determined not to be parallel to any boundary side of the codebook 21, and the determination result is No.

  In the determination process of S104-3, when the candidate extraction unit 22 determines that the error straight line is parallel to any boundary side of the codebook 21 (when the determination result is Yes), the process proceeds to S104-4. To proceed. On the other hand, when the candidate extraction unit 22 determines that the error straight line is not parallel to any of the boundary sides (when the determination result is No), the candidate extraction unit 22 proceeds to S104-5.

  Next, in S104-4, the candidate extraction unit 22 performs a prediction parameter candidate extraction process based on the pattern A, and thereafter, the process proceeds to S111 in FIG. The pattern A prediction parameter candidate extraction process will be described later.

  On the other hand, in S <b> 104-5, the candidate extraction unit 22 performs a process of determining whether or not the error straight line intersects with both the pair of opposing boundary sides in the codebook 21. Here, when the candidate extraction unit 22 determines that the error line intersects both of the pair of opposing boundary sides in the codebook 21 (when the determination result is Yes), the process proceeds to S104-6. The candidate extraction unit 22 performs a prediction parameter candidate extraction process based on the pattern B. Then, the process proceeds to S111 in FIG. On the other hand, when the candidate extraction unit 22 determines in the determination process in S104-5 that the error straight line does not intersect with both of the pair of opposing boundary sides in the codebook 21 (when the determination result is No). Advances the process to S114 of FIG. The prediction parameter candidate extraction process for pattern B will be described later.

  By the way, when the determination result of S104-2 is No, the determination process of S104-7 is performed. In S104-7, the candidate extraction unit 22 performs a process of determining whether or not the error straight line is parallel to the boundary side of the code book 21 described above. This determination process is the same as the determination process of S104-3. Here, when the candidate extraction unit 22 determines that the error straight line is parallel to the boundary side of the codebook 21 (when the determination result is Yes), the process proceeds to S104-8, and the prediction parameter candidate based on the pattern C is obtained. An extraction process is performed, and thereafter, the process proceeds to S111 in FIG. On the other hand, when the candidate extraction unit 22 determines that the error straight line is not parallel to the boundary side of the codebook 21 (when the determination result is No), the candidate extraction unit 22 proceeds to S114 in FIG. Details of the pattern C prediction parameter candidate extraction process will be described later.

By the way, when the determination result in S104-1 is No, the candidate extraction unit 22 performs a prediction parameter candidate extraction process based on the pattern D in S104-9, and thereafter proceeds to S111 in FIG. Details of the pattern D prediction parameter candidate extraction process will be described later.
The prediction parameter candidate extraction process illustrated in FIG. 11 is performed as described above.

Next, prediction parameter candidate extraction processing for each pattern will be described.
First, the pattern A handled in the process of S104-4 will be described. This pattern is a pattern when the error straight line intersects the area of the codebook 21 and is parallel to any boundary side of the codebook 21.

  For example, the pattern A corresponds to the case where the error straight line and the boundary side of the codebook 21 are in the positional relationship shown in FIG. In the positional relationship of FIG. 7, the boundary side of the codebook 21 is parallel to the boundary side parallel to the c2 axis. In this case, the candidate extracting unit 22 extracts, as the prediction parameter candidates, the points having the smallest distance and the same distance from the error line among the points corresponding to the respective prediction parameters of the codebook 21. To do. In the example of FIG. 7, among the points corresponding to the prediction parameters of the codebook 21, the points represented by white circles are present on the error line, and therefore the distance from the error line is zero. Are extracted as prediction parameter candidates.

  Next, the pattern B handled in the process of S104-6 will be described with reference to FIG. This pattern is a pattern in the case where the error straight line is not parallel to any boundary side of the code book 21 but intersects with both of the pair of opposing boundary sides in the code book 21.

In [A] and [B] of FIG. 12, each point corresponding to each prediction parameter stored in the codebook 21 is arranged on the orthogonal two-dimensional plane coordinates defined by the prediction parameters c ₁ and c ₂ . The points where these points are arranged as lattice points are the same as in FIG.

Figure [A] of 12, of a pair of boundary edges that are two pairs of opposed for codebook 21, for both of the pair of boundary edges parallel to the c ₂ axes, if the error straight line intersects Represents an example. In this case, for each value of the prediction parameter C _{1 in} the codebook 21, the corresponding point of the codebook 21 that is closest to the error line is extracted as a prediction parameter candidate. The prediction parameter candidates extracted in this way are the values of the prediction parameter C ₂ that minimizes the prediction error when each value of the prediction parameter C ₁ is used to predict the signal of the center channel.

On the other hand, [B] in FIG. 12 shows an error straight line for both of a pair of boundary sides parallel to the c ₁ axis among a pair of opposing boundary sides for the codebook 21. An example in the case of crossing is shown. In this case, for each value of the prediction parameter C _{2 in} the codebook 21, the corresponding point of the codebook 21 that is closest to the error line is extracted as a prediction parameter candidate. The prediction parameter candidate extracted in this way is the value of the prediction parameter C ₁ that minimizes the prediction error when each value of the prediction parameter C ₂ is used for predicting the signal of the center channel.

  When the cases of [A] and [B] in FIG. 12 are summarized, first, for each lattice point on each of a pair of boundary sides where the error line intersects, the lattice point closest to the error line is the same. Once selected, the prediction parameters corresponding to the selected grid points are extracted as candidates. Furthermore, for each grid point that exists on each line segment that is parallel to the pair of boundary sides where the error line intersects and passes through the grid point, the grid point closest to the error line is selected for each line segment, and the selection is made. Prediction parameters corresponding to the set grid points are extracted as candidates.

  More specifically, the candidate extraction unit 22 may perform the prediction parameter candidate extraction processing by the pattern B as described below, for example.

FIG. 13 illustrates the example in the case of [A] in FIG. 12 again. Here, it is assumed that the error straight line is represented by c ₂ = l × c ₁ . Further, the coordinates of four adjacent points among the lattice points representing the codebook 21 are defined as shown in FIG.

At this time, the following steps [a] and [b] are performed while increasing the value of the variable i by one.
[A] Find c2 _j and c2 _{j + 1} satisfying c2 _j ≦ l × c1 _i ≦ c2 _{j + 1} .
[B] The following (i) and (ii) are divided into cases, and prediction parameter candidates in each case are extracted from the codebook 21.
(I) When | c2 _j −l × c1 _i | ≦ | c2 _{j + 1} −l × c1 _i |, the prediction parameter corresponding to the lattice point (c1 _i , c2 _j ) is used as a codebook as a candidate. 21.
(Ii) When | c2 _j −l × c1 _i | >> c2 _{j + 1} −l × c1 _i |, prediction parameters corresponding to lattice points (c1 _i , c2 _{j + 1} ) are used as candidates. Extract from the codebook 21.

  Next, the pattern C handled in the process of S104-8 will be described with reference to FIG. This pattern is a pattern in the case where the error straight line does not intersect the area of the codebook 21 but is parallel to any boundary side of the codebook 21.

In [A] and [B] of FIG. 14, each point corresponding to each prediction parameter stored in the codebook 21 is arranged on the orthogonal two-dimensional plane coordinates defined by the prediction parameters c ₁ and c ₂ . The points where these points are arranged as lattice points are the same as in FIG.

[A] in FIG. 14, although the error straight line does not intersect the region of the codebook 21 is parallel to the boundary side is parallel to the c ₂ axis, which is an example of the pattern C is applied. In this case, the corresponding points of the codebook 21 that exist on the boundary side closest to the error line among the boundary sides of the codebook 21 are extracted as prediction parameter candidates. The prediction parameters extracted in this way have the same prediction error no matter which one of them is selected and the central channel signal is predicted.

  On the other hand, [B] in FIG. 4 is an example in which the error straight line is not parallel to any boundary side of the codebook 21, and therefore the pattern C is not applied. In the case of [B], the prediction error is minimized when the prediction of the center channel signal is performed using the prediction parameters for the corresponding points with white circles among the corresponding points of the codebook 21. Thus, when other prediction parameters are used, the prediction error becomes large. For this reason, in this embodiment, in such a case, other data is not embedded in the prediction parameter.

  Next, the pattern D handled in the process of S104-9 will be described with reference to FIG. This pattern is a pattern in the case where both the R and L channel signals are zero as described above when the error straight line is not determined in the error straight line determination process of S103.

FIG. 15 is a diagram in which each point corresponding to each prediction parameter stored in the codebook 21 is arranged on orthogonal two-dimensional plane coordinates defined by the prediction parameters c ₁ and c ₂ . The points arranged as grid points are the same as in FIG. In this case, no matter what prediction parameter is selected and the prediction of the center channel is performed according to the formula [1], the signal of the center channel is zero. Therefore, in this case, all the prediction parameters stored in the codebook 21 are extracted as candidates.

The candidate extraction unit 22 extracts the prediction parameter candidates by properly using the above-described prediction parameter candidate extraction processing for each pattern according to the positional relationship between the error line and the area of the codebook 21.
In the prediction parameter candidate estimation process, which is the process of S204 in FIG. 9, the same process as the above-described prediction parameter candidate extraction process for each pattern is performed by the candidate specifying unit 42.

Next, embedding of data different from the data to be embedded by the data embedding device 20 will be described.
Any data may be embedded in the prediction parameter by the data embedding device 20. Here, by adding and embedding other data representing the head of the embedding target data, it becomes easy to find the head of the embedding target data from the data extracted by the data extraction device 40. Further, by adding another data representing the end of the embedding target data and embedding, it becomes easy to find the end of the embedding target data from the data extracted by the data extracting device 40.

Here, a method of embedding data different from the embedding target data will be described.
The first method is a method in which the data embedding unit 23 embeds in the prediction parameter after adding another data representing the presence of the embedding target data and its head or tail before or after the embedding target data. It is. A specific example of this method will be described with reference to FIG.

  In the example of FIG. 16, the embedding target data is the value of [A], that is, “1101010... 01010”.

  In the example of [B], the bit string “0001” is defined in advance as the start data indicating the existence and the head of the embedding target data, and the bit string “1000” is the end indicating the end of the embedding target data. This is an example in the case where data is defined in advance. However, in this case, neither of these two types of bit strings appears in the embedding target data. That is, for example, it is assumed that three or more values “0” do not appear continuously in the embedding target data. In this example, the data embedding unit 23 performs processing for adding start data immediately before embedding target data and further adding end data immediately after embedding target data in the prediction parameter selection processing in S113 of FIG. Do. After that, a bit string corresponding to the embedding capacity in the data to be embedded after adding these data is referred to, and a process of selecting a prediction parameter candidate to which an embedding value that matches the value of this bit string is selected. The data extraction unit 43 of the data extraction device 40 excludes these start data and end data from the data extracted from the prediction parameters by the embedded data extraction process of S213 in FIG. Make output.

  The example of [C] is an example in which the bit string “01111110” is defined in advance as start / end data representing the presence of the embedding target data and the beginning or end. However, in this case, neither of these bit strings appears in the embedding target data. That is, for example, it is assumed that six or more values “1” do not appear continuously in the embedding target data. In this example, the data embedding unit 23 performs a process of adding start / end data immediately before and immediately after the embedding target data in the prediction parameter selection process of S113 of FIG. After that, a bit string corresponding to the embedding capacity in the data to be embedded after adding these data is referred to, and a process of selecting a prediction parameter candidate to which an embedding value that matches the value of this bit string is selected. The data extraction unit 43 of the data extraction device 40 excludes the start / end data from the data extracted from the prediction parameters by the embedded data extraction process of S213 in FIG. 9, and outputs the remaining data thereafter. Like that.

  Next, a second method of embedding data different from the embedding target data will be described. In this method, the data embedding unit 23 selects a prediction parameter to be a result of predictive encoding from extracted candidates according to a predetermined data embedding rule, so that another data is set as the prediction parameter together with the data to be embedded. It is a technique of embedding.

  For example, when the prediction parameter candidates extracted by the candidate extraction unit 22 are extracted as indicated by white circles shown in FIG. 7, the embedding value is given by the data embedding unit 23 as shown in FIG. Consider a case where an embedded value is given. In this case, four of the six white circles shown in FIG. 7 are used for embedding the embedding target data given the embedding value, but the remaining two white circles are the embedding target. It is not used for embedding data. The second method for embedding different data is to use a prediction parameter candidate that is not used for embedding the target data for embedding another data.

That is, as illustrated in FIG. 17, one of the two white circles that are not used for embedding the data to be embedded, for example, the one having a smaller value of the prediction parameter C ₂ , indicates the presence and start of the data to be embedded. It is assigned in advance to what is represented. In addition, the other one, for example, the one having the larger value of the prediction parameter C ₂ is assigned in advance to the one representing the end of the embedding target data. Then, the data embedding unit 23 first selects a prediction parameter candidate assigned to “start of embedding target data” in the selection process of S113 of FIG. 4, and thereafter, the relationship between the embedding target data and the embedding value is determined. Corresponding prediction parameter candidates are sequentially selected. Thereafter, the data embedding unit 23 selects a prediction parameter candidate assigned to “end of embedding target data” after completing the selection of prediction parameter candidates based on the embedding target data.

  In the data extraction unit 43 of the data extraction device 40, in the embedded data extraction process of S213 in FIG. 9, the data extracted from the prediction parameters between the prediction parameters to which the start and end of the embedding target data are respectively assigned. Make output.

Next, a third method for embedding data different from the embedding target data will be described.
As described above, the processing performed in each functional block of the data embedding device 20 is performed for each frequency component signal in each band when one audio frequency band is divided. That is, the candidate extraction unit 22 selects a prediction parameter candidate whose error from a prediction parameter obtained for each frequency band by predictive coding for each frequency band for the signal of the central channel is within a predetermined threshold. Are extracted for each frequency band. Therefore, in the third method, the data embedding unit 23 uses the prediction parameter that is the result of predictive encoding for the first frequency band for embedding the embedding target data in the prediction parameter for the first frequency band. This is done by selecting from the extracted candidates. Then, the data embedding unit 23 sets the prediction parameter, which is the result of predictive encoding for the second frequency band different from the first frequency band, to embed another data in the prediction parameter, in the second frequency band. By selecting from the extracted candidates.

A specific example of the third method of embedding this different data will be described with reference to FIG.
In this example, among the prediction parameter candidates obtained in each of the six frequency bands for each frame of the audio signal, three sets of candidates on the low frequency side are used for embedding data to be embedded, and three sets on the high frequency side are used. Are used for embedding other data. The other data at this time may be, for example, data representing the existence and start or end of the embedding target data, as in the first and second methods described above.

In FIG. 18, a variable i is an integer between zero and i_max, and represents a number assigned to each frame of the audio signal in time order. The variable j is an integer between zero and j_max and represents a number assigned to each frequency band in ascending order of frequency. In the example of FIG. 8, the values of the constant i_max and the constant j_max are “5”. (C ₁ , c ₂ ) _{i, j} represents the prediction parameter of the j-th band of the i-th frame.

  Here, FIG. 19 will be described. FIG. 19 is a flowchart illustrating the processing contents of a modified example of the control processing performed in the data embedding device 20. This flowchart is a process for embedding data to be embedded and other data as in the example illustrated in FIG. 18, and the data embedding unit is the data embedding process in S110 following the process in S104 in the flowchart illustrated in FIG. 23.

Following S104 in FIG. 4, first, in S301, the data embedding unit 23 performs a process of substituting the initial value “0” for the variable i and the variable j, respectively.
S302 following S301 represents a processing loop paired with S312. The data embedding unit 23 repeats the processing from S303 to S311 using the value of the variable i at the time of this processing.

  Subsequent S303 represents a processing loop paired with S310. The data embedding unit 23 repeats the processing from S304 to S309 using the value of the variable j at the time of this processing.

  In subsequent S304, the data embedding unit 23 performs an embedding capacity calculation process. This process is a process for calculating a data capacity that can be embedded by using prediction parameter candidates for the j-th band of the i-th frame, and is the same as S111 in FIG. It is.

  In step S305, the data embedding unit 23 performs an embedding value assignment process. This process is a process for assigning an embedded value to each prediction parameter candidate for the j-th band of the i-th frame according to a predetermined rule, and is shown in S112 in FIG. Is the same process.

  Next, in S306, the data embedding unit 23 performs a process of determining whether the jth band belongs to the low frequency side or the high frequency side. If the data embedding unit 23 determines that the jth band belongs to the low frequency side, the data embedding unit 23 proceeds to S307. If the data embedding unit 23 determines that the jth band belongs to the high frequency side, the data embedding unit 23 proceeds to S308. .

  Next, in S307, the data embedding unit 23 performs a prediction parameter selection process according to the bit string of the embedding target data, and then proceeds to S309. This process refers to a bit string corresponding to the embedding capacity in the embedding target data, and selects a prediction parameter candidate to which an embedding value that matches the value of this bit string is assigned for the j th band of the i th frame. This is a process of selecting from the prediction parameter candidates. The processing content of this processing is the same as the processing in S113 of FIG.

  On the other hand, in S308, the data embedding unit 23 performs a prediction parameter selection process according to a bit string of data different from the data to be embedded, and then proceeds to S309. This process refers to a bit string corresponding to the embedding capacity in the other data, and selects prediction parameter candidates to which an embedding value matching the value of this bit string is assigned for the j-th band of the i-th frame. This is a process of selecting from the prediction parameter candidates. The processing content of this processing is also the same as the processing in S113 of FIG.

Next, in S309, the data embedding unit 23 performs a process of substituting the result obtained by adding “1” to the current value of the variable j into the variable j.
Next, in S310, the data embedding unit 23 performs a process of determining whether or not to continue the process loop represented as a pair with S303. Here, when the data embedding unit 23 determines that the value of the variable j is equal to or less than the constant j_max, the data embedding unit 23 continues to repeat the processing from S304 to S309. On the other hand, if the data embedding unit 23 determines that the value of the variable j exceeds the constant j_max, the data embedding unit 23 ends the process from S304 to S309 and advances the process to S311.

Next, in S311, the data embedding unit 23 performs a process of substituting the result obtained by adding “1” to the current value of the variable i into the variable i.
Next, in S <b> 312, the data embedding unit 23 performs a process of determining whether or not to continue the process loop represented as a pair with S <b> 302. If the data embedding unit 23 determines that the value of the variable i is equal to or smaller than the constant i_max, the data embedding unit 23 continues to repeat the processing from S303 to S311. On the other hand, when the data embedding unit 23 determines that the value of the variable i exceeds the constant i_max, the data embedding unit 23 ends the process from S303 to S311 and thereafter ends the control process.

The data embedding device 20 performs embedding into the prediction parameters of the embedding target data and the different data as illustrated in FIG. 18 by performing the above control processing.
Note that the data extraction unit 43 of the data extraction device 40 performs the same process as illustrated in FIG. 19 in the data extraction process of S210 of FIG. 9 to extract the data to be embedded and the different data. To.

  In the three methods for embedding data different from the embedding target data described above, as an example of other embedding data, data indicating the existence and start or end of embedding target data is given. These data may be embedded in the prediction parameters. For example, when embedding data subjected to error correction coding processing as embedding target data, data indicating whether or not the error correction coding processing has been performed on the embedding target data is embedded in the prediction parameter as separate data. It may be. As an error correction coding method used in this case, for example, a simple method illustrated in FIG. 20 may be used.

  In the example of FIG. 20, the data [A] is the original data before the error correction coding process is performed. This error correction encoding process is to output the value of each bit constituting the original data three times in succession. The data [B] is obtained by performing this error correction coding process on the data [A]. The data embedding device 20 embeds the data [B] in the prediction parameter as the embedding target data, and uses the data indicating that the error correction encoding processing has been performed on the embedding target data as the prediction data. Embed in.

  On the other hand, the data [C] is the embedding target data extracted by the data extraction device 40, and some of the bits are different from the data [B]. In order to restore the original [A] data from the embedding target data, the embedding target data is divided into 3-bit bit strings in the order of arrangement, and a majority process is performed on the values of the three bits included in each bit string. . By arranging the results of the majority processing in the order of arrangement, [D] corrected data is obtained, and it can be seen that this data matches the original data of [A].

The following additional notes are further disclosed with respect to the embodiment including the above examples.
(Appendix 1)
A codebook in which a plurality of prediction parameters are stored in advance;
A candidate extraction unit for extracting a plurality of prediction parameter candidates in which prediction errors in prediction encoding based on signals of other two channels with respect to signals of one channel among signals of a plurality of channels are within a predetermined range from the codebook; ,
A data embedding unit that embeds data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding according to a predetermined data embedding rule from the extracted candidates;
A data embedding device comprising:
(Appendix 2)
The prediction parameter has a component for each of the signals of the other two channels;
The candidate extraction unit determines a straight line that is a set of points at which the prediction error is minimized on a plane defined by two components of the prediction parameter, and stores the straight line and the codebook on the plane. Based on the positional relationship with each point corresponding to each prediction parameter being performed, extraction of the prediction parameter candidates is performed.
The data embedding device according to supplementary note 1, wherein:
(Appendix 3)
The candidate extraction unit extracts, from among the prediction parameters stored in the codebook, those having a plurality of points on the plane corresponding to the prediction parameters on the straight line. The data embedding device according to appendix 2.
(Appendix 4)
The candidate extraction unit determines whether or not a set of points on which the prediction error is minimized on the plane is a straight line, and determines that the set of points is a straight line. 4. The data embedding device according to appendix 2 or 3, wherein a prediction parameter candidate is extracted.
(Appendix 5)
The data embedding device according to appendix 4, wherein the determination is performed using an inner product of signal vectors of the signals of the other two channels.
(Appendix 6)
The plane is a plane of an orthogonal coordinate system, and the two components of the prediction parameter are components in the direction of each coordinate axis,
Each prediction parameter stored in the codebook is such that each point corresponding to the candidate on the plane is arranged as a grid point in a rectangular area with the coordinate axis direction as the direction of each side on the plane. Is set in advance,
When the candidate extraction unit determines that the set of points with the smallest prediction error on the plane is a straight line, the straight line intersects both of a pair of sides facing each other in the rectangle on the plane. If it is determined whether or not to intersect, for each of the pair of sides, a prediction parameter corresponding to the lattice point closest to the straight line is extracted from the lattice points existing on the side, and the pair of sides is extracted. For each line segment in the region that is parallel to the side of the line and passes through the grid point, a prediction parameter corresponding to the grid point closest to the straight line among the grid points existing on the line segment is extracted.
The data embedding device according to appendix 4 or 5, characterized in that:
(Appendix 7)
The candidate extraction unit determines whether the ratio of the signal magnitudes of the other two channels is a constant value for a predetermined period, and the ratio is a constant value for the predetermined period. 7. The data embedding device according to appendix 6, wherein when it is determined that the straight line intersects any one side of the rectangle on the plane.
(Appendix 8)
If the candidate extraction unit determines that the straight line does not intersect both of the two opposing sides of the rectangle on the plane, whether the straight line is parallel to any side of the rectangle or not If it is determined that they are parallel, prediction parameter candidates corresponding to grid points existing on the side closest to the straight line among the sides of the rectangle are extracted. Or the data embedding device according to 7.
(Appendix 9)
The candidate extraction unit determines whether or not the magnitudes of the signals of the other two channels are both zero, and if it is determined that both are zero, it is stored in the codebook 9. The data embedding device according to any one of appendices 1 to 8, wherein all prediction parameters are extracted.
(Appendix 10)
The data embedding device according to any one of appendices 1 to 9, wherein the data embedding unit embeds data different from the data to be embedded together with the data to be embedded.
(Appendix 11)
11. The data embedding according to claim 10, wherein the data embedding unit adds the other data before or after the data to be embedded, and embeds the data to which the other data is added in the prediction parameter. apparatus.
(Appendix 12)
The data embedding unit selects a prediction parameter as a result of the predictive encoding from the extracted candidates according to a predetermined data embedding rule, and thereby sets the other data together with the data to be embedded as the prediction parameter. The data embedding device according to appendix 10, wherein the data embedding device is embedded.
(Appendix 13)
The candidate extraction unit has an error between a prediction parameter obtained for each frequency band based on the prediction coding for each frequency band based on the signals of the other two channels with respect to the signal of the one channel within a predetermined threshold. Extract a plurality of candidate prediction parameters for each frequency band from the codebook,
The data embedding unit selects the prediction parameter that is the result of the predictive coding for the first frequency band from the extracted candidates for the first frequency band, thereby selecting the data to be embedded By selecting from the extracted candidates for the second frequency band a prediction parameter that is embedded in the prediction parameter and that is the result of the predictive coding for a second frequency band that is different from the first frequency band Embed the other data in the prediction parameter;
The data embedding device according to appendix 10, characterized in that.
(Appendix 14)
The data embedding device according to any one of appendices 10 to 13, wherein the another data is data indicating whether or not the data to be embedded is embedded in the prediction parameter.
(Appendix 15)
14. The data embedding according to any one of appendices 10 to 13, wherein the other data is data indicating whether or not error correction coding is performed on the data to be embedded. apparatus.
(Appendix 16)
A data extraction device that extracts data embedded in the prediction parameter by the data embedding device according to any one of appendices 1 to 15,
The codebook;
Candidates for identifying prediction parameter candidates extracted by the candidate extraction unit from the codebook of the data extraction device itself based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels A specific part,
A data extraction unit that extracts the data to be embedded from the specified prediction parameter candidates based on the data embedding rule;
A data extraction apparatus comprising:
(Appendix 17)
A data extraction device that extracts data embedded in the prediction parameter by the data embedding device according to appendix 4,
The codebook;
A candidate identification unit that identifies prediction parameter candidates extracted from the codebook of the data extraction device itself, based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels;
A data extraction unit that extracts the data to be embedded from the specified prediction parameter candidates based on the data embedding rule;
With
The candidate specifying unit determines whether or not a set of points with the smallest error on the plane is a straight line, and determines that the set of points is a straight line, the prediction based on the positional relationship Identify parameter candidates,
The data extraction unit extracts the data to be embedded when the candidate identification unit identifies the prediction parameter candidate.
A data extraction apparatus characterized by that.
(Appendix 18)
A data extraction device that extracts data embedded in the prediction parameter by the data embedding device according to attachment 14,
The codebook;
A candidate identification unit that identifies prediction parameter candidates extracted from the codebook of the data extraction device itself, based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels;
When the other data is extracted from the identified prediction parameter candidates, and the extracted other data indicates that the data to be embedded is embedded, the specified prediction parameter A data extraction unit that extracts the data to be embedded from the candidates,
A data extraction apparatus comprising:
(Appendix 19)
A data extraction device that extracts data embedded in the prediction parameter by the data embedding device according to attachment 15,
The codebook;
A candidate identification unit that identifies prediction parameter candidates extracted from the codebook of the data extraction device itself, based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels;
A data extraction unit for extracting the data to be embedded and the other data from the identified prediction parameter candidates;
With
When the extracted other data indicates that error correction encoding is performed on the data to be embedded, error correction is performed on the extracted data to be embedded. Feature data extraction device.
(Appendix 20)
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range Multiple extraction from
Embedding the data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding according to a predetermined data embedding rule from the extracted candidates.
A data embedding method characterized by the above.
(Appendix 21)
A data extraction method for extracting data,
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range The data is embedded in the prediction parameter by selecting a plurality of prediction parameters from the extracted candidates according to a predetermined data embedding rule from the extracted candidates.
The data extraction method is:
Based on the prediction parameter determined as a result of the predictive encoding and the signals of the other two channels, the parameter candidates are identified from the codebook,
Extracting the data from the identified prediction parameter candidates based on the data embedding rules;
A data extraction method characterized by that.
(Appendix 22)
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range Multiple extraction from
Embedding the data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding according to a predetermined data embedding rule from the extracted candidates.
A program that causes a computer to execute processing.
(Appendix 23)
A program for causing a computer to extract data,
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range The data is embedded in the prediction parameter by selecting a plurality of prediction parameters from the extracted candidates according to a predetermined data embedding rule from the extracted candidates.
The program is
Based on the prediction parameter determined as a result of the predictive encoding and the signals of the other two channels, the parameter candidates are identified from the codebook,
Extracting the data from the identified prediction parameter candidates based on the data embedding rules;
A program that causes a computer to execute processing.

DESCRIPTION OF SYMBOLS 10 Encoder apparatus 11 Time frequency conversion part 12 1st downmix part 13 2nd downmix part 14 Stereo encoding part 15 Predictive encoding part 16 Multiplexing part 20 Data embedding apparatus 21, 41 Codebook 22 Candidate extraction part 23 Data embedding Unit 30 decoder unit 31 separation unit 32 stereo decoding unit 33 first upmix unit 34 second upmix unit 35 frequency time conversion unit 40 data extraction device 42 candidate identification unit 43 data extraction unit 50 computer 51 MPU
52 ROM
53 RAM
54 hard disk device 55 input device 56 display device 57 interface device 58 recording medium drive device 59 bus line 60 portable recording medium

Claims (14)

A codebook in which a plurality of prediction parameters are stored in advance;
A candidate extraction unit for extracting a plurality of prediction parameter candidates in which prediction errors in prediction encoding based on signals of other two channels with respect to signals of one channel among signals of a plurality of channels are within a predetermined range from the codebook; ,
A data embedding unit that embeds data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding according to a predetermined data embedding rule from the extracted candidates;
A data embedding device comprising: The prediction parameter has a component for each of the signals of the other two channels;
The candidate extraction unit determines a straight line that is a set of points at which the prediction error is minimized on a plane defined by two components of the prediction parameter, and stores the straight line and the codebook on the plane. Based on the positional relationship with each point corresponding to each prediction parameter being performed, extraction of the prediction parameter candidates is performed.
The data embedding device according to claim 1.   The candidate extraction unit extracts, from among the prediction parameters stored in the codebook, those having a plurality of points on the plane corresponding to the prediction parameters on the straight line. The data embedding device according to claim 2.   The candidate extraction unit determines whether or not a set of points on which the prediction error is minimized on the plane is a straight line, and determines that the set of points is a straight line. 4. The data embedding apparatus according to claim 2, wherein extraction of prediction parameter candidates is performed.   5. The data embedding apparatus according to claim 4, wherein the determination is performed using an inner product of signal vectors of the signals of the other two channels. The plane is a plane of an orthogonal coordinate system, and the two components of the prediction parameter are components in the direction of each coordinate axis,
Each prediction parameter stored in the codebook is such that each point corresponding to the candidate on the plane is arranged as a grid point in a rectangular area with the coordinate axis direction as the direction of each side on the plane. Is set in advance,
When the candidate extraction unit determines that the set of points with the smallest prediction error on the plane is a straight line, the straight line intersects both of a pair of sides facing each other in the rectangle on the plane. If it is determined whether or not to intersect, for each of the pair of sides, a prediction parameter corresponding to the lattice point closest to the straight line is extracted from the lattice points existing on the side, and the pair of sides is extracted. For each line segment in the region that is parallel to the side of the line and passes through the grid point, a prediction parameter corresponding to the grid point closest to the straight line among the grid points existing on the line segment is extracted.
6. The data embedding device according to claim 4 or 5,   The candidate extraction unit determines whether the ratio of the signal magnitudes of the other two channels is a constant value for a predetermined period, and the ratio is a constant value for the predetermined period. The data embedding device according to claim 6, wherein if it is determined that the straight line intersects any one side of the rectangle on the plane.   If the candidate extraction unit determines that the straight line does not intersect both of the two opposing sides of the rectangle on the plane, whether the straight line is parallel to any side of the rectangle or not The prediction parameter candidate corresponding to the lattice point existing on the side closest to the straight line among the sides of the rectangle is extracted when it is determined that they are parallel. 8. The data embedding device according to 6 or 7.   The candidate extraction unit determines whether or not the magnitudes of the signals of the other two channels are both zero, and if it is determined that both are zero, it is stored in the codebook The data embedding device according to claim 1, wherein all prediction parameters are extracted. A data extraction device that extracts data embedded in the prediction parameter by the data embedding device according to any one of claims 1 to 9,
The codebook;
Candidates for identifying prediction parameter candidates extracted by the candidate extraction unit from the codebook of the data extraction device itself based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels A specific part,
A prediction parameter results have been of the predictive coding, said data embedding based on the rules, the data extractor for extracting data of the embedded object from the candidates of the specified prediction parameters,
A data extraction apparatus comprising: A candidate extraction unit included in the data embedding device selects a prediction parameter candidate in which a prediction error in prediction encoding based on a signal of another two channels with respect to a signal of one channel in a signal of a plurality of channels is within a predetermined range. , Extracting a plurality of prediction parameters from a codebook in which a plurality of prediction parameters are stored in advance,
A data embedding unit provided in the data embedding device selects a prediction parameter as a result of the predictive encoding from the extracted candidates according to a predetermined data embedding rule, thereby selecting data to be embedded as the prediction parameter. Embedded in the
A data embedding method characterized by the above. A data extraction method performed by a data extraction device for extracting data ,
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range The data is embedded in the prediction parameter by selecting a plurality of prediction parameters from the extracted candidates according to a predetermined data embedding rule from the extracted candidates.
The data extraction method is:
The candidate specifying unit provided in the data extraction device selects the extracted prediction parameter candidates from the codebook based on the prediction parameters determined as a result of the prediction encoding and the signals of the other two channels. Identified from
Data extracting portion to which the data extracting apparatus is equipped with the prediction parameters and the results of the predictive coding, on the basis of said data embedding rule, to extract the data from the candidate of the specified prediction parameters,
A data extraction method characterized by that. A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range Multiple extraction from
Embedding the data to be embedded in the prediction parameter by selecting a prediction parameter as a result of the predictive encoding according to a predetermined data embedding rule from the extracted candidates.
A program that causes a computer to execute processing. A program for causing a computer to extract data,
A codebook in which a plurality of prediction parameters are stored in advance for prediction parameter candidates in which prediction errors in prediction coding based on signals of the other two channels with respect to signals of one channel in signals of a plurality of channels are within a predetermined range The data is embedded in the prediction parameter by selecting a plurality of prediction parameters from the extracted candidates according to a predetermined data embedding rule from the extracted candidates.
The program is
Based on the prediction parameter determined as a result of the prediction encoding and the signals of the other two channels, the extracted prediction parameter candidates are identified from the codebook,
The prediction parameters and the results of the predictive coding, on the basis of said data embedding rule, to extract the data from the candidate of the specified prediction parameters,
A program that causes a computer to execute processing.