KR100964096B1 - System and method of extracting supplement information from acoustic signal - Google Patents

Abstract

The present invention relates to a system and an extraction method for extracting additional information from an audio signal. The system for inserting additional information according to the present invention includes a frame dividing unit for dividing an audio signal into a plurality of frames, and dividing the frame into subframes. A subframe dividing unit, a Fourier transform unit for Fourier transforming each subframe, a subband dividing unit dividing the Fourier transformed subframe into subbands, and a plurality of subbands for additional information about the sound signal It is equipped with an information insertion unit that inserts by controlling the energy of the speaker, such as music, audiobooks, telephone calls, etc. When all sound signals are transmitted, the relevant information is inserted into the sound signal and transmitted to the speakers of streets, various public transportation means, and public places. Additional information can be obtained from the sound signal output from the By being output to obtain it it is possible to enjoy an improved mobile.

Description

System and method of extracting supplement information from acoustic signal

The present invention relates to a method of processing a sound signal in which additional information is inserted, and more particularly, to a system and a method for extracting additional information from an acoustic signal.

In recent years, with the rapid development of wireless mobile communication technology, the possession of personal wireless devices has become very common. The convergence of various technologies and services in these wireless devices continues to improve functionality and ease of use.

Meanwhile, various music and audiobooks can be listened to at any time through various audio reproducing apparatuses while the desired audio data is secured by the listener. However, when listening is made in various places that are unspecified, for example, public places, advertisements and exhibition halls, and radio stations, the listener must retain information about the music when he / she wants to listen to it later.

However, there is a very problem in securing data about the music for re-listening because it does not have any information about the music in the state of listening only in an unspecified place.

The present invention relates to a system and an extraction method for extracting additional information from an acoustic signal.

Synchronization to receive the sound signal from the speaker outputs the sound signal inserted by the energy modulation of the digital signal according to the present invention, and to synchronize to find the starting point for the additional information inserted in the received sound signal part; A frame dividing unit dividing the received sound signal into a plurality of frames; A sub frame dividing unit dividing the frame into a plurality of sub frames; A Fourier transform unit for Fourier transforming each subframe; A subband dividing unit dividing a frequency band of the Fourier transformed subframe into a plurality of subbands; And an information extraction unit configured to compare energy levels of the subbands and extract additional information included in the sound signal.
The information extracting unit may include a high subband having the highest energy level and a low subband having the lowest energy level, and the high subband and the low subband according to the order of the energy sizes of the subbands. Distinguish the intermediate subbands having an intermediate energy magnitude, and classify the digital signal of the side information according to whether the energy magnitude of the intermediate subband is close to the energy magnitude of the high subband or the energy magnitude of the low subband. Can be extracted.
The synchronization unit may find the starting point of the additional information by checking the energy level of the intermediate subband.
The synchronization unit may include an early skip unit that skips a predetermined length at an initial position of the sound signal, a synchronization frame divider that divides the sound signal from the skipped position into a plurality of frames, and divides the frame into a plurality of server frames. A synchronization subframe divider for dividing, a Fourier transform for Fourier-transforming the subbands, a synchronization subband divider for dividing a frequency band of the Fourier-transformed subframe into a plurality of subbands, A sync scoring unit may be provided to compare the magnitudes of energy and perform scoring on the frames.
The sync scoring unit determines that the digital signal is inserted into the subframe when the intermediate subband is close to the energy level of the high subband or the low subband, and scores a relatively high score. Close to the midpoint of the energy level of the high subband or the low subband can score a relatively low score.
Receiving the sound signal from a speaker that outputs a sound signal inserted into the energy signal of the digital signal according to the present invention, synchronizes to find a starting point for the digital signal inserted in the sound signal, the reception Divide a sound signal into a plurality of frames, divide the frame into a plurality of subframes, Fourier transform each of the subframes, divide a frequency band of the Fourier transformed subframes into a plurality of subbands, The subbands may be arranged in the order of energy magnitude, and the intermediate energy level of the high subband having the highest energy level and the low energy level and the intermediate energy level of the high subband and the low subband may be determined. Has a middle subband, and the middle subband Depending on whether the close-up energy is the energy amount of the high sub-band energy, or close to the size of the lower sub-band is extracted by separating the digital signal.
The synchronization may find the starting point of the digital signal by checking the energy magnitude of the intermediate subband.
The synchronization may determine the energy level of the intermediate subbands from a position where a predetermined sampling position is skipped at an initial sampling position of the sound signal.
When the intermediate subband is close to the energy level of the high subband or the low subband, it may be determined that the digital signal is inserted into the subframe.

delete

delete

delete

delete

delete

delete

delete

delete

The system and extraction method for extracting additional information from an acoustic signal according to the present embodiment inserts and transmits additional information related to a sound signal such as music, audiobooks, telephone calls, etc. in a street signal and various public transportation means. And by extracting and obtaining additional information from a sound signal output from a speaker such as a public place using a wireless device, it is possible to enjoy a more improved mobile environment.

FIG. 1 is a diagram for describing an example of acquiring additional information about an acoustic signal being listened to by a wireless device.

As shown in FIG. 1, in the present embodiment, when a user of the wireless device 11 listens to an audio signal output through the speaker 10 at any place, the sound signal output from the speaker 10 is within a predetermined range. Receives an acoustic signal using the wireless device 11, the wireless device 11 obtains the additional information 13 included in the acoustic signal from the received acoustic signal and outputs it to the display of the wireless device 11. do.

Accordingly, the user can enjoy a more advanced mobile environment because the user can acquire various additional information about the corresponding sound while listening to various sounds such as music and audiobook in real time without being constrained by space and place.

Hereinafter, an embodiment of a method of inserting additional information into a sound signal and obtaining additional information from a sound signal into which the additional information is inserted will be described in order to provide such a mobile environment.

2 is a diagram illustrating a system for inserting additional information into an acoustic signal according to the present embodiment.

As shown in FIG. 2, a system for inserting additional information into a sound signal according to the present embodiment includes a frame divider 100 which first divides a music signal, an audio book, and a sound signal received in various calls into a plurality of frames; And a subframe division unit 101 for dividing each frame into a plurality of subframes, and a Fourier transform unit 102 for Fourier transforming each subframe.

And a subband dividing unit 103 for dividing a plurality of subbands in the frequency band of each subframe obtained by the Fourier transform, and an energy deviation standardizing unit 104 for comparing energy magnitudes of the subbands and normalizing energy deviations. And an information insertion section 105 for inserting additional information, which is a digital signal, by subband energy modulation.

The system also includes a binarization unit 110 for digitally encoding additional information inserted into an acoustic signal, for example, a music name, a singer's name, Internet connection information for listening, and the like, and error correction encoding the digital signal. An error correction encoder 111 and a header inserter 112 for inserting a header into the error corrected coded digital signal, wherein the digital signal with the header is inserted into the acoustic signal by energy modulation in the information inserter 105. Is inserted.

The system includes a Fourier inverse transform unit 120, a subframe combiner 121, and a frame combiner 122 that perform Fourier inverse transform on each subframe into which additional information is inserted.

Hereinafter, the operation state of each configuration and system will be described in detail.

3 is a diagram illustrating an example of dividing an acoustic signal into a frame and a subframe according to the present embodiment. As shown in FIG. 3, the frame dividing unit 100 includes a plurality of sound signals recorded on a wireless device 11 of a user, for example, a mobile phone, a PDA, a notebook computer, and the like, having the same length on a time axis. Split into frames. The subframe dividing unit 101 divides a frame of a predetermined position (for example, a start position at which additional information is to be inserted) among the divided frames into a plurality of subframes P1, P2, and P3.

In the present embodiment, the frame may be divided into three subframes. However, the number of divisions of the subframe may be implemented in various ways. The frame dividing unit 100 and the sub-frame dividing unit 101 use a trapezoidal or raised cosine-shaped window at the time of dividing so that a smooth overlap is formed between the frames or the sub-frames.

Hereinafter, each subframe is referred to as a first subframe P1, a second subframe P2, and a third subframe P3. Each subframe P1, P2, or P3 may be sampled and digitalized. The Fourier transform unit 102 performs Fourier transform on each of the first sub frame P1, the second sub frame P2, and the third sub frame P3.

4 is a diagram illustrating an example of dividing a subband after Fourier transforming each subframe according to the present embodiment, and FIG. 5A illustrates energy of subbands at a specific position before inserting additional information into an acoustic signal. FIG. 5B is a diagram illustrating levels of each subframe, and FIG. 5B is a diagram illustrating energy levels of subbands at a specific position after inserting additional information into an acoustic signal.

As shown in FIG. 4, the subband divider 103 locates each Fourier-transformed subframe P1, P2, and P3 in a range of a predetermined frequency or more, and includes a plurality of frequency bands for inserting additional information. Split into subbands. The number of divisions of the subbands may vary depending on the size of the additional information input. In addition, the range of a predetermined frequency band for subband division may use a higher or lower frequency band around 4kHz that is sensitive to the human ear. Therefore, it can be used in various ways such as about 5 ~ 10 kHz or 1 ~ 3kHz.

As shown in FIG. 5A, a subband of a specific position divided in the first subframe P1 is referred to as a first subband S1 and a subband of the specific position divided in the second subframe P1. The band is called a second subband S2, and the subband at the specific position divided in the third subframe P3 is called a third subband S3. The specific position here may be the same frequency band position.

The energy deviation standardization unit 104 calculates the energy of each subband S1, S2, and S3 obtained after the Fourier transform, for example, the standard deviation or the average amplitude, and then calculates the first subband ( The energy magnitudes of S1), the second subband S2, and the third subband S3 are distinguished. At this time, the subband having the largest energy magnitude and the subband having the smallest energy magnitude are compared so that the energy difference between the two subbands is equal to or greater than the preset difference value d.

For example, as shown in FIG. 5A, the energy deviation normalization unit 104 may have the highest energy value of the third subband S3 of the third subframe P3 of the first frame F1. It is determined that the second subband S2 of the two subframes P2 has the smallest energy value, and the difference value d preset by the energy difference between the third subband S3 and the second subband S2 is set in advance. Energy deviation standardization does not apply if it is determined to be above (or above).

However, if the energy difference between the subband with the largest energy and the subband with the smallest energy in another frame F2 (F3) (F4) is less than (or less than) the difference value d, then the largest energy magnitude is obtained. Two subbands by increasing the energy value of the subband, or by reducing the energy level of the subband having the smallest energy size, or by partially reducing the energy level of one subband, and increasing the energy level of the other subband. Allow the energy difference of the band to be above (or above) the difference.

That is, in the case of the embodiment of FIG. 5A, all the frames F2, F3, and F4 after the first frame F1 are subjected to energy deviation standardization. The result of this energy deviation normalization may refer to FIG. 5B.

The energy deviation standardization process prevents the additional information from being extracted properly after the energy size difference between the subframes is too small to effectively insert the signal for indicating the additional information.

The information inserting unit 105 inserts additional information of a digital signal, and the additional information is inserted through energy modulation of subbands P1, P2, and P3. Before explaining the operation of inserting additional information into the information insertion section 105, the components for processing the additional information before insertion will be described first.

The additional information for insertion is encoded by the binarization unit 110 into a digital signal. The additional information, which is a digital signal in the binarization unit 110, is encoded by the error correction encoder 111 as an error correction code. In this embodiment, the error correcting code uses BCH codes (Bose, Chaudhuri, Hocquenghem Code). However, other error correction codes may be used.

The header insertion unit 112 inserts a header at the front of the additional information to generate a digital sequence for the additional information to be finally inserted. In this case, the header is a fixed PN sequence (Pseudo Random Noise Sequence) and the system for extracting additional information should know this PN sequence. When the digital sequence of the additional information is completed as described above, the digital sequence is transferred to the information inserting unit 105.

When the digital sequence of the additional information is provided and the energy deviation standardization for each frame is completed, the information inserting unit 105 then inserts the additional information into the sound signal.

Information insertion of the information insertion unit 105, as shown in Figure 5b, when the digital signal of the information to be inserted "0" as shown in the first frame (F1) and the second frame (F2) as shown in The energy level of the subband is lowered to the same state as the energy level of the subband having the smallest energy level. When the information to be inserted is "1", the third and fourth frames F3 and F4 Likewise, the energy size of the subband having the medium energy size is increased to be the same as the energy size of the subband having the largest energy size. Or insert each digital signal in the opposite way.

In this insertion method, all digital sequences of additional information are inserted in sequence for each subband of each frame. When the additional information is inserted by such an insertion method, the subbands may be divided into a large amount according to the length of the additional information. Accordingly, a large amount of digital data is inserted into one frame. That is, for example, when eight subbands are divided in one subframe, one byte of a digital signal may be stored in one subframe.

The sound signal may be a mono signal or a stereo signal. Therefore, if the audio signal is a mono signal, as described above, information insertion is performed, but in the case of a stereo signal, processing for the left and right two signals is required.

FIG. 6A is a diagram illustrating energy levels of subbands before inserting additional information into a stereo sound signal, and FIG. 6B is a diagram illustrating energy levels of subbands after inserting additional information into a stereo sound signal. One drawing.

As shown in FIGS. 6A and 6B, when the acoustic signal is a stereo signal, an energy difference for each subband is selected among the left and right signals to insert information in the same manner as the information insertion method for the mono signal. In this embodiment, since the energy difference on the right side is illustrated as being larger, information is inserted into the sound signal on the right side. The signal on the left side forces the correlation of energy magnitude in each frame to be equal to the correlation on energy magnitude on the right side.

The Fourier inverse transform unit 120 performs Fourier inverse transform on each subband, and the subframe combiner 121 combines the Fourier inverse transformed subbands for each subframe. Each subframe is combined by the frame combiner 122 and finally output as an audio signal.

As described above, when the sound signal in which the additional information is inserted is output, the listener who listens to the additional information may extract the additional information from the corresponding sound signal by using the mobile device 11 having the system for extracting the additional information.

Hereinafter, a system for extracting additional information embedded in an audio signal will be described. FIG. 7 is a block diagram of a system for extracting additional information from a sound signal in which additional information is inserted according to the present embodiment.

As shown in FIG. 7, the system includes a frame synchronizer 200 that finds the starting point of the minimum fully preserved frame in the acoustic signal. The frame synchronizer 200 skips an early part of an audio signal, an initial skip unit 201, a synchronization frame division unit 202, a synchronization subframe division unit 203, and a synchronization Fourier transform unit 204. ), A synchronization subband divider 205, a frame sync scoring unit 206, and a frame sync alignment unit 207.

When the frame synchronization is completed in the synchronization unit 200, the system actually extracts additional information from the sound signal. The frame divider 210, the subframe divider 211, the Fourier transform unit 212, and the subband divider 213, an information extraction unit 214, a header detection unit 215, an error correction decoding unit 216, and a binarization recovery unit 217.

Hereinafter, each structure and the effect | action of these structures are demonstrated concretely.

The initial skip unit 201 of the synchronization unit 200 samples the acoustic signal and skips by "n" samples from the initial sample. At this time, "n" starts at "0" and repeatedly increases until it becomes one frame length.

When skipping is performed in the initial skip unit 201, the synchronization frame dividing unit 202 divides the sound signal into a plurality of frames on each time axis. The synchronization subframe divider 203 divides each frame into three subframes. In addition, the synchronization Fourier transform unit 204 performs Fourier transform on each subframe. The subband dividing unit 205 divides a predetermined frequency band (ie, a frequency band used for inserting additional information) after the Fourier transform into a plurality of subbands.

The frame sync scoring unit 206 calculates the energy of each subframe in the order of the larger energy levels by comparing the energy magnitudes of the respective subbands corresponding to each subframe. In this case, when the energy size of the subband having the intermediate energy size is close to the energy size of the subband having the largest energy size or the subband having the smallest energy size, a relatively high score is given. On the other hand, a relatively low score is given when the energy magnitude of the subband having the intermediate energy magnitude is close to the midpoint of the energy magnitude of the subband having the largest energy magnitude or the subband having the smallest energy magnitude. The final score is calculated by summing the scores calculated for each subband.

This process is repeated until it becomes one frame length. The frame sync alignment unit 207 finds a sampling position where the final score is maximum, and calculates an initial skip length. Frame synchronization is completed by skipping the first part of the signal by this initial skip length.

When frame synchronization is completed, the frame dividing unit 210 divides the sound signal in units of frames from the synchronization position. The subframe divider 211 divides each frame into three subframes. In addition, the Fourier transform unit 212 performs Fourier transform on each subframe, and the subband divider 213 divides the frequency band used when the additional information is inserted into the Fourier transformed frequency band into a plurality of subbands.

Subsequently, the information extracting unit 214 extracts a digital signal by comparing energy levels for each subband. Extraction of the digital signal calculates the energy magnitude of the corresponding subband of each subframe and distinguishes each subband in the order of the largest energy magnitude. At this time, if the subband having the middle size is close to the energy size of the subband having the largest energy size, it is extracted as "1", and if it is close to the energy size of the subband having the small energy size, it is extracted as "0" or the In the opposite way, a digital signal is extracted and a binary string is created by listing the extracted digital signals for each frame.

Alternatively, in another embodiment, the threshold value for the energy level of the subband having the intermediate size is preset, and it is determined that the subband having the intermediate size is above or below this threshold to determine and extract the digital signal of the additional information. can do.

 The header detector 215 finds the closest part of the PN sequence inserted into the header of the extracted binary string to find the starting point of the inserted information. The error correction decoding unit 216 recovers the additional information expressed in binary form by performing error correction decoding starting from the start point of the inserted information. The binarization restoration unit 217 recovers the additional information finally inserted into the sound signal by restoring the restored additional information to the original information form.

Then, the restored system is displayed through a display or other output or display device such as the wireless device 11 interworking with the system, so that the user acquires additional information on the corresponding sound signal.

On the other hand, in the configuration of the additional information extraction system described above, the synchronization frame divider 202 and the frame divider 210, the synchronization subframe divider 203 and the subframe divider 211, and the Fourier transform for synchronizer 204 and Fourier transform 212, and the synchronization subband divider 205 and subband divider 213 are implemented in one signal processing configuration in the system so that one configuration achieves two different purposes. It can serve as a configuration for. That is, these configurations are functionally divided for convenience of understanding in this embodiment, and the hardware may be the same component, and other components may be implemented by merging or dividing the hardware for implementation.

In the experimental example using the system and the insertion method for inserting additional information into the acoustic signal according to the present embodiment as described above, the result of experimenting by separating the left and right sides of the "egosys nEar 05" speaker by 50cm and setting the bit rate to 84.8bps Additional information included in the acoustic signal could be obtained at a distance of 5 m or more forward.

Therefore, the system and extraction method for extracting additional information from the acoustic signal of the present embodiment inserts and transmits additional information related to music, audiobooks, telephone calls, and the like when the audio signal is transmitted to the street and various public transportations. By extracting and obtaining additional information using the wireless device 11 or the like from the acoustic signal output from the speaker such as a means and a public place, it is possible to enjoy a more improved mobile environment.

FIG. 1 is a diagram for describing an example of acquiring additional information about an acoustic signal being listened to by a wireless device.

2 is a configuration diagram of a system for inserting additional information into an acoustic signal according to the present embodiment.

3 is a diagram illustrating an example of dividing an acoustic signal into a frame and a subframe according to the present embodiment.

4 is a diagram illustrating an example of dividing a subband after Fourier transforming each subframe according to the present embodiment.

FIG. 5A is a diagram illustrating energy levels of subbands at a specific position for each subframe before inserting additional information into an acoustic signal. FIG.

5B is a diagram illustrating energy levels of subbands in a specific position for each subframe after additional information is inserted into an acoustic signal.

FIG. 6A is a diagram illustrating energy levels of subbands before inserting additional information into a stereo sound signal. FIG.

6B is a diagram illustrating energy levels of subbands after inserting additional information into a stereo sound signal.

FIG. 7 is a block diagram of a system for extracting additional information from a sound signal in which additional information is inserted according to the present embodiment.

Claims (9)

A synchronization unit configured to receive the sound signal from a speaker to which the sound signal into which the additional information as a digital signal is inserted through energy modulation is output and to find a starting point for the additional information inserted into the received sound signal; A frame dividing unit dividing the received sound signal into a plurality of frames; A subframe dividing unit dividing the frame into a plurality of subframes; A Fourier transform unit for Fourier transforming each subframe; A subband dividing unit dividing a frequency band of the Fourier transformed subframe into a plurality of subbands; And an information extraction unit for comparing the energy levels of the subbands and extracting additional information included in the sound signal. 2. The information extractor of claim 1, wherein the information extracting unit comprises a high subband having the highest energy level and a low subband having the lowest energy level and the high subband according to the order of the energy sizes of the subbands. And an intermediate subband having an intermediate energy magnitude of the low subband, and adding the intermediate subband according to whether the energy magnitude of the intermediate subband is close to the energy magnitude of the high subband or the energy magnitude of the low subband. A system for extracting additional information from an acoustic signal, characterized in that for dividing and extracting the digital signal of the information. The system of claim 2, wherein the synchronization unit finds a starting point of the additional information by checking an energy level of the intermediate subband. 4. The apparatus of claim 3, wherein the synchronization unit comprises: an initial skip unit for skipping a predetermined length at an initial position of the sound signal; a synchronization frame divider for dividing the sound signal from the skipped position into a plurality of frames; A subframe division for synchronizing the subframe into a plurality of server frames, a Fourier transform for synchronizing the subbands, and a subband for synchronizing the frequency bands of the Fourier transformed subframes into a plurality of subbands And a sink scoring unit for comparing the energy levels of the subbands to perform scoring on the frames. The method of claim 4, wherein the sync scoring unit determines that the digital signal is inserted into the subframe when the intermediate subband is close to an energy level of the high subband or the low subband, and scores a relatively high score. And if the intermediate subband approaches a midpoint of the energy level of the high or low subband, scoring with a relatively low score. Receiving the sound signal from a speaker outputting the sound signal inserted by the energy modulation of the digital signal, and synchronized to find a starting point for the digital signal inserted in the sound signal, and the received sound signal Divide the frame into a plurality of frames, divide the frame into a plurality of subframes, Fourier transform each of the subframes, divide a frequency band of the Fourier transformed subframe into a plurality of subbands, and divide the subbands. According to the order of energy magnitude, among the energy magnitudes of the subbands, a high subband having the highest energy magnitude and a low subband having the lowest energy magnitude, and an intermediate subband having an intermediate energy magnitude of the high subband and the low subband. , The energy magnitude of the intermediate subband And dividing the digital signal according to whether the energy level of the high subband or the energy level of the low subband is close. 7. The method of claim 6, wherein the synchronization finds the starting point of the digital signal by checking the energy level of the intermediate subband. 8. The method of claim 7, wherein the synchronization checks the energy level of the intermediate subbands from a position where a predetermined sampling position is skipped at an initial sampling position of the acoustic signal. . 8. The method of claim 7, wherein when the intermediate subband is close to the energy level of the high subband or the low subband, it is determined that the digital signal is inserted into the subframe. How to.

Images