JP2009044365A - Voice processor and processing method, mobile terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable execution of compensating procedure for sound leakage when the sound leaks from a headphone. <P>SOLUTION: A voice processing unit 35 of the voice processor supplies reproduced voice data to the headphone 33. An external microphone 34 acquires voices from the surroundings to convert these voices to voice data. A control unit 10 executes, for the reproduced voice data supplied to the headphone 33, the processing to extract the frequencies that leak easily and the time conversion processing to consider a delay time until the voice leaking to the external side from the headphone 33 reaches the microphone 34. Moreover, the control unit 10 also extracts the frequencies that leak easily among the voice data in the surroundings from the microphone 34 and detects that the sound leaks from the headphone 33 when the reproduced voice data completing the processing is almost matched with the voice data in the surroundings. Thereafter, the control unit 10 executes the processing to compensate leak of the sound. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a sound processing apparatus and method for outputting reproduced sound such as music through headphones, and a portable terminal.

  Conventionally, for example, when listening to music played by an audio playback device with headphones, for example, the user can enjoy the music. However, for example, when listening to music in a public place such as a public place, the playback sound may leak out to the surroundings depending on the performance of the headphones. May be caused.

  In JP-A-8-314473, an open patent publication (Patent Document 1) converts a sound signal of music into sound waves by a speaker and outputs the sound wave, and a sound signal whose phase is inverted from another plurality of speakers. By converting the sound into sound waves and outputting them, the sound leaking out of the reproduction sound field is canceled, and for the audio signals supplied to the other speakers, phase adjustment by a delay circuit and volume adjustment by an amplifier A soundproofing device is disclosed in which the acoustic characteristics of the sound field can be freely set by applying the above.

  In Japanese Patent Laid-Open No. 3-247098 (Patent Document 2), the sound leakage band of the audio signal supplied to the headphones is attenuated, and at the same time, the attenuation amount is made variable according to the level of the audio signal. Accordingly, a portable playback device and headphones that make it possible to reduce sound quality deterioration while making it difficult for surrounding people to hear sound leaking from the headphones are disclosed.

JP-A-8-314473 (FIG. 1) Japanese Patent Laid-Open No. 3-247098 (FIG. 1)

  By the way, when listening to music, etc. with headphones, it is almost always the case that there is a person near the person and the sound from the headphones is unpleasant. It happens because it is leaking and the person is not aware of it. That is, even if the sound leaks from the headphones, for example, if there is no person near the person, it will not cause trouble to others, and for example, even if there is a person near the person, If the person notices that the sound is leaking from the headphones, and the person himself takes corrective measures against the sound leakage (for example, a measure such as lowering the volume), the surrounding people can be prevented from being disturbed. However, when listening to music or the like with headphones, it is very often not noticed at all until the surrounding people point out that the sound is leaking.

  Further, if a sound leakage prevention technique for reducing sound leakage itself is used as in the above-mentioned Japanese Patent Application Laid-Open Nos. 8-314473 and 3-247098, it is less likely to inconvenience surrounding people. However, in reality, there are many cases where the sound leaks, and in this case, the person who listens to the music does not notice that the sound leakage makes the surrounding people feel uncomfortable.

  The present invention has been proposed in view of such circumstances, and when listening to music or the like with headphones, it is possible to take corrective measures for the sound leakage when the sound is leaking from the headphones. An object is to provide a voice processing apparatus and method, and a portable terminal.

  In the audio processing apparatus and method of the present invention and the portable terminal, the audio data supply unit supplies audio data to the headphones, and the sound leak detection unit detects the sound leaked to the outside from the headphones attached to the ear. When the sound leakage from the headphones to the outside is detected, the sound leakage correction processing unit executes a predetermined sound leakage correction process to solve the above-described problem.

  That is, according to the present invention, when sound is output from a headphone worn on the ear, a phenomenon in which sound leaks from the headphone to the outside is obtained by acquiring peripheral sound with a microphone. It can be detected.

  In the present invention, when listening to music or the like with headphones, sound leakage from the headphones can be detected, and when sound leakage from the headphones is detected, corrective measures are taken against it. It is possible to avoid a situation in which the surrounding person feels uncomfortable due to sound leakage that is not intended by the user.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  In the following embodiments, a mobile phone terminal is cited as an application example of the voice processing apparatus and method and the mobile terminal of the present invention. Of course, the content described here is merely an example, and the present invention is not limited to this. Needless to say, the examples are not limited.

[Schematic internal structure of mobile phone terminal]
FIG. 1 shows a schematic internal configuration of a mobile phone terminal according to an embodiment of the present invention.

  In FIG. 1, a communication antenna 12 is, for example, a built-in antenna, and transmits and receives signal radio waves for calls and packet communications. The communication circuit 11 performs frequency conversion, modulation, demodulation, and the like of transmission / reception signals. The call voice data received by the communication antenna 12 and the communication circuit 11 is sent to the voice processing unit 35 via the data line, and the other received data is once sent to the control unit 10 and processed. Thereafter, it is sent from the control unit 10 to each unit as necessary.

  The built-in speaker 30 is an output speaker such as a receiving speaker or a ringer (ringing tone), alarm sound, warning sound, voice, music, etc. provided in the mobile phone terminal, and includes a digital / analog converter and an amplifier. Voice data such as call voice, ringer sound, music, etc. are output after digital / analog conversion and amplification. As a result, voices such as call voices, ringer sounds, and music are output to the air.

  The built-in microphone 31 is a microphone for transmission and external sound collection provided in a mobile phone terminal, and includes an analog / digital converter and an amplifier. The audio signal input via the built-in microphone 31 is amplified to a predetermined level by an amplifier, converted to digital audio data by an analog / digital converter, and sent to the audio processing unit 35 via a data line. After being encoded, the data is sent to the communication circuit 11 via the data line, subjected to various processes such as modulation and frequency conversion, and transmitted from the communication antenna 12.

  The external audio terminal 32 is a terminal for connecting an external headphone 33 or an external microphone 34 to the mobile phone terminal main body. For example, the external audio terminal 32 can be used to adjust the volume of playback sound such as music, fast-forward playback, fast-rewind playback, channel switching, and a remote command for performing speech instructions during a call using the external microphone 34 and headphones 33. A control terminal is also included.

  The digital broadcast receiving section 38 includes a digital broadcast receiving tuner section and an AV codec section. The digital broadcast receiving section 38 demodulates a digital television broadcast signal and a digital radio broadcast signal received by the digital broadcast antenna 39, analog / digital conversion, Processing such as decompression decoding of the compression-encoded data is performed, and among the digital broadcast data demodulated and decompressed, the moving image data of the digital television broadcast is sent to the image processing unit 36, and the digital television broadcast is transmitted. Audio data and digital radio broadcast audio data are sent to the audio processing unit 35.

  When the compressed encoded data of the received voice is supplied from the communication antenna 12 and the communication circuit 11, the voice processing unit 35 decompresses and decodes the compressed encoded data of the received voice, and receives the received voice data after the decompression decoding. Is sent to the speaker 30 via the data line or supplied to the headphones 33 connected to the external audio terminal 32. In addition, the operation mode of the mobile phone terminal according to the present embodiment is a reproduction mode of music data content (music file) or moving image data content (moving image file) that is compressed and stored in the memory unit 16 or the like, for example. In the case where it is made, the audio processing unit 35 performs decompression decoding of the audio data of the music data content and the moving image data content read from the memory unit 16 under the control of the control unit 10, and performs the decompression decoding. The reproduced audio data after conversion is sent to the speaker 30 via the data line or to the headphones 33 connected to the external audio terminal 32. Further, when the operation mode of the mobile phone terminal of this embodiment is, for example, a digital broadcast reception and viewing mode, the audio processing unit 35 is supplied from the digital broadcast reception unit 38 under the control of the control unit 10. The audio data of the digital television broadcast or digital radio broadcast is sent to the speaker 3 via the data line or supplied to the headphones 33 via the external audio terminal 32. It should be noted that the function of the voice processing unit 35 can be realized by software by the control unit 10.

  The display unit 13 includes a display device such as a liquid crystal display or an organic EL (Elector Luminescence) panel and a display drive circuit for the display, and displays characters, various messages, still images, moving images, and the like on the display.

  The image processing unit 36 generates image data such as a standby screen, an e-mail message, an address book, various user interface screens, a moving image data playback moving image, a digital television broadcast received moving image, and the like. Is sent to the display of the display unit 13 via the data line and displayed. Note that the function of the image processing unit 36 can also be realized by software by the control unit 10.

  The operation unit 14 includes keys such as a numeric keypad, an utterance key, and an end / power key provided on a casing (not shown) of the mobile phone terminal according to the present embodiment, and an operator such as a cross key and a jog dial. An operation signal generator that generates an operation signal when the operation element is operated. In the present embodiment, each operation element of the operation unit 14 is displayed on the display by a user on / off operation of a power source, operation for instructing outgoing / incoming calls, character number input operation such as telephone number input or e-mail, and the like. It is used for a menu item selection operation, a shutter operation in a digital camera function, a digital broadcast reception instruction operation, a reproduction instruction operation such as music data content or moving image data content, and related instruction operations.

  The clock unit 15 generates time information such as date and time, time information used when the control unit 10 performs various processes, and the like. The time, time, and the like by the clock unit 15 are displayed on a display, used for schedule management, alarm notification, and the like, and are also used for management of playback time and playback date and time of music data content, for example.

  The vibrator 37 rotates / stops a small motor having an eccentric weight fixed to the rotation shaft, thereby generating vibrations in the mobile phone terminal according to the present embodiment.

  The memory unit 16 includes a ROM (Read Only Memory) and a RAM (Random Access Memory). The ROM includes a rewritable storage medium such as a NAND-type flash memory. For example, an OS (Operating System) program and a control program for the control unit 10 to control each unit, for example, music Audio reproduction management and control program for audio reproduction output management and control, such as audio reproduction output of data contents and the like, audio reproduction output of digitally broadcast music data, etc., sound leakage detection and sound leakage described later according to the present invention Various application programs including programs (sound leakage detection and control programs) for executing processing such as notification and control for corrective actions, compression-coded music data content and moving image data content, as well as various Stores default values, font data, dictionary data, model name information, terminal identification information, etc. That. The RAM stores data as needed as a work area when the control unit 10 performs various data processing. The memory unit 16 may include a removable external memory card or a small hard disk drive. In the present embodiment, the content storage unit 18 of the memory unit 16 stores compression-coded music data content, moving image data content, and the like. The music data content and moving image data content stored in the content storage unit 18 include those acquired via a communication line, those received by the digital broadcast receiver 38, and those stored in an external memory card. . In the program storage unit 17 of the memory unit 16, various application programs including the sound leakage detection and control program of this embodiment are stored. In addition, the sound leak detection parameter storage unit 19 stores information on “sound leaky sound frequency” that is used when processing such as notification and control for sound leak detection and sound leak correction measures, which will be described later, is performed. Also, information on “sound arrival delay time” according to the spatial distance between the headphones 33 and the external microphone 34, information on “minimum volume” that requires sound leakage correction measures, and the like are parameters for sound leakage detection. Is stored as The extracted / converted playback audio data storage unit 20 extracts a “voice frequency that is likely to leak sound” from the sound data of music or digital broadcast and detects the sound arrival delay time when detecting sound leak described later. Prepared to temporarily store the audio data (reproduced audio data after extraction / conversion).

  The control unit 10 controls communication in the communication circuit 11 and controls each unit such as the audio processing unit 35, the image processing unit 36, and the digital broadcast receiving unit 38. In particular, in the case of the present embodiment, although details will be described later, the control unit 10 executes sound leakage detection and control programs stored in the program storage unit 17 of the memory unit 16, thereby performing sound such as music. Is output from the headphone 33 via the external audio terminal 32, processing such as detection and control for detection of unnecessary sound leakage from the headphone 33 and corrective measures for it is executed.

  In addition, although not shown in FIG. 1, the mobile phone terminal of the present invention includes a digital camera unit for taking a photographic image, an LED (light emitting diode) for key illumination and incoming light, and its driving unit. , A battery that supplies power to each unit, and a power management IC unit that controls the power, so-called Bluetooth method (Bluetooth: registered trademark), UWB (Ultra Wide Band) method, wireless LAN (Local Area Network), etc. For general mobile phone terminals such as short-range wireless communication units for information processing, electronic wallets for information storage and transmission / reception of signals for electronic money, GPS (Global Positioning System) communication units, external memory slots, etc. It is provided also about each component provided.

[Sound leak detection, sound leak corrective action notification, control explanation]
Hereinafter, notification and control for sound leakage detection and sound leakage correction measures in the mobile phone terminal of the present embodiment will be described using the configuration of FIG. 1 and the flowcharts of FIGS. Note that the processing shown in the flowcharts of FIGS. 2 to 4 includes the sound reproduction management and control program and sound leakage detection and control of this embodiment stored in the program storage unit 17 of the memory unit 16 of the mobile phone terminal of this embodiment. This is a process realized by the control unit 10 executing the program.

[Outline of each parameter]
First, in the present embodiment, for example, in the product manufacturing stage of a mobile phone terminal, the parameters of “sound arrival delay time”, “sound frequency that is likely to leak sound”, and “minimum volume” are determined and those parameters are stored in the memory unit 16. It is stored in the sound leak detection parameter storage unit 19.

  The “sound arrival delay time” parameter is information determined in advance based on the distance between the headphone 33 and the external microphone 34 in actual use. Specifically, the sound output from the headphone 33 such as music is output. Is the time information until the sound leaking from the headphone 33 reaches the external microphone 34. That is, when a certain sound is output from the headphones 33 and detected by the external microphone 34, the difference between the time when the sound is output from the headphones 33 and the time when the sound reaches the external microphone 34 is the above-mentioned “sound. It is determined as “arrival delay time”. The parameter of “voice arrival delay time” is determined at the product manufacturing stage of the mobile phone terminal, and is stored in the sound leak detection parameter storage unit 19 of the memory unit 16.

  In the above description, an example in which the “voice arrival delay time” is determined at the product manufacturing stage is described. However, the mobile phone terminal according to the present embodiment is provided between the headphones 33 and the external microphone 34 in actual use. When the distance changes, a function of detecting the change in the distance and appropriately obtaining the “voice arrival delay time” based on the detected distance may be provided. As an example for realizing such a function, the cord of the headphone 33 and the external microphone 34 is a winding type, and when the cord is actually used, the cord between the headphone 33 and the external microphone 34 can be wound and pulled out. In such a case, the distance between the headphones 33 and the external microphone 34 is measured based on, for example, the winding amount or the drawing amount of the cord, and the “voice arrival delay” is determined based on the measured distance information. A process for appropriately calculating “time” can be mentioned. In addition, for example, when the “voice arrival delay time” fluctuates due to atmospheric pressure, humidity, temperature, etc., a barometer, hygrometer, thermometer, etc. are installed in the mobile phone terminal, and the measured values are adjusted accordingly. Then, a process for appropriately calculating the “voice arrival delay time” may be performed.

  The parameter “sound frequency at which sound leaks easily” is a sound that is likely to leak out from the headphone 33 (easy to leak out from the ear) when the headphone 33 is actually used, that is, for example, inserted into the user's ear. This is information representing a frequency band. Here, the audio frequency band that is likely to leak to the outside from the headphones 33 inserted in the user's ear is actually quite wide, but in particular, the sound that the surrounding people feel uncomfortable is in the high frequency band. Sound (high sound). For this reason, in the present embodiment, as an example, a high frequency of 4 kHz or more is determined as the above-mentioned “voice frequency at which sound leaks easily”. The parameter “sound frequency at which sound is likely to leak” is determined at the product manufacturing stage of the mobile phone terminal, and is stored in the sound leak detection parameter storage unit 19 of the memory unit 16.

  The above 4 kHz is an example, and may be a frequency band higher or lower. In the above description, an example is described in which the “voice frequency at which sound leaks easily” is determined at the product manufacturing stage. As an example, the volume of sound leaking from the headphones 33 to the outside or the volume of ambient environmental sound is described. If the frequency of the sound that the surrounding people feel uncomfortable changes, the above-mentioned “sound leaking sound frequency” is appropriately changed based on the volume of the leaked sound and the ambient environmental sound. Also good.

  The “minimum volume” parameter determines whether to operate the sound leakage detection and sound leakage correction function (whether the function is set to “valid”) in the mobile phone terminal of the present embodiment. Specifically, when 90 dB is set as a threshold and the volume of sound output from the headphones 33 is equal to or higher than the threshold, the sound leakage detection and sound leakage correction functions are operated. On the other hand, when the volume is lower than the threshold, the information is for controlling the sound leakage detection and sound leakage correction functions not to operate. That is, when the volume of the sound output from the headphones 33 is very low and it can be determined that the sound leakage does not make the surrounding person uncomfortable, the sound leakage detection and sound leakage correction functions are not operated. Thus, the person who listens to the music from the headphones 33 can concentrate on the music. The “minimum volume” parameter is determined at the product manufacturing stage of the mobile phone terminal and is stored in the sound leakage detection parameter storage unit 19 of the memory unit 16.

  Note that whether or not the sound leaking from the headphones 33 gives discomfort to the surrounding people changes depending on the volume of the surrounding environmental sound at that time, and therefore, the “minimum volume” is based on the volume of the surrounding environmental sound. May be appropriately changed. That is, in the above description, an example in which the “minimum volume” is determined at the product manufacturing stage is described, but as an example, if the ambient environmental sound volume is very high, the threshold value of the “minimum volume” is increased. Conversely, when the volume of the ambient environmental sound is very low, the threshold value of the “minimum volume” may be changed.

  In addition, the ambient sound may be acquired not only by the external microphone 34 but also by the built-in microphone 31.

[Explanation of processing flow]
Next, in the mobile phone terminal of the present embodiment, the headphone 33 and the external microphone 34 are connected to the external audio terminal 32 and the control unit 10 is configured to use the memory unit 16 at the time of actual use for outputting sound such as music from the headphone 33. By executing the audio reproduction management and control program of the present embodiment stored in the program storage unit 17, the audio reproduction related process as shown in the flowchart of FIG. 2 is executed. Further, when executing the audio reproduction process, the control unit 10 executes the sound leakage detection and control program of the present embodiment stored in the program storage unit 17 of the memory unit 16 to thereby execute the flowcharts of FIGS. 3 and 4. The sound leak detection and sound leak correction related processing as shown in FIG.

[Audio playback processing flow]
In the mobile phone terminal according to the present embodiment, the voice reproduction management and control program is activated, and for example, a user selects a desired music file from a plurality of music files from a plurality of menu items displayed on the menu selection screen. Assuming that the menu item to be selected and played is selected, the control unit 10 first responds to all music data contents stored in the content storage unit 18 of the memory unit 16 as the processing of step S1 in the flowchart of FIG. In step S2, the image processing unit 36 is controlled to generate an image of the music file list, and the music file list image is displayed on the display screen of the display unit 13. Display.

  Next, when the user selects a desired music file from the music file list via the operation unit 14 via the operation unit 14 and further inputs an instruction to start playback of the music file via the operation unit 14. In step S3, the music data content corresponding to the music file selected by the user is read from the content storage unit 18 and the audio processing unit 35 is controlled to perform decompression decoding of the music data content. Thus, the reproduction process of the music file is started.

  Then, the control unit 10 controls the audio processing unit 35 to execute the music file reproduction process as step S4, and sends the reproduced music data to the headphones 33 through the external audio terminal 32. Thus, the reproduced music is output from the headphones 33, and the user can enjoy the music of the desired music file (music data content).

  Furthermore, the control unit 10 determines whether or not the music file has been played back as the process of step S5 during the process of step S4, or whether or not an instruction such as the end of playback has been input from the user through the control unit 14. When the music file has been played back or when an instruction input such as playback end has been made by the user, the processing of the flowchart of FIG.

[Sound Leakage Countermeasure Flow (Playback Audio Processing)]
When the process of the flowchart shown in FIG. 2 is started, the control unit 10 outputs, for example, music from the headphones 33 as the process of step S11 of the flowchart of FIG. 3 at least before the process of step S3 of FIG. Sound leakage prevention function (sound leakage detection and sound leakage correction function) is set in advance to determine whether or not it has been set, and the sound leakage countermeasure setting is turned on (enabled) The process proceeds to step S12 and subsequent steps only when it is determined that, otherwise the process of the flowchart of FIG. 3 is terminated.

  When it is determined in step S11 that the sound leakage countermeasure setting is ON (valid) and the process proceeds to step S12, the control unit 10 reads “minimum volume” from the sound leakage detection parameter storage unit 19 of the memory unit 16. ”And the current setting value of the volume for the current voice output of the mobile phone terminal, it is determined whether or not the output volume based on the current volume setting is equal to or greater than the“ minimum volume ”threshold. If it is determined by the determination processing in step S12 that the output volume according to the current volume setting is lower than the threshold value of the “minimum volume”, even if sound is output with the volume setting, the sound leaks to surrounding people. Since it is considered that there is little unpleasant feeling, the control part 10 complete | finishes the process of the flowchart of the said FIG. On the other hand, if it is determined by the determination processing in step S12 that the output volume based on the current volume setting is equal to or greater than the threshold value of the “minimum volume”, if sound is output with the volume setting, sound leaks. Since there is a possibility that surrounding people may feel uncomfortable, the control unit 10 advances the processing to step S13 and subsequent steps.

  When the process proceeds to step S13, the control unit 10 executes a music file reproduction process, that is, the process of step S4 in the flowchart of FIG.

  Further, when the process of step S13 is started, the control unit 10 reads the parameter of “frequency at which sound leaks easily” from the sound leak detection parameter storage unit 19 of the memory unit 16 as the process of step S14. Based on the parameter “sound frequency that is likely to leak sound”, a process of extracting the “sound frequency that is likely to leak sound” portion from the reproduced sound data by the music file reproduction process in step S13.

  Next, as the process of step S15, the control unit 10 reads the parameter “speech arrival delay time” from the sound leak detection parameter storage unit 19 of the memory unit 16, and extracts the extracted voice data obtained by the process of step S14. Then, a time conversion process taking into account the “speech arrival delay time”, that is, a time conversion process for delaying time information attached to the extracted voice data by the “speech arrival delay time” is performed.

  Then, as the process of step S16, the control unit 10 extracts the extracted audio data (extracted / converted reproduced audio data) after the time conversion process in step S15, and the extracted / converted reproduced audio data storage unit 20 of the memory unit 16 To store. The data stored in the extracted / converted reproduced audio data storage unit 20 is only the latest extracted / converted reproduced audio data for a certain period of time, and old data is overwritten by new data. Is desirable. That is, if all of the extracted / converted playback audio data is stored, the capacity of the memory unit 16 is compressed, so that in step S26 in the flowchart of FIG. By storing only data for a certain period of time necessary for determining whether or not to implement sound leakage countermeasures by comparing the above, it is possible to avoid wasting the capacity of the memory unit 16 wastefully. Is desirable.

  The processing of the flowchart shown in FIG. 3 is repeated until playback of the music file is completed or an instruction to finish playback is input from the user through the control unit 14 in step S5 of the flowchart of FIG. become.

  In the flowchart of FIG. 3, either of the processes of step S14 and step S15 may be performed first. That is, the time conversion process may be performed on the reproduced audio data first, and then the frequency portion may be extracted.

[Sound leakage countermeasure flow (peripheral voice side processing)]
Further, the control unit 10 executes the process of the flowchart of FIG. 4 in parallel with the execution of the process of the flowchart of FIG. Note that step S21 and step S22 in the flowchart of FIG. 4 are the same processing as step S11 and step S12 in the flowchart of FIG. 3 and are performed at the same time, and are not performed separately from these steps S11 and S12. However, for the convenience of explanation, FIG. 4 describes them as step S21 and step S22.

  In the flowchart of FIG. 4, the control unit 10 takes measures for sound leakage when outputting music or the like from the headphones 33 as the process of step S21 of the flowchart of FIG. 4 at least before the process of step S3 of FIG. 2. It is determined whether or not the setting for enabling the function (sound leakage detection and sound leakage correction function) has been made in advance, and only when it is determined that the sound leakage countermeasure setting is set to ON (enabled), step S22 and subsequent steps. In other cases, the process of the flowchart of FIG. 4 is terminated.

  Further, when it is determined in step S21 that the sound leakage countermeasure setting is ON (valid) and the process proceeds to step S22, the control unit 10 reads from the sound leakage detection parameter storage unit 19 in the memory unit 16 " The parameter of “minimum volume” is read out, and it is determined from the current setting value of the volume for the current voice output of the mobile phone terminal whether the output volume by the current volume setting is equal to or higher than the threshold value of “minimum volume”. If it is determined by the determination processing in step S22 that the output volume based on the current volume setting is lower than the “minimum volume” threshold, the processing in the flowchart of FIG. If it is determined that the output volume by is equal to or greater than the threshold value of the “minimum volume”, the process proceeds to step S23 and subsequent steps.

  In step S23, the control unit 10 sets the external microphone 34 to valid (ON), and then proceeds to step S24.

  When the process proceeds to step S24, the control unit 10 takes in the peripheral audio data acquired by the external microphone 34, and proceeds to step S25.

  When the processing proceeds to step S25, the control unit 10 reads out the parameter “Frequency of sound leakage” from the sound leakage detection parameter storage unit 19 of the memory unit 16, and sets the parameter of “Frequency of sound leakage”. Based on the ambient audio data acquired by the external microphone 34 in step S24, the process of extracting the “audio frequency that is likely to leak sound” portion is performed.

  Next, in step S26, the control unit 10 acquires the external microphone 34 from the extracted / converted playback audio data stored in the extracted / converted playback audio data storage unit 20 of the memory unit 16. The data having the same time information as the reproduced sound (reproduced sound that has been leaked) included in the peripheral sound, that is, the delay time from the headphone 33 to the external microphone 34 is added. The extracted / converted reproduced audio data is read, and it is analyzed whether or not the extracted / converted reproduced audio data is included in the extracted peripheral audio data obtained in step S25. Note that the analysis process can be realized, for example, by a matching determination process using pattern matching or the like of both data.

  If the control unit 10 determines in step S26 that the extracted peripheral audio data includes the extracted / converted reproduced audio data, that is, the headphone is included in the peripheral audio accumulated by the external microphone 34. When it is determined that a sound component leaking out from 33 to the outside and including a sound component that takes into account the arrival delay time from the headphone 33 to the external microphone 34 is included, sound leakage has occurred. And the process proceeds to step S27.

  When the processing proceeds to step S27, the control unit 10 performs sound leakage countermeasure processing for improving the state of sound leakage. Specifically, a process for automatically lowering the volume output from the headphones 33 to a level at which sound does not leak, a process for automatically attenuating a frequency portion that is likely to leak sound, and a reproduced sound waveform that leaks to the outside due to sound leak The process of generating a sound waveform with an antiphase on the headphone 33, the sound leakage state is displayed on the screen of the display unit 13, the alarm sound is output, the vibration of the vibrator 37, etc. are notified so that the user can recognize them. Perform processing such as. Note that the notification to the user by screen display, alarm sound output, vibrator vibration, etc. is a notification for prompting the user to correct the sound leakage (for example, operating the volume to lower the volume). Thereby, in the mobile phone terminal of the present embodiment, the sound leakage state can be improved.

  The processing of the flowchart shown in FIG. 4 is repeated until playback of the music file is completed or an instruction to finish playback is input from the user through the control unit 14 in step S5 of the flowchart of FIG. become.

  In this embodiment, the extracted / converted playback audio data is temporarily stored in the extracted / converted playback audio data storage unit 20 in step S16 of FIG. 3, and after the extraction / conversion in step S26 of FIG. The extracted / converted reproduced audio data read from the reproduced audio data storage unit 20 is compared with the extracted peripheral audio data. For example, the time conversion processing in step S15 in FIG. By temporarily comparing the reproduced audio data time-delayed by the delay circuit and the extracted peripheral audio data, the audio data may not be temporarily stored.

[Summary]
As described above, in the mobile phone terminal according to the embodiment of the present invention, when sound such as music is output using the headphone 33, the surrounding sound is monitored by the external microphone 34 and integrated by the external microphone 34. Whether or not sound leaked from the headphone 33 is included in the surrounding sound, and when sound leakage is detected, countermeasure processing for correcting the sound leakage is performed, Alternatively, the user is requested to take measures to correct the sound leakage. Thereby, according to the mobile phone terminal of the present embodiment, it is possible to prevent the surrounding person from being uncomfortable due to sound leakage from the headphones, and the person who is listening to music or the like with headphones can be prevented. Sound leaks that are difficult to notice are easily noticed by the person himself, making it easier to take corrective actions for sound leaks.

  The above description of each embodiment of the present invention is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and it is needless to say that various modifications can be made according to the design or the like as long as the technical idea according to the present invention is not deviated.

  The audio processing apparatus according to the present embodiment is not limited to a mobile phone terminal, and a PDA (Personal Digital Assistant), a personal computer, a portable video game apparatus, and a portable digital television reception having a music data content storage and playback function. The present invention can also be applied to various terminals such as mobile devices and car navigation devices, and stationary moving image playback devices.

It is a block diagram which shows the schematic internal circuit structure of the mobile telephone terminal of embodiment of this invention. It is a main flowchart of an audio | voice reproduction process. It is a flowchart which shows the process by the reproduction | regeneration audio | voice side at the time of a sound leak countermeasure. It is a flowchart which shows the process by the side audio | voice side at the time of a sound leak countermeasure.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Control part, 11 Communication circuit, 12 Communication antenna, 13 Display part, 14 Operation part, 15 Clock part, 16 Memory part, 17 Program storage part, 18 Content storage part, 19 Sound leak detection parameter storage part, 20 After-conversion playback audio data storage unit, 30 speaker, 31 microphone, 32 external audio terminal, 33 headphones, 34 external microphone, 35 audio processing unit, 36 image processing unit, 37 vibrator, 38 digital broadcast receiving unit, 39 for receiving digital broadcast Antenna

Claims (7)

An audio data supply unit for supplying audio data to the headphones;
A sound leak detection unit that performs processing for detecting sound leaked to the outside from the headphones worn in the ear;
A sound leakage correction processing unit that executes a predetermined sound leakage correction process when the sound leakage detection unit detects leakage of sound from the headphones to the outside,
A speech processing apparatus characterized by that. The sound leak detection unit
Extraction processing of a predetermined frequency portion from the audio data supplied to the headphones, and time conversion processing that takes into account the arrival delay time until the sound leaked from the headphones to the microphone reaches the microphone A voice extraction conversion unit to perform,
A peripheral sound extraction unit that performs processing of extracting the predetermined frequency portion from the sound data supplied from the microphone that acquires the peripheral sound and converts it into sound data;
The voice data after the extraction process by the peripheral voice extraction unit and the voice data after the extraction process and the time conversion process by the voice extraction conversion unit are determined to match, and it is determined that the two voice data are substantially matched. 2. The audio processing apparatus according to claim 1, further comprising: a coincidence determination unit that detects that audio leaks from the headphones attached to the ear to the outside. It has a notification unit for making notifications recognizable by the user,
2. The sound leak correction processing unit, as the sound leak correction process, causes the notification unit to make a notification for requesting a user for a predetermined sound leak correction measure. The speech processing apparatus according to the description.   The sound leakage correction processing unit generates a sound waveform having a phase opposite to the sound waveform output from the headphones to the outside, the process of reducing the volume of the sound output from the headphones, the process of attenuating the predetermined frequency portion The speech processing apparatus according to claim 1, wherein any one of the processes to be performed is executed as the sound leakage correction process.   The sound volume output from the headphones is compared with a predetermined threshold, and the sound leakage detection unit and the sound leakage correction process are performed only when the volume of the sound output from the headphones is equal to or higher than the predetermined threshold. The speech processing apparatus according to claim 1, further comprising a function setting control unit that enables the function of each unit. An audio data supply unit supplying audio data to the headphones;
A step in which the sound leak detection unit detects the sound leaked to the outside from the headphones worn in the ear; and
A sound leakage correction processing unit that executes a predetermined sound leakage correction process when the sound leakage detection unit detects a leakage of sound from the headphones to the outside.
And a voice processing method. An audio data supply unit for supplying audio data to the headphones;
A sound leak detection unit that performs processing for detecting sound leaked to the outside from the headphones worn in the ear;
A sound leakage correction processing unit that executes a predetermined sound leakage correction process when the sound leakage detection unit detects leakage of sound from the headphones to the outside,
A portable terminal characterized by that.

Images