JP2006010809A - Personal identification system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a personal identification system which can increase security level more and whose security can hardly be broken so long as the person himself or herself who is alive does not operate it directly. <P>SOLUTION: Speech information of an object person to be authenticated is detected by both a bone conduction sound detection part 340 and an air duct sound detection part 304 to perform identification processing based upon both their bone conduction speech information and air duct speech information. Feature information which can not be known with a bone conduction sound or air duct sound alone and originates from differences between both the waveforms can be grasped to greatly increase the security level of the personal identification. The bone conduction speech information and air duct speech information are homogeneous speech information in terms of information kind, so processes by hardware and software can easily be made common and the feature information originating from the differences between the waveforms can easily be extracted through operations. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a personal authentication system using voice.

JP 2000-259828 A Japanese Patent Laid-Open No. 2004-80080 JP 2003-58190 A

  As a personal authentication method, a so-called speaker recognition technique using personality information included in a voice wave of a person to be authenticated is widely used. For example, recently, as disclosed in Patent Documents 1 to 4, various personal authentication methods including speaker recognition have been proposed in order to increase the security level of mobile phones. In recent years, the proliferation of mobile phones has been increasing rapidly, and competition for new model development has intensified, so the model replacement cycle has also been shortened. Since mobile phones serve as a storage medium for personal data such as a telephone directory and mail address list, a problem has been pointed out that waste telephones with data remaining are sold as junk and cause personal information leakage. In addition, mobile phones equipped with information communication terminal functions such as the Internet connection are becoming standardized, and are widely used for payment for information provision, payments for shopping, mobile banking, etc. Since diversion to lock operation terminals such as buildings and automobiles is also considered, a higher security level is required. Patent Documents 1 and 2 disclose a technique for raising a security level by combining not only voice authentication but also an authentication method using another means such as face image matching or fingerprint matching.

  As another problem, since the accuracy of speaker recognition is lowered in a place with a high noise level, Patent Document 3 discloses a mobile phone equipped with an authentication function based on speaker recognition using a bone conduction microphone. ing. Since the bone conduction microphone detects sound through the human skeleton and tissue, there is an advantage that the bone conduction microphone is hardly affected by air noise.

  In recent years, as security systems become more sophisticated, criminal techniques that illegally break them have become more sophisticated or bold. For example, as in Patent Document 1 and Patent Document 2, when a method that combines authentication using images such as fingerprints and faces and authentication using voice is adopted, it seems that it is very difficult to break through security. However, if the following method is adopted, it is not impossible to pass through all the security steps spread over a plurality of stages. In other words, the presence or absence of a legitimate user using a photo or video for the face, a recording tape for the voice, a photo-engraved stamp for the fingerprint, or an arm or finger cut from the person to be authenticated. Is reproduced virtually and the acceptance certification is obtained sequentially. This method can break through security even if the living person is not on the spot, and does not necessarily require a risky method such as kidnapping and abduction. In addition, even if a violent crime involving kidnapping is involved, once the information necessary for authentication is obtained from the person, it is sufficient to use a copy or an acquired product (finger etc.), so it is used There is a possibility that the trap will not work even if the person is killed for the purpose of sealing.

  On the other hand, although the technique of Patent Document 3 certainly contributes to the improvement of the sensitivity of speaker recognition to noise, it uses only bone conduction sound, so it does not necessarily have abundant feature information for reliably identifying and authenticating individuals. However, it does not contribute much to improving the security level itself.

  It is an object of the present invention to provide a personal authentication system that can further improve the security level and is difficult to break through unless a living person directly operates.

Means and actions / effects for solving the problems

The present invention relates to a personal authentication system that authenticates a person who is a subject of authentication processing based on a voice issued by the person subject to the authentication process.
A bone conduction sound detection unit for detecting the voice information of the person to be authenticated by bone conduction sound;
A bone conduction speech information storage unit for storing the detected bone conduction speech information;
An air conduction sound detection unit for detecting voice information of the person to be authenticated by air conduction sound;
An air conduction sound information storage unit for storing detected air conduction sound information;
And an authentication processing means for performing an authentication process based on both the bone conduction voice information and the air conduction voice information.

  In the authentication method based on speaker recognition, as is clear from the disclosures of Patent Documents 1 to 3, only the detection accuracy due to noise or the like is taken into consideration in the voice detection step, and is released from the vocal cords through the airway into the air. Whether the air conduction sound (in the present invention, this is referred to as “air conduction sound”) is detected by a normal microphone or the bone conduction sound is detected by a dedicated bone conduction microphone, the sound environment of the system It may be selected appropriately depending on whether it is used below, and there was no idea of using both in combination.

  However, air conduction sound is air as a medium through which sound waves are transmitted, whereas bone conduction sound medium is a human tissue that is interposed between a bone conduction sound detection unit (specifically, a bone conduction microphone) and a vocal cord. It is a skeleton and the acoustic impedance structure is completely different. As a result, the detected speech waveform is also affected, and there are considerable differences between the air conduction sound, the conduction sound, and the detection waveform even though the sound sound is emitted from a common vocal cord. The propagation path of bone conduction sound is complicated and inhomogeneous compared to air of air conduction sound medium because human tissue and skeleton are involved, and parameters that affect sound propagation such as propagation speed, amplitude, and acoustic resonance frequency Therefore, the original sound waveform from the vocal cords undergoes much greater alteration than the air conduction sound in the process of propagation as a bone conduction sound. Naturally, there are individual differences in the human body tissue and skeleton serving as a propagation path, and accordingly, there is a unique difference in the waveforms of the air conduction sound and the bone conduction sound.

The present inventor has paid attention to such a difference between the bone conduction voice information and the air conduction voice information, and found that combining these both produces various epoch-making effects on personal authentication technology. The invention has been completed. Specifically, the following specific effects that cannot be achieved by the bone conduction voice information and the air conduction voice information alone are produced.
(1) It becomes possible to newly grasp characteristic information derived from the difference between the two waveforms, which could not be known solely by the bone conduction sound and the air conduction sound. As a result, the security level of personal authentication can be significantly increased.
(2) Since both the bone conduction voice information and the air conduction voice information are the same kind of voice information as the information type, it is easy to share the processing of hardware and software, resulting from the difference in waveform It is also easy to extract feature information by calculation.

More preferably, the bone conduction sound information and the air conduction sound information are generated by simultaneously detecting the sound produced by the person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit. As a result, the following new effects are produced.
(1) The bone conduction sound is relatively difficult to accurately reproduce by recording because of the human body contact at the time of detection, and it must be sampled simultaneously with the air conduction sound. Unless it is, it is very difficult to break through security.
(2) Bone conduction sound and air conduction sound are compared to the case where the bone conduction sound and air conduction sound have the same waveform source, and separately uttered sounds are detected as bone conduction sound or air conduction sound. Since the correlation as a voice waveform is strengthened, the difference component unique to the authentication target in the waveform difference, that is, the feature information that can be used for authentication can be grasped more clearly, and the authentication accuracy can be improved.

  The authentication processing means includes: a standard voice feature information storage unit that stores standard voice feature information that is a collation destination of collation source voice feature information based on both bone conduction voice information and air conduction voice information; and a collation source voice feature It can be configured as having a matching means for matching information with the standard voice feature information. Standard voice feature information is created in advance based on the air conduction sound information and the bone conduction sound information of the authentication specific target person (the person who should be accepted and authenticated (that is, the person who should be authenticated as “correct”)), By using this as a verification destination of verification source voice feature information acquired from the authentication processing target person at the time of authentication, it is possible to simplify the authentication processing and improve accuracy. In the case where authentication is performed using a phase difference as described later as standard voice feature information, the standard voice of the person to be authenticated is used as a bone conduction sound detection unit and an air conduction sound detection unit provided outside the system. It is also possible to detect and create by. However, from the viewpoint of reducing the influence of differences in characteristics between hardware, standard voice feature information is detected and created by the bone conduction sound detection unit (provided in the system itself) and the air conduction sound detection unit. Is more effective, and the process of creating standard audio feature information is naturally simplified.

  The audio feature information can include a frequency spectrum of bone conduction sound and a frequency spectrum of air conduction sound. In this case, the collation means collates the frequency spectrum with each standard frequency spectrum of the bone conduction sound and the air conduction sound included in the standard voice feature information, and accepts the collation match result in both of them. It can be authenticated. Even if the voice of the same person, the frequency spectrum of bone conduction sound and the frequency spectrum of air conduction sound are different from each other, so the frequency spectrum of bone conduction sound and air conduction sound is compared with the corresponding standard frequency spectrum. This increases the accuracy of personal authentication. This effect is obtained by simultaneously detecting the sound generated by the person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit for both the frequency spectrum and the standard frequency spectrum to be authenticated. Increased especially when used. Since verification is performed using the frequency spectrum of both bone conduction sound and air conduction sound, the authentication method includes characteristic information derived from the difference between the two waveforms, which cannot be specified by each waveform alone. Become.

  On the other hand, the personal authentication system of the present invention is a composite that can be calculated only when both the bone conduction sound waveform detected by the bone conduction sound detection unit and the air conduction sound waveform detected by the air conduction sound detection unit are used. It can be configured as having composite voice feature information calculation means for calculating voice feature information, and the authentication processing means can perform authentication processing based on the composite voice feature information. This method is nothing but a method of extracting and grasping the feature information derived from the difference between the two waveforms that cannot be specified by each waveform of the bone conduction sound and the air conduction sound as composite voice feature information. The improvement effect of the authentication accuracy and the security level by the combination of voice information can be further enhanced.

  The composite voice feature information calculation means can calculate the phase difference between the air conduction sound waveform and the bone conduction sound waveform as the composite sound feature information. As described above, in the human body tissue and skeleton that are the propagation paths of the bone conduction sound, the biological characteristics of the individual are directly reflected in the distribution state of the acoustic impedance. Specifically, because the living wave (that is, the individual to be authenticated) differs in the formation of reflected waves and the phase delay at impedance discontinuities (eg, tissue boundaries), the bone conduction sound waveform is Each individual to be authenticated with respect to the air conduction sound waveform has a different phase difference, and has personal identification. Therefore, if the phase difference between the air conduction sound waveform and the bone conduction sound waveform is obtained by calculation, this can be used as effective and important information for multi-person authentication. In this case, in order to accurately calculate the phase difference, it is necessary to use the bone conduction sound and the air conduction sound that are simultaneously detected for the same sound.

  In this case, the phase difference between the air conduction sound waveform and the bone conduction sound waveform peculiar to the authentication identification subject specified in advance is obtained as a standard phase difference, and the authentication processing means determines that the calculated phase difference is the standard level. Authentication processing can be performed based on whether or not it matches the phase difference. The waveform phase difference itself is a relatively simple waveform calculation (for example, a method of calculating a difference or addition waveform by setting the phase difference between two waveforms in various ways to obtain a phase difference that minimizes or maximizes the integrated amplitude). There is an advantage that the calculation load can be reduced as compared with spectrum matching or the like.

  Since the air conduction sound and the bone conduction sound also have a difference in frequency spectrum, more accurate calculation of the phase difference is possible by obtaining the phase difference by extracting the frequency components that are included in both waveforms in common. . In this case, the extraction of the frequency component can be performed using a known digital filter technique.

  The composite voice feature information is not limited to the phase difference between the two waveforms as described above. For example, the difference spectrum of each frequency spectrum between the air conduction sound and the bone conduction sound can be used. The bone conduction sound has individual differences in acoustic characteristics such as attenuation or resonance of the human body intervening in the propagation path. As a result, the frequency component that is insufficient or emphasized with respect to the air conduction sound also varies depending on the individual. Therefore, the difference spectrum between the air conduction sound and the bone conduction sound has personal identification. There are also spectra obtained equivalently by mathematical operations of individual frequency spectra and the above difference spectra, such as the common spectrum of air conduction sound and bone conduction sound (subtracting the above difference spectrum from each frequency spectrum). Naturally, it can be used as composite voice feature information.

  The cause of the phase difference and difference spectrum as described above is mainly due to the skeleton and the mechanical structure of the human tissue that form the propagation path of the bone conduction sound. There is an advantage that misidentification is less likely to occur even if some alteration occurs.

  Further, the authentication processing means performs the authentication process by performing a first authentication process for comparing at least one of the frequency spectrum of the bone conduction sound and the frequency spectrum of the air conduction sound with the standard frequency spectrum, and the first based on the composite voice feature information. It can also be implemented in combination with the second authentication process. The conventional voice authentication method based on either the frequency spectrum of the bone conduction sound or the frequency spectrum of the air conduction sound has high personal identification by the method of spectrum matching, but it is also used for misrepresentation using recording etc. There are also security holes. However, it is possible to effectively prevent the occurrence of the security hole as described above by combining the authentication processing based on the composite voice feature information (particularly, a phase difference that is easy to calculate) as described above.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In this embodiment, a case where the function of the personal authentication system of the present invention is incorporated in a mobile phone will be described as an example. FIG. 1 is an external perspective view showing an example of a mobile phone 1. The mobile phone 1 is provided with a receiver 303 on the upper side of the main body and a transmitter 304 on the lower side, and a liquid crystal display device (for example, a color liquid crystal display device) between the two. An on-hook / off-hook switch 306 that switches the liquid crystal monitor 308, the input unit 305, and the mobile phone 1 between an on-hook state and an off-hook state is provided. In the present embodiment, the mobile phone 1 can access not only a line telephone communication network but also an information communication network such as the Internet. The input unit includes a call dial key 305a that is also used as an information input keyboard, a cursor movement key 305b, and a mode switching key 305c for switching use modes such as a call mode and an information search mode.

  The transmitter 304 is composed of a microphone that also serves as an air conduction sound detection unit. On the other hand, the handset 303 is constituted by a bone conduction speaker in the present embodiment, and a bone conduction microphone 340 as a bone conduction sound detecting unit is disposed in the vicinity thereof. The basic configuration of the bone conduction speaker is, for example, Japanese Patent No. 2967777 or Japanese Patent Laid-Open No. 2003-340370, and the basic configuration of the bone conduction microphone is, for example, Japanese Utility Model Laid-Open No. 55-146785, Japanese Patent Laid-Open No. 58-18297. Since it is well known in Japanese Patent Publication No. 63-173991 or Japanese Patent No. 34888749, detailed description is omitted. All of these are applied to the ear or the jawbone below the ear.

  FIG. 2 is a block diagram showing an example of the electrical configuration of the mobile phone 1. The main parts of the circuit are an I / O port 311, a CPU 312 (which constitutes an authentication processing unit, a collating unit, and a composite voice feature information calculation unit), a ROM 313, and a RAM 314 (bone conduction voice information storage unit and memory unit) connected thereto. And a control unit 310 including a conductive voice information storage unit. The input unit 305 and the on-hook / off-hook switch 306 described above are connected to the I / O port 311. The receiver 303 is connected via an amplifier 315 and a D / A converter 316, the transmitter 304 is connected via an amplifier 317 and an A / D converter 318, and the bone conduction microphone 340 is connected to an amplifier 320 and an A / D converter. Each of them is connected to the I / O port 311 via the H.321. A communication connection circuit 323 is connected to the I / O port 311. The connection circuit 323 includes a connection interface 331 for connecting to the control unit 310, a modulator 332, a transmitter 333, a frequency synthesizer 334, a receiver 335, a demodulator 336, a duplexer 337, and the like connected thereto. ing. The data signal from the controller 310 is modulated by the modulator 332 and further transmitted from the antenna 339 via the duplexer 337 by the transmitter 333. On the other hand, the received radio wave is received by the receiver 335 via the antenna 339 and the duplexer 337, demodulated by the demodulator 336, and then input to the I / O port 311 of the control unit 310. When making a call, for example, an audio signal input from the transmitter 304 is amplified by the amplifier 317, further digitally converted by the A / D converter 318, and input to the control unit 310. The signal is processed by the control unit 310 as necessary, and then output from the receiver 303 via the D / A converter 316 and the amplifier 315.

  On the other hand, a control radio wave transmitter 338 that transmits a control radio wave P is connected to the connection interface 331. The control radio wave P is transmitted from the antenna 339 via the duplexer 337. When the mobile phone 1 moves to another communication zone 102, the network-side radio network controller 104 performs a well-known handover process based on the reception status of the control radio wave P.

  Next, in the ROM 314, a communication program, which is a basic control program for wireless telephone communication, and a display program for controlling the screen display of the liquid crystal monitor 308 are installed. Further, as shown in FIG. 4, in ROM 314, an authentication processing means is realized by being executed by CPU 312 for authenticating whether or not the user of mobile phone 1 is a regular user. Is also installed. In the present embodiment, the authentication process is specifically performed by speaker recognition / collation processing using both the sound waveform of the air conduction sound and the sound waveform of the bone conduction sound. The authentication program is the main program 201. And a sub-module group used by the main program 201, specifically, the air conduction sound sampling module 202, the bone conduction sound sampling module 203, the air conduction sound / bone conduction sound phase difference calculation / collation determination module 204, the air conduction sound. / Bone conduction sound difference spectrum calculation / collation judgment module 205, waveform spectrum collation / judgment module 206, etc. These programs are all executed by the CPU 312 using the RAM 313 in FIG. 2 as a work area.

  In addition, as master data for authentication 322 (to be verification source voice feature information), audio spectrum master data used when voice authentication is performed in spectrum verification processing (modules involved are reference numerals 205 and 206), specifically Are provided with air conduction sound spectrum master data 321, bone conduction sound sound spectrum master data 222, and master data 223 of their difference spectra. Prior to performing the authentication process, these data are generated by causing a legitimate user (authentication identification target person) to sound a predetermined sound (“on”), word or sentence for verification, and to receive it. The waveform is detected by 303 (air conduction sound) and a bone conduction microphone 340 (bone conduction sound), and a spectrum is generated by performing a well-known Fourier transform operation. These data are different for each user, and the rewritable ROM, specifically, the EEPROM shown in FIG. (Electrically Erasable Programmable Read Only Memory) 322 is stored so as to be rewritable, and is loaded into the authentication data memory of the RAM 313 and used as necessary.

  In the following description, a plurality of specific voice authentication methods will be described. However, there are modules and data that are not particularly used depending on the method, and therefore, necessary modules and data are selected and used ( Of course, it is possible to omit modules and data not used in the authentication method).

  Since the method for using the mobile phone 1 is well known for the telephone portion, detailed description thereof is omitted, and authentication processing prior to use will be described in detail below. FIG. 10 is a flow of authentication main processing by the main program 201 (FIG. 4). In order to perform the authentication process, it is necessary to perform an initialization process including registration of data for verification (S1). This initialization process is skipped once it is performed, except when the verification data is updated. S3 is a voice authentication process that is the center of the process, and an authentication flag indicating whether or not to permit the use of the function of the mobile phone 1 is set in, for example, the RAM 313 (FIG. 2) based on the authentication result. In S5, the authentication flag is read, and when the prescribed condition is satisfied, the lock is released (S7: that is, usage is permitted), and when not satisfied, the lock is released (S8: that is, that the usage is not permitted). .

  The function of the cellular phone 1 that is unlocked by authentication is not limited to a well-known telephone function (including connection to a telephone communication network or the Internet, a mail function, etc.), for example, lock / unlock of a car. Or, it can be a wireless remote control unit function for automobile functions, such as starting the engine, turning on / off the headlights and interior lights.

  The flow of each process of the initialization process and the voice recognition process is shown in FIGS. 11 and 15 to 18. In either case, the main part of the process consists of voice data processing responsible for acquisition and processing of voice data (S301 in the initial process and S402 in the voice authentication process). The audio data processing will be described in detail first with reference to FIG. In S501, voice is input. In the speaker authentication technology, various methods have been devised for causing the authentication target person to pronounce the voice for authentication for the purpose of improving security and the initial data acquisition method differs depending on the method, but both methods are well known. Therefore, only an outline will be described.

(1) Method of uttering only one character (or sound (for example, vowel)) A character to be uttered is designated and generated by display or the like, and sampling is performed.
(2) Method of sequentially uttering by combining multiple characters Basically the same as (1). The order of utterance is guided by display or the like, and waveform sampling is performed sequentially. During actual verification, the utterance order may be fixed, or the utterance order may be changed every time using a random number (in the latter case, the utterance order of characters designated at the time of authentication is changed randomly) There is an advantage that even if you record what you say in a fixed order, you can use it.
(3) Method of uttering a word There may be only one type of word to be used (in this case, it is the same as (2)), or there is a method of selecting from a plurality of types. In the latter case (refer to FIG. 1 below), a selection list of words to be collated is displayed on the screen 108 and selected by the input unit 305, and then the selected word is uttered and sampled. In addition, there is a method in which the number of characters (or recording time) is designated, a user's favorite word is arbitrarily input by the input unit 305, and utterance / sampling is performed. In this case, it is clear that the word is substituted for the password. Further, as a more elaborate method, there is a method in which a question that only a legitimate user can understand is voice-output and a registered answer corresponding to this is voice-inputted. In this case, in the initialization process, it is necessary to input or select the question contents to be output and the answer contents to the question contents.
(4) Method of inputting a sentence This is basically the same as (3), and when a question / answer format is adopted, there may be a method of inputting a plurality of questions and answers in an interactive format.

  When comparing the bone conduction sound and the air conduction sound, the sound component derived from the vocal cord vibration such as a vowel tends to be emphasized from the air conduction sound because the bone conduction sound is closer to the vocal cord. In addition, since the frictional sound and the plosive sound are associated with sound-generating elements other than the vocal cords such as the tongue and lips, the air conduction sound appears more emphasized. Therefore, when performing authentication based on the difference in the waveform or spectrum between bone conduction sound and air conduction sound (especially the difference spectrum), as speech waveform data (bone conduction sound and air conduction sound) to be authenticated, Sounds that include vowels, friction sounds, and plosives (preferably, sound strings in which the most abundant sounds are one of these sound types: for example, “Sashisuseso”, “Sushishinchu no Mushi”, “Aiueo”, etc .: Of course, it may be desirable to specify “sa line”, “ta line” or “a line”. In addition, even in the same vowel, sounds such as “i, e” that use the front part of the tongue for articulation are air conduction sounds, and conversely, sounds such as “u, o” that use the back part of the tongue are bone conduction sounds. Therefore, it is also effective to specify a sound string mainly including either the former or the latter, such as “No (house)” and “Koubo (yeast)”.

  FIG. 12 is a flowchart showing the flow of audio data processing. In step S <b> 501, the designated voice input is input using both the transmitter 304 and the bone conduction microphone 340. In S502, the sampling is performed (implemented by executing the air conduction sound sampling module 202 and the bone conduction sound sampling module 203 of FIG. 4). Since the user emits the requested sound sequence only once, sampling must be performed simultaneously in time series. In this case, when a single CPU is used, it is executed as parallel processing by time division as shown in FIG. Specifically, the sampling counter is reset in S101, and thereafter, while the sampling counter is incremented, reading of the microphone input port for air conduction sound (S102) and writing of the read value to the memory (RAM 313) (S103), Read of the input port of the bone conduction microphone (S104) and writing of the read value to the memory (S105) are repeated alternately. Depending on the length of the audio data to be sampled, the total sampling time (the value of the sampling counter can be used instead, but other timer means may be used), and the sampling is aborted when the time is up (S107) Unless both the bone conduction sound waveform and the air conduction sound waveform are sampled simultaneously, it is impossible to normally obtain the data of both sounds, for example, by sequential sound input using a tape recorder or the like. It is possible to effectively prevent deception and the like.

  In addition, when inputting speech data using words or sentences, it is determined whether or not the input of speech with a predetermined content (meaning) is completed by using a well-known speech recognition technology, and if completed, sampling is terminated. It can also be configured as follows. In this case, the timer means is not always necessary. Although the hardware is somewhat complicated, the sampling of the air conduction sound and the bone conduction sound can be performed independently by separate (ie, two) CPUs. Both audio waveforms can be sampled in parallel without processing.

  Returning to FIG. 12, when the sampling of the sound waveforms of the air conduction sound and the bone conduction sound is completed as described above, it is checked in step S503 whether each sound is sampled simultaneously. There are various possible check methods. For example, if the air conduction sound and the bone conduction sound are input at a deliberately shifted timing, one of them will protrude outside the sampling time, and the acquired data will have a large blank period. There is a way to take advantage of this because it should happen. In this case, it is checked whether at least one of the acquired air conduction sound waveform and bone conduction sound waveform has a period during which the audio amplitude is equal to or lower than a predetermined lower limit value continuing for a certain period. If is present, it is determined that there is no simultaneity. If it is determined in S503 that there is no simultaneity, the process proceeds to S511, where the processing is terminated and an error or warning is output.

  If simultaneity is satisfied, the process proceeds to S505 and S506, and the detected air conduction sound waveform data and bone conduction sound waveform data are stored and registered in the memory. The following is a calculation process of the composite voice feature information used for authentication (the function of the composite voice feature information calculation means is realized). In S507, the phase difference between the air conduction sound waveform and the bone conduction sound speech waveform is calculated as the composite sound feature information (implemented by executing the air conduction sound / bone conduction sound phase difference calculation / collation determination module 204). As shown in FIG. 8, both the air conduction sound waveform and the bone conduction sound sound waveform are obtained by simultaneously sampling the same sound by individual microphones, and both waveforms when the waveforms are superimposed on the basis of the sampling start timing. Is the reference superposition phase. Since the two waveforms include many common frequency components based on the same sound, as shown in FIG. 9, the overlapping phase of both waveform data is the phase difference (inherently present in the reference overlapping phase) ( That is, if the differential waveform is calculated by relatively shifting so as to eliminate the phase difference (φ to be obtained), the integrated amplitude (average amplitude) of the differential waveform is minimized by the superposition phase ( (See the bottom of FIG. 9). Therefore, if the superposition phase of both waveform data is changed variously while calculating the integral amplitude of the difference waveform and the superposition phase where the integral amplitude is minimized is found, this is obtained as the phase difference φ between both waveforms to be obtained. Obtainable.

In consideration of use as personal feature information used for authentication processing, it is only necessary to obtain a parameter that uniquely corresponds to the phase difference φ to be obtained. The phase difference is not limited to the following, and the following can be substituted.
(1) Phase difference that maximizes the integrated amplitude of the differential waveform
(2) Phase difference that minimizes the integrated amplitude of the sum waveform
(3) Phase difference that maximizes the integrated amplitude of the added waveform

  Hereinafter, the process of obtaining the phase difference φ that minimizes the integral amplitude of the differential waveform will be described with reference to the flowchart of FIG. In S201, the superposition phase difference Σt (the waveform is a superposition of various sinusoidal waveforms, so the calculation unit of the phase difference is not an angle but time) is reset. Next, one of the air conduction sound waveform and the bone conduction sound waveform is set as the first waveform, and the other is set as the second waveform, and the phase of the second waveform is shifted by a predetermined minute time Δt in S202 to obtain the first waveform. Is fixed, and the differential waveform is calculated in S203. In S204, the integral amplitude A of the difference waveform is calculated. The method of calculating the integral amplitude is well known, but can be calculated as follows, for example. First, assuming that the waveform is f (t), all the values of f (t) corresponding to each sampling timing t are added and divided by the number of samplings N to obtain the waveform center line f0. Then, | f (t) −f0 | is calculated for each value of t, and this is added for all t and divided by N to obtain an integrated amplitude. In S205, the value of Σt at that time is set as the phase difference φ and stored in association with the value of the integral amplitude A.

  Next, in S206, Σt is incremented by Δt, and the processes in S202 to S206 are repeated until Σt reaches a predetermined maximum value Σtmax. Considering that the user can speak naturally as the voice designated for authentication, it is desirable to secure the length of the voice sample, for example, for 1 second or longer. The amount of waveform shift required to find the phase difference is sufficient for 0.5 to 2 wavelengths. Therefore, considering that the human voice frequency is on average 1 to 2 kHz, Σt is 0. It should be set to about 5 to 2 ms. The sampling period Δt is preferably about 1/1000 to 1/10 of Σt. Note that the second waveform shift interval may be calculated by setting the interval only in one direction, positive or negative, with the reference overlap phase difference as the origin, or by setting the interval in each of positive and negative. It may be.

  When the above calculation is completed, the process proceeds to S208, the stored minimum value A0 of the integrated amplitude A is found, and the phase difference φ0 corresponding to A0 is determined as the phase difference φ0 to be obtained in S209. As shown in FIG. 6, there is a considerable difference in spectrum between the bone conduction sound and the air conduction sound, and there are frequency components that are not common to each other (for example, in the case of bone conduction sound, the frequency is high). The spectral intensity of the range tends to be missing). Therefore, when calculating the phase difference, it may be desirable to perform waveform calculation after extracting a frequency region with many common components by filtering. This is the end of the description of the phase difference calculation.

  Returning to FIG. 12, in S508 and S509, the frequency spectrum of each waveform of the air conduction sound and the bone conduction sound is calculated, and the result is stored. As described above, this calculation can be performed by performing a well-known Fourier transform process on the original waveform. However, in speaker recognition, the spectral outline shown below (mainly information that reflects voice quality) is used rather than the spectral waveform containing the fine structure shown in FIG. However, it is known that it has excellent measurement reproducibility, is sufficiently effective as personal identification information, and can be easily verified. This spectral outline is also referred to as spectral envelope, and various well-known speech analysis algorithms (for example, short-time accident correlation analysis method, short-time spectrum analysis method, cepstrum analysis method, band filter bank analysis in the case of non-parametric analysis method) In the case of using a parametric analysis method such as a method or a zero-crossing product method, extraction and calculation can be performed by a linear prediction analysis method, a maximum likelihood spectrum estimation method, a covariance method, a PARCOR analysis method, an LSP analysis method, or the like.

  Returning to FIG. 12, in S510, as shown in FIG. 6, the difference between the frequency spectra of the air conduction sound and the bone conduction sound obtained as described above is calculated and stored as difference spectrum data. The above processing is performed by executing the air conduction sound / bone conduction sound difference spectrum calculation / collation determination module 205 and the waveform spectrum comparison / determination module 206 of FIG. This is the end of the description of the audio data processing.

Returning to FIG. 11, the flow of the initialization process will be described.
In the voice data processing of S301, voice input is performed by the voice of the authorized user (authentication specific target person), and the phase difference, the air conduction sound or the bone conduction sound frequency spectrum or the difference spectrum data is obtained by the method described above. In step S302, these are registered in the EEPROM 322 (FIG. 4) as master data (standard voice feature information: standard phase difference, standard frequency spectrum, or standard difference spectrum) used in the subsequent voice authentication process.

  FIG. 15 shows an example of voice recognition processing. In S401, the user inputs a designated voice for authentication. In S402, the audio data processing described above is executed, and the phase difference φ is calculated. In S403, the phase difference φ is compared with the standard phase difference φ0 stored as master data. Here, the difference φ−φ0 is calculated. In S406, it is checked whether or not the deviation between the phase difference φ and the standard phase difference φ0 is within the allowable range. If the deviation is within the allowable range, the authentication flag is set to be permitted (S407). (S408). Instead of registering the standard phase difference φ0 as the master, an allowable phase difference range (given by the maximum values φmax and φmin) including the standard phase difference φ0 is registered, and φ belongs to the range. Authentication can also be performed depending on whether or not there is.

  FIG. 16 is an example of a voice authentication process using a difference spectrum instead of a phase difference (the same step numbers are assigned to the steps common with FIG. 15 and the description is omitted). In S402, voice data processing is executed. In S410, as shown in FIG. 6, the calculation result of the difference spectrum between the air conduction sound and the bone conduction sound is read, and in S411, the master data of the difference spectrum (FIG. 4: reference numeral 223). Compare with If it is determined in S412 that both match, the authentication flag is set to permit (S413), and if it is out of range, it is set to non-permitted (S414).

  As shown in FIG. 6, the air conduction sound spectrum and the bone conduction sound spectrum have the same main part, but a significant difference is observed in the spectrum intensity in a specific frequency band (for example, a high frequency component). The air conduction sound spectrum appears stronger than the bone conduction sound spectrum). Therefore, matching / mismatching matching can be performed by comparing the shape of the difference spectrum in the frequency band with that of the master. In particular, it is a spectrum envelope peak (as indicated by “x” in FIG. 6) that exists in one of the air conduction sound spectrum and the bone conduction sound spectrum and does not exist in the other, and the peak position is authenticated. When it varies depending on an individual to be detected, the peak can be detected in the difference spectrum, and verification of the peak position (frequency) can be easily performed with high accuracy authentication verification.

  FIG. 17 is an example of a voice authentication process in which each spectrum of the bone conduction sound and the air conduction sound is individually verified with the master (the same step numbers are assigned to the steps common to FIG. 15 and the description is omitted). Audio data processing is executed in S402, and calculation results of each frequency spectrum of the air conduction sound and the bone conduction sound are read out (S420, S423). These are individually compared with master data (FIG. 4: reference numerals 221, 222). Only in cases where it is determined in S422 and S425 that both the bone conduction sound and the air conduction sound match, the authentication flag is set to permit (S426), and if it is out of range, it is set to non-permitted (S427). .

  As shown in FIG. 6, each frequency spectrum of the air conduction sound and the bone conduction sound generates a unique peak position in accordance with the sound in the spectrum envelope. Therefore, the input sound is determined according to the number and position of the peaks. It is possible to identify whether (for example, a word or a character) is the same as the voice indicated by the master (that is, voice recognition). If the content of the voice is the same, the peak position and intensity (or intensity ratio between peaks) can be compared with the master, and it can be authenticated whether it is a legitimate user or not according to the match / mismatch (that is, Speaker recognition).

  Further, the voice authentication process in FIG. 18 includes an authentication process based on the phase difference in FIG. 15 (second authentication process: S401 to S406) and an authentication process based on spectrum matching in FIG. 17 (first authentication process: S420 to S422). The authentication flag is set to permit only when it is determined that both match (S426), and when it is out of the range, it is set to non-permit (S427). In spectral matching, only air conduction sound is used, but bone conduction sound may be used, or both may be used. However, the calculation of the phase difference is simpler than the spectrum calculation, and spectrum verification is performed only for one of the air conduction sound and the bone conduction sound (for the other, the spectrum calculation itself is omitted), and authentication by the phase difference is assisted. If it is used, it is possible to simultaneously reduce the processing weight and improve the authentication accuracy.

  In the above embodiment, the acquisition of data required for authentication and the authentication process using the data are all completed within the mobile phone (the upper concept is an authentication terminal). It is also possible to place a part on a device outside the mobile phone. For example, in a mobile phone, only acquisition of voice waveform data is performed, and the waveform data is directly or after being processed into a spectrum or the like, transferred to an authentication data processing apparatus configured by another computer by communication (in this case, verification Master data must be transferred to the authentication data processing device in advance). The authentication data processing device receives the transferred data, performs authentication processing by collation in the same manner as described above, and sends the result (data content in the same format as the authentication flag) to the mobile phone. return. The mobile phone performs the unlocking (use permission) or unlocking release (use disapproval) process already described according to the contents of the received result.

  In FIG. 2, the authentication data processing apparatus is an authentication host computer 352 connected to a communication network 351 such as the Internet, and the mobile phone 1 is authenticated via a radio base station 350 by radio wave communication by a communication connection circuit 323. A host computer 352 is connected. Note that the authentication host computer 352 may be connected via a short-range wireless communication network such as a wireless LAN or Blue Tooth, or may be connected via a connector and a cable.

  Furthermore, in the above embodiment, the description has been given with reference to application to a mobile phone as a specific example, but the personal authentication system of the present invention is not limited to a mobile phone. For example, as shown in FIG. 3, the present invention can also be applied to an intercom personal authentication system 100 provided at the entrance of a building or an entrance gate to a high security zone in the same building (or site). In this example, a microphone 304 for air conduction sound is provided in the main body of the interphone, and a bone conduction microphone 340 and a bone conduction speaker 303 serving as a receiver are provided on the hand unit 101 side connected to the main body by a curl cord 339. Yes. By uttering the hand unit 101 against the jawbone or the like, the authentication process can be performed in exactly the same flow as already described (note that a microphone 304 for air conduction sound may be provided on the hand unit side). Although the electrical configuration is almost the same as that in FIG. 2, a part related to communication (for example, the communication connection circuit 323) can be omitted as a matter of course.

1 is an external perspective view showing an example of a mobile phone equipped with a personal authentication system of the present invention. The block diagram which shows an example of the electrical constitution of the mobile telephone carrying the personal authentication system of FIG. The external appearance perspective view which shows the example which applied the personal authentication system of this invention to the intercom. FIG. 3 is a schematic diagram showing storage contents of a ROM and an EEPROM in FIG. 2. The graph which shows the example of an audio | voice spectrum and a spectrum envelope. The conceptual diagram of the individual frequency spectrum of an air conduction sound and a bone conduction sound, and those difference spectra. The schematic waveform diagram which shows the concept used after filtering an audio | voice waveform. The schematic waveform diagram explaining the phase difference of an air conduction sound and a bone conduction sound. Explanatory drawing of the method of calculating | requiring the phase difference of an air conduction sound and a bone conduction sound by a waveform difference. The flowchart which shows the flow of an authentication main process. The flowchart which shows the flow of an initialization process. The flowchart which shows the flow of an audio | voice data process. The flowchart which shows the flow of an air conduction sound / bone conduction sound wave form sampling process. The flowchart which shows the flow of an air conduction sound / bone conduction sound phase difference calculation process. The flowchart which shows the flow of the 1st example of a speech recognition process. The flowchart which similarly shows the flow of a 2nd example. The flowchart which similarly shows the flow of a 3rd example. The flowchart which similarly shows the flow of a 4th example.

Explanation of symbols

1 Mobile phone (personal authentication system)
100 intercom (personal authentication system)
304 Mic (microphone: air conduction sound detector)
340 Bone conduction microphone (bone conduction sound detector)
312 CPU (authentication processing means, verification means, composite voice feature information calculation means)
313 RAM (bone conduction voice information storage unit, air conduction voice information storage unit)
322 EEPROM (standard audio feature information storage unit)

Claims (9)

A personal authentication system for authenticating a person who is an authentication process based on a voice uttered by the person who is the authentication process,
A bone conduction sound detector for detecting the voice information of the person to be authenticated by bone conduction sound;
A bone conduction speech information storage unit for storing the detected bone conduction speech information;
An air conduction sound detecting unit for detecting voice information of the person to be authenticated by air conduction sound;
An air conduction sound information storage unit for storing detected air conduction sound information;
Authentication processing means for performing authentication processing based on both the bone conduction voice information and the air conduction voice information;
A personal authentication system characterized by comprising: The bone conduction sound information and the air conduction sound information are generated by simultaneously detecting the sound uttered by the person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit. The personal authentication system according to claim 1. The authentication processing means includes a standard voice feature information storage unit storing standard voice feature information as a collation destination of collation source voice feature information based on both the bone conduction voice information and the air conduction voice information, The personal authentication system according to claim 2, further comprising collation means for collating the original voice feature information with the standard voice feature information. The personal authentication system according to claim 3, wherein the standard voice feature information is created by detecting a standard voice of a person to be authenticated by the bone conduction sound detection unit and the air conduction sound detection unit. The voice feature information includes a frequency spectrum of the bone conduction sound and a frequency spectrum of the air conduction sound, and the collating means uses the frequency spectrum as the bone conduction sound included in the standard voice feature information. 5. The personal authentication system according to claim 4, wherein the personal authentication system is configured to collate with each standard frequency spectrum with the air conduction sound and to accept and authenticate when a collation match result is obtained in both. Composite sound for calculating composite sound feature information that can be calculated only when both the bone conduction sound waveform detected by the bone conduction sound detection unit and the air conduction sound waveform detected by the air conduction sound detection unit are used. The personal authentication system according to claim 1, further comprising a feature information calculation unit, wherein the authentication processing unit performs the authentication process based on the composite voice feature information. The personal authentication system according to claim 6, wherein the composite voice feature information calculation means calculates a phase difference between the air conduction sound waveform and the bone conduction sound waveform as the composite sound feature information. The phase difference between the air conduction sound waveform and the bone conduction sound waveform peculiar to the authentication identification target specified in advance is obtained as a standard phase difference, and the authentication processing means determines that the calculated phase difference is the standard difference. The personal authentication system according to claim 7, wherein the authentication processing is performed based on whether or not the phase difference matches. The authentication processing means includes a first authentication process for comparing at least one of a frequency spectrum of the bone conduction sound and a frequency spectrum of the air conduction sound with a standard frequency spectrum, and the composite voice feature information. The personal authentication system according to any one of claims 6 to 8, wherein the personal authentication system is implemented in combination with a second authentication process based on the above.

Images