JP6726082B2 - Sound determination method and information processing apparatus for crime prevention - Google Patents

Description

The present invention relates to a technique for detecting a sound related to crime prevention.

Conventionally, there is a technique for detecting a scream. Specifically, an example of detection by measuring the duration of a vowel, an example of detection processing based on voice power information, harmonic information, and fundamental frequency, and detection by the volume of voice in two bands are used. Examples of doing it are known. There is also a technique for detecting voices other than words, destructive sounds, and the like by pattern matching.

However, these conventional techniques cannot accurately detect the occurrence of screams and other sounds that should be identified for crime prevention in a space where various environmental sounds and conversational sounds exist.

JP, 9-251583, A JP, 2001-313292, A JP, 2011-53557, A JP 2012-48173 A

Therefore, an object of the present invention is, according to one aspect, to provide a technique for accurately detecting the occurrence of a sound to be identified for crime prevention.

The determination method according to the present invention includes: (A) a first parameter value indicating the degree of variation in the spectrum of the sound related to the sound data and the first degree of whiteness of the sound related to the sound data with respect to the input sound data. A step of calculating a second parameter value and a third parameter value indicating a degree of a harmonic structure in the sound related to the sound data; (B) a first parameter value, a second parameter value and a third parameter Determining whether or not the sound data includes a predetermined sound to be identified for crime prevention based on the value.

According to one aspect, it is possible to accurately detect the occurrence of a sound that should be identified for crime prevention.

FIG. 1 is a diagram showing an outline of a system according to an embodiment. FIG. 2 is a diagram showing a relationship between a sound to be identified for crime prevention and a parameter value. FIG. 3 is a diagram showing a processing flow according to the embodiment. FIG. 4 is a diagram showing a processing flow according to the embodiment.

FIG. 1 shows a system configuration example according to an embodiment of the present invention.

A microphone 1a for collecting sound is connected to the information processing apparatus 100 that executes the main processing according to the embodiment. Here, it is assumed that sound data obtained by digitizing analog signals of surrounding sounds is input to the information processing apparatus 100 from the microphone 1a. However, in some cases, an analog signal of sound is input to the information processing device 100, and digitized sound data is generated in the information processing device 100.

In some cases, the microphone 1b is connected to a terminal device 300 such as an IoT (Internet of Things) gateway, and the terminal device 300 is connected to the information processing device 100 via a computer network 200 such as the Internet. It may be done. In this case, sound data is obtained by digitizing the analog signal of sound in the microphone 1b or the terminal device 300, and the sound data is input to the information processing device 100 via the computer network 200.

The microphone 1a or 1b is arranged in an area where security is to be provided. The information processing device 100 may be installed near the area where security is to be provided, or may be installed in a remote place. The information processing device 100 may be a physical server provided in a cloud or the like, or may be realized as a virtual machine.

The information processing device 100 includes a sound data storage unit 102, a calculation unit 103, a second data storage unit 104, a determination unit 105, and an output unit 106.

The sound data storage unit 102 stores the sound data from the microphone 1a or 1b. The calculation unit 103 calculates a parameter value that is characteristic in this embodiment. More specifically, the calculation unit 103 includes a preprocessing unit 1031, a first data storage unit 1032, a first parameter value calculation unit 1033, a second parameter value calculation unit 1034, and a third parameter value calculation unit 1035. And a fourth parameter value calculation unit 1036.

The preprocessing unit 1031 executes a process for generating data commonly used by the first to fourth parameter value calculation units 1033 to 1036, and stores the processing result in the first data storage unit 1032. Whether or not the fourth parameter value calculation unit 1036 is used is arbitrary.

The first parameter value calculation unit 1033 calculates a first parameter value indicating the degree of variation of the sound spectrum. The second parameter value calculation unit 1034 calculates a second parameter value indicating the whiteness degree of sound. The third parameter value calculation unit 1035 calculates a third parameter value indicating the degree of harmonic structure in the sound. The fourth parameter value calculation unit 1036 calculates the fourth parameter value that represents the main frequency of the sound. The first to fourth parameter values will be described in detail later.

The second data storage unit 104 stores the parameter value calculated by the calculation unit 103. The determination unit 105 determines whether or not a predetermined sound to be identified for crime prevention is generated based on the parameter value stored in the second data storage unit 104. When the determination unit 105 determines that a predetermined sound that should be identified for crime prevention is generated, the output unit 106 outputs a notification indicating that or a detected sound type. For example, a warning sound or a warning voice message is output. Alternatively, a warning message is displayed on the monitor of the terminal device connected to the information processing device 100 or the terminal device connected via the computer network 200 or the like. A warning sound or a warning voice message may be output from such a terminal device. The warning voice message and the warning message include information about the detected sound type.

Here, the first to fourth parameter values used in this embodiment will be described. The first parameter value is an index value that represents the degree of variation of the sound spectrum, and is, for example, a value that represents the variation of the spectrum envelope or a value that represents the variation of the spectrum. The specific calculation method will be described in the description of the processing flow.

The second parameter value is an index value that represents the degree of whiteness of sound, and is, for example, information entropy (also referred to as spectrum entropy in the present embodiment) calculated by regarding the spectrum of sound related to sound data as a probability distribution. is there. A specific calculation method will be described in the description of the processing flow.

The third parameter value is an index value indicating the degree of the harmonic structure in the sound related to the sound data, and is, for example, the maximum value within a predetermined range in the cepstrum of the sound related to the sound data (in the present embodiment, the harmonic structure. It is also called strength). The specific calculation method will be described in the description of the processing flow.

The fourth parameter value is the main frequency of the sound related to the sound data, and is, for example, the center-of-gravity frequency of the spectrum of the sound related to the sound data (also referred to as the spectrum center-of-gravity in the present embodiment). The specific calculation method will be described in the description of the processing flow.

On the other hand, in the present embodiment, the types of sounds to be identified for crime prevention are screams, sounds when the security gravel is stomped, which produces a special sound when stepped on, glass breaking sound, or explosion sound.

It has now been found that the relationship between these sounds and the first to third parameter values is as shown in FIG.

Specifically, if it is a scream, the first parameter value is “low”, the second parameter value is “high”, and the third parameter value is “high”. That is, it is possible to specify the occurrence of a sound having a small change in timbre, a natural sound, and a high degree of harmonic structure by the first to third parameter values.

With respect to the sound when the security gravel is stepped on, the first parameter value is “low”, the second parameter value is “high”, and the third parameter value is “low”. That is, it is possible to specify the occurrence of a sound with a small change in timbre, a natural sound, and a low degree of harmonic structure by the first to third parameter values.

Regarding the glass breaking sound and the explosion sound, the first parameter value is “high”, the second parameter value is “high”, and the third parameter value is “low”. That is, it is possible to specify the occurrence of a sound with a large change in timbre, a natural sound, and a low degree of harmonic structure by the first to third parameter values.

As described above, the scream, the sound when the security gravel is stepped on, the glass breaking sound, and the explosion sound are examples, and the sounds having the above-described properties can be detected.

In the case of glass breaking sound, the fourth parameter value representing the main frequency of the sound related to the sound data is “high”, and in the case of explosive sound, the fourth parameter value is “low”. Therefore, in order to distinguish the glass breaking sound and the explosion sound, the fourth parameter value may be used.

Therefore, if the thresholds of the first to fourth parameter values (generally, the values that define the range) are determined by experiments using various sound samples, the determination unit 105 generates a predetermined sound to be identified for crime prevention. Can be detected.

Next, a specific process executed by the information processing device 100 will be described with reference to FIGS. 3 and 4.

The preprocessing unit 1031 reads unprocessed sound data for a predetermined period from the sound data stored in the sound data storage unit 102 (FIG. 3: step S1). Then, the preprocessing unit 1031 executes a predetermined preprocessing and stores the processing result in the first data storage unit 1032 (step S2).

The pre-processing of step S2 includes window processing for sound data for a predetermined period. The window process is, for example, a process of dividing a predetermined period into a plurality of sub periods and multiplying each by a window function. For example, a Hanning window is used for the window function. For window processing and window functions, see <http://www.ni.com/white-paper/4844/ja/>, for example.

Further, in the pre-processing, FFT (Fast Fourier Transform) is performed on the sound data after the window processing for each sub period, and the absolute value of the complex number of the FFT processing result is calculated. Then, for each frequency, the value a[i] (i is the index value corresponding to the frequency) is obtained.

Then, the first parameter value calculation unit 1033 calculates the first parameter value using the data stored in the first data storage unit 1032 and stores it in the second data storage unit 104 (step S3).

If the first parameter value is a value indicating the fluctuation of the spectrum, the first parameter value calculation unit 1033 calculates the variance of a[i] for each i (that is, the frequency) for all sub-periods included in the predetermined period. .. Then, by summing the variances calculated for each frequency, a value representing the fluctuation of the spectrum is obtained. It should be noted that instead of the variance, another statistic representing the variation such as the standard deviation may be used.

On the other hand, if the first parameter value is a value representing the variation of the spectral envelope, the first parameter value calculation unit 1033 causes the logarithm of the square of a[i] (=log(a[i] ² )) for each sub period. ) Is calculated, the calculated value is regarded as a signal, and inverse FFT (Inverse FFT) is executed to calculate the cepstrum. In the cepstrum, it is known that a low-order spectral envelope appears. Note that in the cepstrum, what corresponds to the frequency is called kefrenshi and j is its index, the cepstrum is expressed as b[j]. Therefore, the first parameter value calculation unit 1033, for all sub-periods included in the predetermined period, for each j in the low-order portion (for example, up to the 8th order (excluding the 0th order when the sampling frequency is 16000 Hz)) b[. j] is calculated. Then, by summing the variances calculated for each kefrency, a value representing the variation of the spectrum envelope is obtained. It should be noted that instead of the variance, another statistic representing the variation such as the standard deviation may be used.

Further, the second parameter value calculation unit 1034 calculates the second parameter value using the data stored in the first data storage unit 1032 and stores it in the second data storage unit 104 (step S5).

For example, the second parameter value calculation unit 1034 calculates the sum asum (=a[0]+a[1]+...+a[max]) of a[i] for each sub period, and a[0 ]/Asum, a[1]/asum, a[2]/asum,. ． ． , A[max]/asum are further calculated. Then, the information entropy H when these are regarded as the probability density is calculated. Specifically, it is expressed as follows.
H=Σ ^max _i=0 a[i]/asum*log(a[i]/asum)

In this way, the spectral entropy for each sub period can be obtained. Then, by calculating the average value of the spectral entropy of the sub period, the spectral entropy of the predetermined period is calculated. Note that instead of the average value, a median value or other statistical amount may be used.

Further, the third parameter value calculation unit 1035 calculates the third parameter value using the data stored in the first data storage unit 1032 and stores it in the second data storage unit 104 (step S7).

For example, the third parameter value calculation unit 1035 calculates the cepstrum for each sub period, as described above. In cepstrum, it is known that higher order spectral fine structure appears. Therefore, for example, the maximum value of the cepstrum in the range of the kefrenshi corresponding to the range of the screaming fundamental frequency is specified for each sub period. It should be noted that the range of keffrency is, for example, a range of orders corresponding to 70 to 600 Hz in the case of frequencies, and 27th to 229th orders at a sampling frequency of 16000 Hz. The maximum value of this cepstrum is the harmonic structure strength of each sub-period. Then, by calculating the average value of the harmonic structure strength of the sub period, the harmonic structure strength of the predetermined period is calculated. Note that instead of the average value, a median value or other statistical amount may be used.

Further, the fourth parameter value calculation unit 1036 calculates the fourth parameter value using the data stored in the first data storage unit 1032 and stores it in the second data storage unit 104 (step S9).

For example, the fourth parameter value calculation unit 1036 calculates the index cog according to the following formula.
cog＝(a[0]*0 + a[1]*1 + a[2]*2 + a[3]*3 + ・・・・+ a[max]*max)/asum

This index cog becomes the spectrum centroid of the sub-period. Therefore, the spectrum centroid of the predetermined period is calculated by calculating the average value of the spectrum centroid of the sub period. It should be noted that instead of the average value, the median value or other statistics may be used.

Although the calculation of the first to fourth parameter values has been described above, these processes may be executed in parallel, and the order of execution thereof does not matter. The calculation of the cepstrum may be executed by the preprocessing unit 1031. Further, the parameter calculation unit that has performed the calculation of the cepstrum may output the processing result to another parameter calculation unit.

Then, the determination unit 105 determines whether or not any of the first to third parameter values stored in the second data storage unit 104 meets any of the predetermined conditions (step S11). Since there is a tendency as shown in FIG. 2, based on the threshold values set for the first to third parameter values, any one of the conditions of the scream, the sound when the security gravel is stepped on, the explosion sound or the glass breaking sound is met. It is determined whether to do. The processing shifts to FIG. 4 via the terminal A.

If the scream condition is satisfied, that is, if the first parameter value is in the “low” range, the second parameter value is in the “high” range, and the third parameter value is in the “high” range (step (S13: Yes route), the determination unit 105 causes the output unit 106 to output a notification indicating a scream (step S15). The screaming notification may be a warning sound, a voice message, a display message, or an instruction to another device. Then, the process proceeds to step S31.

On the other hand, when the condition for screaming is not satisfied (step S13: No route) and the condition for explosive sound or glass breaking sound is satisfied, that is, the first parameter value falls within the “high” range, If the second parameter value is in the “high” range and the third parameter value is in the “low” range (step S17: Yes route), the determination unit 105 determines that the second data storage unit 104 stores the first data stored in the second data storage unit 104. The determination based on the four parameter values is executed (step S19). As described above, it is determined which one is based on the threshold value (generally, a value representing a range) for distinguishing the explosion sound and the glass breaking sound. If it is a glass breaking sound (step S21: Yes route), the determination unit 105 causes the output unit 106 to output a notification indicating the glass breaking sound (step S23). The notification is performed in the same manner as in step S15. Then, the process proceeds to step S31.

On the other hand, if it is not the glass breaking sound (step S21: No route), the determination unit 105 causes the output unit 106 to output the notification indicating the explosion sound (step S25). The notification is performed in the same manner as in step S15. Then, the process proceeds to step S31.

Moreover, when the conditions of the explosion sound or the glass breaking sound are not satisfied (step S17: No route), and the conditions of the sound when the security gravel is stepped on are satisfied, that is, the first parameter value is " If the second parameter value falls within the “high” range and the third parameter value falls within the “low” range (step S27: Yes route), the determination unit 105 causes the output unit 106 to A notification indicating the sound of stepping on the security gravel is output (step S29). The notification is performed in the same manner as in step S15. Then, the process proceeds to step S31.

On the other hand, when the condition of the sound when the security gravel is stepped on is not satisfied (step S27: No route), it means that the predetermined sound to be identified for crime prevention has not been detected, so the process proceeds to step S31. To do.

In step S31, for example, if the preprocessing unit 1031 does not determine that the administrator has not instructed to end the process (step S31: No route), the process returns to step S1 via the terminal B. On the other hand, if it is determined that the end of processing has been instructed, the processing ends.

By performing the processing as described above, it is possible to accurately detect the occurrence of a predetermined sound to be identified for crime prevention.

In the processing flow of FIG. 4, the judgment is made in the order of the scream, the explosion sound or the glass breaking sound, and the sound when the security gravel is stepped on, but the judgment order is not necessary. Further, if it is not required to distinguish these sounds, a notification indicating detection of a predetermined sound to be identified for crime prevention may be output when any of the conditions is satisfied.

Further, when it is not required to distinguish between the explosion sound and the glass breaking sound, the calculation of the fourth parameter value and the determination based on the fourth parameter value need not be performed.

Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, the functional block configuration shown in FIG. 1 is an example and may not match the program module configuration. Furthermore, the processing flows of FIGS. 3 and 4 are also examples, and the processing order may be exchanged or the processes may be executed in parallel as long as the processing result does not change.

Further, although an example in which the threshold value is determined for each parameter value has been shown, the range may be determined for each type of sound to be identified for crime prevention. However, the tendency as shown in FIG. 2 is retained.

Further, although it has been described above that thresholds and the like are determined by experiments and the determination is performed by the determination unit 105, for example, a combination of a sound type and the above-described three or four types of parameter values is machine-learned or another method. The determination unit 105 may be configured by learning.

The information processing apparatus 100 described above is a computer apparatus, and includes a memory, a CPU (Central Processing Unit), a hard disk drive (HDD: Hard Disk Drive), a display control unit connected to a display device, and a removable storage unit. A disk drive device, an input device, and a communication control unit for connecting to a network are connected by a bus. An operating system (OS) and application programs for executing the processing in this embodiment are stored in the HDD, and are read from the HDD to the memory when executed by the CPU. The CPU controls the display control unit, the communication control unit, and the drive device in accordance with the processing content of the application program to perform a predetermined operation. Further, the data in the process of processing is mainly stored in the memory, but may be stored in the HDD. In the embodiment of the present invention, the application program for performing the above-described processing is stored in a computer-readable removable disk, distributed, and installed in the HDD from the drive device. It may be installed in the HDD via a network such as the Internet and a communication control unit. Such a computer device realizes the various functions described above by organically cooperating the hardware such as the CPU and the memory described above with the programs such as the OS and the application programs.

The present embodiment described above can be summarized as follows.

The determination method according to the present embodiment includes (A) a first parameter value indicating the degree of variation of the spectrum of the sound related to the sound data and the whiteness degree of the sound related to the sound data, with respect to the input sound data. A calculation step of calculating a second parameter value indicating the third parameter value indicating the degree of the harmonic structure in the sound related to the sound data; (B) a first parameter value, a second parameter value, and The determination step of determining whether or not the sound data includes a predetermined sound to be identified for crime prevention based on the parameter value of 3.

By using such three kinds of parameter values as the determination index, the detection accuracy of the predetermined sound to be identified for crime prevention becomes high.

In the determination step described above, based on the first parameter value, the second parameter value, and the third parameter value, the sound data is at least a scream, a sound when the security gravel is stepped on, and a glass. It is also possible to determine which of the destruction and the explosion sound is included. The type of sound to be identified for crime prevention may be specified, or the type may not be specified.

Further, the above-described calculation step may include a step of calculating the fourth parameter value representing the main frequency of the sound related to the sound data. In this case, the determination step described above may further include a step of determining whether the sound is a glass breaking sound or an explosion sound based on the fourth parameter value.

In the case where the above-mentioned first parameter value is, for example, either a value representing a variation in the spectrum envelope of the sound related to the sound data or a value representing a variation in the spectrum of the sound related to the sound data. is there.

Further, the above-mentioned second parameter value may be the information entropy calculated, for example, by regarding the spectrum of the sound related to the sound data as the probability distribution. This is also called spectral entropy.

Further, the third parameter value described above may be the maximum value within a predetermined range in the cepstrum of the sound related to the sound data, for example. This is also called harmonic structure strength.

Furthermore, the fourth parameter value described above may be, for example, the centroid frequency of the spectrum of the sound related to the sound data. This is also called the spectral centroid.

A program for executing the above processing can be created, and the program can be read by a computer such as a flexible disk, an optical disk (CD-ROM, DVD-ROM, etc.), a magneto-optical disk, a semiconductor memory, a hard disk, etc. Stored in another storage medium or storage device. The intermediate processing result is temporarily stored in a storage device such as a main memory.

100 information processing device 102 sound data storage unit 103 calculation unit 104 second data storage unit 105 determination unit 106 output unit 1031 preprocessing unit 1032 first data storage unit 1033 first parameter value calculation unit 1034 second parameter value calculation unit 1035 3 parameter value calculation unit 1036 4th parameter value calculation unit

Claims (9)

With respect to the input sound data, a first parameter value indicating the degree of variation of the sound spectrum related to the sound data, a second parameter value indicating the whiteness degree of the sound related to the sound data, and the sound data. A calculation step of calculating a third parameter value representing the degree of the harmonic structure in the sound according to
A determination step of determining whether or not the sound data includes a predetermined sound to be identified for crime prevention, based on the first parameter value, the second parameter value, and the third parameter value;
A program for causing a computer to execute. In the determination step,
Based on the first parameter value, the second parameter value, and the third parameter value, the sound data is at least a scream, a sound when the security gravel is stepped on, and a glass break or explosion sound. The program according to claim 1, wherein it is determined whether or not to include. The calculation step is
Calculating a fourth parameter value representing a main frequency of a sound related to the sound data,
The determination step,
The program according to claim 2, further comprising a step of determining whether the sound is the glass breaking sound or the explosion sound based on the fourth parameter value. The first parameter value is one of a value representing a variation in a spectrum envelope of a sound related to the sound data and a value representing a variation in a spectrum of a sound related to the sound data. Program described in one. The program according to any one of claims 1 to 4, wherein the second parameter value is information entropy calculated by regarding a sound spectrum of the sound data as a probability distribution. The program according to claim 1, wherein the third parameter value is a maximum value within a predetermined range in a cepstrum of the sound related to the sound data. The program according to claim 3, wherein the fourth parameter value is a centroid frequency of a spectrum of a sound related to the sound data. With respect to the input sound data, a first parameter value indicating the degree of variation of the sound spectrum related to the sound data, a second parameter value indicating the whiteness degree of the sound related to the sound data, and the sound data. A calculation step of calculating a third parameter value representing the degree of the harmonic structure in the sound according to
A determination step of determining whether or not the sound data includes a predetermined sound to be identified for crime prevention, based on the first parameter value, the second parameter value, and the third parameter value;
And a computer-implemented determination method. With respect to the input sound data, a first parameter value indicating a degree of variation of a sound spectrum related to the sound data, a second parameter value indicating a whiteness degree of the sound related to the sound data, and the sound data. A third parameter value representing the degree of the harmonic structure in the sound according to
A determination unit that determines whether or not the sound data includes a predetermined sound to be identified for crime prevention, based on the first parameter value, the second parameter value, and the third parameter value;
Information processing device having a.

Images